libscan

 

Function

Scans a database with a set of models that represents a family

Description

Scans each signature, profile or HMM in a directory against a sequence database and writes a signature hits file for each one. Or scans sequences against such a library of discriminating elements and writes a library scan file for each one.

This program is part of a suite of EMBOSS applications that directly or indirectly make use of the protein structure databases pdb and scop. This program is part of an experimental analysis pipeline described in an accompanying document. We provide the software in the hope that it will be useful. The applications were designed for specific research purposes and may not be useful or reliable in contexts other than the described pipeline. The development of the suite was coordinated by Jon Ison to whom enquiries and bug reports should be sent (email jison@hgmp.mrc.ac.uk).

The advantage of screening a relatively small library of discriminating elements with a sequence is that it is sufficient for the sequence to detect its true family (discriminator) in the first rank for an effective prediction. This is in contrast to searching a larger sequence database to identify homology, where biologically significant hits may achieve statistically insignificant scores and therefore be missed. A library might help the detection of such proteins because they may still score their true discriminator higher than the others in the library regardless of statistical estimates. Further improvements to predictions are gained when multiple sources of evidence, in this case the different types of discriminating element, are considered.

The libscan application allows a protein sequence or sequences to be screened against the library of discriminating elements. The results of a screen are returned to the user in a library scan file containing a list of top-scoring SCOP domains rank-ordered by p-value for each individual type of discriminator, and also for all of the discriminator types in combination (combined prediction). For the combined prediction, the p-value is derived from an empirically derived distribution of the product of the p-values of the individual methods.

libscan runs in one of two modes either (i) database search mode or (ii) library screen mode. In database search mode libscan reads one or more directories each containing a single type of discriminating element, the permitted types are sparse sequence signature, Gribskov profile, Henikoff profile or hidden Markov model. Each directory must contain a discriminating element for each of one or more scop families; files for indivdual families should have the same name (possibly with different file extensions) in the different directories, e.g. the scop family with the identifier 46445 might be represented by files 46445.hmm, 46445.sig, 46445.gribs and 46445.henik in their respective directories. libscan scans each discriminating element against a sequence set that must also be provided by the user and generates a signature hits file (of scored hits to sequences in the set) for each individual discriminating element. In mode (i), the user must also provide a scop validation file so that a classification of hits can be given in the signature hits files. The user specifies a maximum number of false hits that are written to the signature hits files.

In library screen mode, libscan reads a sequence set, screens each sequence against the library (directories of discriminating elements) and writes a library scan file (of top-scoring families) for each sequence. In either mode, a scop classification file is required as a source of family classification data. The paths and extensions of the sequence set and scop classification files (input) different types of discriminating elements (input), and paths and extension of signature hits files or library scan file (output) are specified by the user.

Algorithm

The approach for calculating p-values for individual discriminator-sequence matches is the same irrespective of which mode libscan is run in and is summarised as follows: (i) Each discriminating element is scanned (scored) against each sequence. (ii) The ditribution of scores is generated for each different type of element. (iii) Using these distributions, a p-value is calculated for each discriminator-sequence match. The approach for calculating the 'combined p-value' for each individual sequence is as follows: (i) p-values of the matches for the different types of discriminator versus a sequence are multiplied to yield the 'product of p-values'. (ii) A distribution of 'product of p-values' is generated for each individual sequence. (iii) Using these distributions, a so-called 'combined p-value', reflecting the overall significance of the matches of the sequence to the different types of discriminators, is generated. Importantly, two (or more) p-values are only ever multiplied together if corresponding hit is to the same region of the protein. 'Same' is defined as two hits overlapping by a user-defined number of residues.

Usage

Command line arguments

   Standard (Mandatory) qualifiers (* if not always prompted):
   -mode               menu       Libscan runs in one of two modes either (i)
                                  database search mode or (ii) library screen
                                  mode. In database search mode libscan reads
                                  one or more directories each containing a
                                  single type of discriminating element, the
                                  permitted types are sparse sequence
                                  signature, Gribskov profile, Henikoff
                                  profile or hidden Markov model. In library
                                  screen mode, libscan reads a sequence set,
                                  screens each sequence against the library
                                  (directories of discriminating elements) and
                                  writes a library scan file (of top-scoring
                                  families) for each one.
   -db                 seqset     In database search mode libscan scans each
                                  discriminating element against a sequence
                                  set. In library screen mode, libscan reads a
                                  sequence set and screens each sequence
                                  against the library (directories of
                                  disciminating elements)
   -[no]grib           boolean    Use Gribskov profile
   -[no]henik          boolean    Use Henikoff profile
   -[no]hmm            boolean    Use HMM profile
   -[no]sig            boolean    Use signature
*  -hmmpath            string     Location of HMM profile files (input)
*  -hmmextn            string     Extension of HMM profile files (input)
*  -hmmoutpath         string     Location of signature hits/library scan
                                  files for hmm searches (output)
*  -hmmoutextn         string     Extension of signature hits/library scan
                                  files for hmm searches (output)
*  -gbvpath            string     Location of Gribskov profile files (input)
*  -gbvextn            string     Extension of Gribskov profile files (input)
*  -gbvgapo            float      Gap insertion penalty
*  -gbvgape            float      Gap extension penalty
*  -gbvoutpath         string     Location of signature hits/library scan
                                  files for Gribskov profile searches (output)
*  -gbvoutextn         string     Extension of signature hits/library scan
                                  files for Gribskov profile searches (output)
*  -hnfpath            string     Location of Hennikoff profile files (input)
*  -hnfextn            string     Extension of Hennikoff profile files (input)
*  -hnfgapo            float      Gap insertion penalty
*  -hnfgape            float      Gap extension penalty
*  -hnfoutpath         string     Location of signature hits/library scan
                                  files for Henikoff profile searches (output)
*  -hnfoutextn         string     Extension of signature hits/library scan
                                  files for Henikoff profile searches (output)
*  -sigpath            string     Location of signature files (input)
*  -sigextn            string     Extension of signature files (input)
*  -nterm              menu       Select number
*  -sub                matrixf    Residue substitution matrix
*  -siggapo            float      Gap insertion penalty
*  -siggape            float      Gap extension penalty
*  -sigoutpath         string     Location of signature hits/library scan
                                  files for signature searches (output)
*  -sigoutextn         string     Extension for signature hits/library scan
                                  files for signature searches (output)
   -scopf              infile     In either mode, a 'scop classification file'
                                  is required as a source of family
                                  classification data. A scop classification
                                  file contains classification and other data
                                  for domains from the scop database. The file
                                  is in embl-like format and is generated by
                                  scopparse. Domain sequence information can
                                  be added to the file by using scopseqs.
*  -targetf            infile     A 'scop validation file' contains sequence
                                  relatives (hits) for each of a number of
                                  different scop families, superfamilies and
                                  folds. The file contains the collated
                                  results from psiblast searches of a sequence
                                  database for the indvidual scop families;
                                  hits of unambiguous family assignment are
                                  listed under their respective family,
                                  otherwise a hit is assigned as relatives to
                                  a scop superfamily or fold instead. The scop
                                  validation file is generated by seqnr and
                                  is in embl-like format.
*  -maxhits            integer    The maximum number of hits to be written to
                                  the signature hits file.
*  -overlap            integer    When estimating the statistical significance
                                  of scored matches of different types of
                                  discriminating element to a sequence, two
                                  (or more) p-values are only ever multiplied
                                  together if corresponding hit is to the same
                                  region of the protein. 'Same' is defined as
                                  two hits overlapping by a user-defined
                                  number of residues.
   -mrgoutpath         string     Location of signature hits / library scan
                                  files for merged results (output)
   -mrgoutextn         string     Extension for signature hits / library scan
                                  files for merged results (output)

   Additional (Optional) qualifiers: (none)
   Advanced (Unprompted) qualifiers: (none)
   Associated qualifiers:

   "-db" associated qualifiers
   -sbegin              integer    Start of each sequence to be used
   -send                integer    End of each sequence to be used
   -sreverse            boolean    Reverse (if DNA)
   -sask                boolean    Ask for begin/end/reverse
   -snucleotide         boolean    Sequence is nucleotide
   -sprotein            boolean    Sequence is protein
   -slower              boolean    Make lower case
   -supper              boolean    Make upper case
   -sformat             string     Input sequence format
   -sdbname             string     Database name
   -sid                 string     Entryname
   -ufo                 string     UFO features
   -fformat             string     Features format
   -fopenfile           string     Features file name

   General qualifiers:
   -auto                boolean    Turn off prompts
   -stdout              boolean    Write standard output
   -filter              boolean    Read standard input, write standard output
   -options             boolean    Prompt for standard and additional values
   -debug               boolean    Write debug output to program.dbg
   -verbose             boolean    Report some/full command line options
   -help                boolean    Report command line options. More
                                  information on associated and general
                                  qualifiers can be found with -help -verbose
   -warning             boolean    Report warnings
   -error               boolean    Report errors
   -fatal               boolean    Report fatal errors
   -die                 boolean    Report deaths


Standard (Mandatory) qualifiers Allowed values Default
-mode Libscan runs in one of two modes either (i) database search mode or (ii) library screen mode. In database search mode libscan reads one or more directories each containing a single type of discriminating element, the permitted types are sparse sequence signature, Gribskov profile, Henikoff profile or hidden Markov model. In library screen mode, libscan reads a sequence set, screens each sequence against the library (directories of discriminating elements) and writes a library scan file (of top-scoring families) for each one.
1 (Database search mode)
2 (Library screen mode)
1
-db In database search mode libscan scans each discriminating element against a sequence set. In library screen mode, libscan reads a sequence set and screens each sequence against the library (directories of disciminating elements) Readable set of sequences Required
-[no]grib Use Gribskov profile Boolean value Yes/No Yes
-[no]henik Use Henikoff profile Boolean value Yes/No Yes
-[no]hmm Use HMM profile Boolean value Yes/No Yes
-[no]sig Use signature Boolean value Yes/No Yes
-hmmpath Location of HMM profile files (input) Any string is accepted ./
-hmmextn Extension of HMM profile files (input) Any string is accepted .hmm
-hmmoutpath Location of signature hits/library scan files for hmm searches (output) Any string is accepted ./
-hmmoutextn Extension of signature hits/library scan files for hmm searches (output) Any string is accepted .hmmout
-gbvpath Location of Gribskov profile files (input) Any string is accepted ./
-gbvextn Extension of Gribskov profile files (input) Any string is accepted .gribs
-gbvgapo Gap insertion penalty Any numeric value 1.0
-gbvgape Gap extension penalty Any numeric value 1.0
-gbvoutpath Location of signature hits/library scan files for Gribskov profile searches (output) Any string is accepted ./
-gbvoutextn Extension of signature hits/library scan files for Gribskov profile searches (output) Any string is accepted .gribout
-hnfpath Location of Hennikoff profile files (input) Any string is accepted ./
-hnfextn Extension of Hennikoff profile files (input) Any string is accepted .henik
-hnfgapo Gap insertion penalty Any numeric value 1.0
-hnfgape Gap extension penalty Any numeric value 1.0
-hnfoutpath Location of signature hits/library scan files for Henikoff profile searches (output) Any string is accepted ./
-hnfoutextn Extension of signature hits/library scan files for Henikoff profile searches (output) Any string is accepted .henikout
-sigpath Location of signature files (input) Any string is accepted ./
-sigextn Extension of signature files (input) Any string is accepted .sig
-nterm Select number
1 (Align anywhere and allow only complete signature-sequence fit)
2 (Align anywhere and allow partial signature-sequence fit)
3 (Use empirical gaps only)
1
-sub Residue substitution matrix Comparison matrix file in EMBOSS data path EBLOSUM62
-siggapo Gap insertion penalty Any numeric value 10.0
-siggape Gap extension penalty Any numeric value 0.5
-sigoutpath Location of signature hits/library scan files for signature searches (output) Any string is accepted ./
-sigoutextn Extension for signature hits/library scan files for signature searches (output) Any string is accepted .sigout
-scopf In either mode, a 'scop classification file' is required as a source of family classification data. A scop classification file contains classification and other data for domains from the scop database. The file is in embl-like format and is generated by scopparse. Domain sequence information can be added to the file by using scopseqs. Input file Required
-targetf A 'scop validation file' contains sequence relatives (hits) for each of a number of different scop families, superfamilies and folds. The file contains the collated results from psiblast searches of a sequence database for the indvidual scop families; hits of unambiguous family assignment are listed under their respective family, otherwise a hit is assigned as relatives to a scop superfamily or fold instead. The scop validation file is generated by seqnr and is in embl-like format. Input file Required
-maxhits The maximum number of hits to be written to the signature hits file. Any integer value 50
-overlap When estimating the statistical significance of scored matches of different types of discriminating element to a sequence, two (or more) p-values are only ever multiplied together if corresponding hit is to the same region of the protein. 'Same' is defined as two hits overlapping by a user-defined number of residues. Any integer value 10
-mrgoutpath Location of signature hits / library scan files for merged results (output) Any string is accepted ./
-mrgoutextn Extension for signature hits / library scan files for merged results (output) Any string is accepted .mrgout
Additional (Optional) qualifiers Allowed values Default
(none)
Advanced (Unprompted) qualifiers Allowed values Default
(none)

Input file format

libscan reads any normal sequence USAs.

The format of the scop classification file is described in scopparse

The format of the signature files is explained in siggen

The format of the profile and hmm files is explained in documentation for the propechy application and the hmmer package respectively.

Output file format

The format of the signature hits file is described in sigscan

Data files

EMBOSS data files are distributed with the application and stored in the standard EMBOSS data directory, which is defined by the EMBOSS environment variable EMBOSS_DATA.

To see the available EMBOSS data files, run:

% embossdata -showall

To fetch one of the data files (for example 'Exxx.dat') into your current directory for you to inspect or modify, run:


% embossdata -fetch -file Exxx.dat

Users can provide their own data files in their own directories. Project specific files can be put in the current directory, or for tidier directory listings in a subdirectory called ".embossdata". Files for all EMBOSS runs can be put in the user's home directory, or again in a subdirectory called ".embossdata".

The directories are searched in the following order:

libscan requires a residue substitution matrix.

Notes

None.

References

None.

Warnings

The score distributions and therefore p-values are calculated only from the data that libscan is provided. Therefore, meaningful p-values are only guaranteed if the sequence set (database search mode) or number of families (library screeen mode) is sufficiently large.

Diagnostic Error Messages

None.

Exit status

It always exits with status 0.

Known bugs

None.

See also

Program nameDescription
contactsReads coordinate files and writes files of intra-chain residue-residue contact data
fraggleRemoves fragment sequences from files of hits for scop families
hmmgenGenerates a hidden Markov model for each alignment in a directory by using the HMMER package
interfaceReads coordinate files and writes files of inter-chain residue-residue contact data
profgenGenerates various profiles for each alignment in a directory
rocplotProvides interpretation and graphical display of the performance of discriminating elements (e.g. profiles for protein families). rocplot reads file(s) of hits from discriminator-database search(es), performs ROC analysis on the hits, and writes graphs illustrating the diagnostic performance of the discriminating elements
scopalignGenerate alignments for families in a scop classification file by using STAMP
scoprepReorder scop classificaiton file so that the representative structure of each family is given first
scopresoRemoves low resolution domains from a scop classification file
seqalignGenerate extended alignments for families in a scop families file by using CLUSTALW with seed alignments
seqsearchGenerate files of hits for families in a scop classification file by using PSI-BLAST with seed alignments
seqsortReads multiple files of hits and writes (i) a scop families file and (ii) a scop ambiguities file
seqwordsGenerate file of hits for scop families by searching swissprot with keywords
siggenGenerates a sparse protein signature from an alignment and residue contact data
sigplotGenerates data files of signature performance
sigscanScans a signature against swissprot and writes a signature hits file

Hidden Markov models can be generated for scop families by using hmmgen which uses the HMMER package.

Simple frequency matrices, Gribskov profiles and Hennikoff profiles can be generated for scop families by using profgen.

A 'signature file' contains a sparse sequence signature. The files are generated by siggen.

A 'signature hits file' contains the results of a search of a discriminating element (one of protein signature, hidden Markov model, simple frequency matrix, Gribskov profile or Hennikoff profile) against a sequence database. The files are generated by sigscan and modelscan.

A 'library scan file' holds the results of a search of a sequence against a library of discriminating elements for scop families (one directory for each type of element, e.g. sparse sequence signatures, profiles or HMMs).

The library scan file contains, for each invividual type of discriminating element and for the combination of the elements, classification and other data for each of a user-defined number of top-scoring families (discriminators) from the scop database.

A 'scop classification file' contains classification and other data for domains from the scop database. The file is in embl-like format and is generated by scopparse. Domain sequence information can be added to the file by using scopseqs.

A 'scop validation file' contains sequence relatives (hits) for each of a number of different scop families, superfamilies and folds. The file contains the collated results from psiblast searches of a sequence database for the indvidual scop families; hits of unambiguous family assignment are listed under their respective family, otherwise a hit is assigned as relatives to a scop superfamily or fold instead. The scop validation file is generated by seqnr and is in embl-like format.

Author(s)

Jon Ison (jison © rfcgr.mrc.ac.uk)
MRC Rosalind Franklin Centre for Genomics Research Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SB, UK

Ranjeeva Ranasinghe (rranasin © hgmp.mrc.ac.uk)
HGMP-RC, Genome Campus, Hinxton, Cambridge CB10 1SB, UK

History

Written (2003) - Jon Ison & Ranjeeva Ranasinghe

Target users

This program is intended to be used by everyone and everything, from naive users to embedded scripts.