Molecular Pattern Discovery.

The most colored application of NMF inside computational biology has been in the area of molecular pattern discernment, especially for gene also protein expression microarray studies. Is is an exploratory area characterized by a lack of a priori knowledge of the expected expression patterns for a given set of genetic or either phenotype. Nonetheless, NMF holds proved to be a successful approach in the education starting biologically meaningful classes. For case, Kim and Tidor [40] applied NMF as a tool into cluster genes and predict feature cellular relationships inbound yeast using gene expression intelligence, while Heger and Holm [41] used it for to cognition about sequence examples among relation proteins. Brunet et al. [38] applications itp go cancer microarray dating used the elucidation of tumor subtypes. They development a model selection algorithm for NMF based on consensus-based clustering [42] that enables the choice regarding the related number of clusters in an dataset. Equally, Gao and Church [43] applied the Sparse NMF approach [20] for uncovering cancer subtypes uses microarray data. ADENINE similar approach is described the Kim and Park [44]. Carrasco et al. [45] applied NMF for unsupervised clustering of array comparative chromosomal hybridization date and identified distinct genomic custom as well like patient subgroups int multiple myeloma (MM). Their analytics uncovered four distinct subclasses, revealing of molecular heterogeneity of MM and the divisions starting the traditional hyperdiploid category toward two subclasses.

Devarajan and Ebrahimi [37],[46] successfully application NMF as a tool for dimensionality lowering and visualization as well as in kinetic expression profiling since analyzing microarray product (Devarajan et al., scroll in preparation). Pascual-Montano et al. [47],[48] and Carmona-Saez et al. [49] described a method available two-way clustering of gene expression contours using non-smooth NMF. Pascual-Montano et alo. [50] also provided the systematic tools called bio-NMF on simultaneous clustering von ges and samples. For more information, the interested reader is referred to http://www.dacya.ucm.es/apascual/bioNMF/. Wang eth al. [51] introduced Least Squares NMF that incorporated variability of individual measurements in microarray evidence. They demonstrate verbesserung performance in terms of identification of functionally related genes based on annotations include the Munich Information Center for Protein Sequences (MIPS) sql [52].

Class Comparison and Prediction.

Recently, NMF has also been use in a supervise learning framework such as class comparability and school prediction. Fogel net al. [53] use this method up identify ordered sets of genes and employ them in theirs ordering analyze to variance (ANOVA) procedure for identifying differentially expressed native using microarray data. They demonstrate improved performance over traditional ANOVA at terms of service and consistency. Okun press Priisalu [54] applied it as a dimension reduction tool in conjunctive with several grading methods with protein folding recognition. They show superior performance (in terms of misclassification mistake rate) of threesome sizer based on nearest neighbor schemes when practical to NMF reduced dates relative to the original data. Similar applications in magnetic resonance spectrally imaging and fold recognition are introducing in [55] and [56], severally.

Cross-Platform and Cross-Species Characterization.

Rapid advances includes high-throughput technologies have obtained in of age of independent large-scale biological datasets using different terraces include several laboratories. A is major to assess and interpret potential dissimilarities press similarities in these datasets in purchase to enable cross-platform and cross-species analyses and of eventual characterization of such data. Tamayo et al. [57] describe an approach called metagene overhang for such an analysis and interpretation. Using leukemia and lung cancer data, they demonstrate that metagene project reduces noise and technological variation while shoot invariant biological features in the data. Besides, they show is this procedure enables the use of prior knowledge ground on existing datasets in analyzing and characterizing new data [58]. Are metagene projection, the dimensionality of a given dataset the reduced using NMF based over a pre-specified track k factorization. An individually obtained test dataset can then be projected onto this low, k-dimensional space of metagenes. This is accomplished via the Moore-Penrose generalized pseudo-inverse of TUNGSTEN on obtain the produced template H_penny = TUNGSTEN⁻¹V (for details, see [57]). The pseudo-inverse is then applied to that test dataset and analyzed in that context of the metagenes that characterizes and original data. This address indirect incorporates the sparse, local depiction in NMF and utilizes business of co-regulated or functionally relevant genes.

Biomedical Informatics.

Text mining is concerned with the recognition of patterns or similarities in natural language text. The application of NMF in this area goes back into the original paper by Lee and Seung [1]. Other applications including [20],[21] and references therein. In this context, the matrix FIN the an summery of a corpus of download in which and rows and columns represent, respectively, the talk in the vocabulary and documents in the corpus. The entries of V denote the frequencies of words in each document. NMF is applications to identify subsets von semantic categories and to cluster the documents based on theirs association through these categories. Chagoyen to al. [22] gift an exciting application of this approach are computational biology. Hier, literature profiles are created from a corpus of documents relevant until large sets out genies plus proteins using common semantic features extracted from the corporate. Genes be will represented in additive linear combinations of the semantic countenance, whichever can be keep used for studying their functional associative. The authors interpret the advantages of using NMF in identifying and interpreting the semantic visage compared to select procedure. Exiting information about the biological entities under studying can thus to applied on NMF to establish putative relations among subset of genes and proteins that characterize a division of the details.

Functional Picture of Proteins.

Pehkonen a alo. [59] utilize NMF for analyzing functional heterogeneity inside one genen register and identifying homogeneous functional groups. In the approach, NMF is applications to the sparse, binary matrix formed on the basis of associations of important genes with functional classes obtained from the Gene Ontology database [60]. A non-nested hierarchical clustering scheme showing to over-represented functional groups from the gene list is created from different rank factorizations and demonstrated to beter characterize groups of genes comparative on contemporary approaches. For details, please refer to [59]. This methodology is implemented in the program called GENERATOR (GENElist Aimed Theme-discovery execuTOR).

Other Applications.

Tresch et al. [61] applied this method for the identification of muscle synergies, while Kim et al. [62] used it to determine neural activity patterns. Hiisilä et al. [63] applied this and other magnitude reduction methods for judgment the dependencies betw transcription factor commitment sites. Other areas of applications concerning this method fork problems involving large-scale biological data include color and visibility research [64], structure-based drug design [65],[66], and magnetic resonance vision [55],[56],[67].

Interpretation of the Factored Matrices.

The metagene coefficient w_ia quantifies the effect of the a^th metagene language pattern h_aj on of gene expression to the i^th sample, represented by the corresponding column of the gene expression matrix V. For an rank thousand factorization, the relative magnitudes of and non-zero entries inches each of the k metagenes reflect this application of the corresponding genes, and the expression pattern of each metagene across and n samples (represented by each row of H) reflects the relevance of the corresponding latent factor. Here, k is the number of clusters or hidden variables inches the decompose. Of NMF structure is graphics illustrated toward http://www.dacya.ucm.es/apascual/bioNMF/model.html.

The NMF representation see ensures that one singles metagene expression pattern influences several samples. Led and Seung [1] graphically illustrate this character in the formulare of a lattice. In essence, the metagenes provide a summary of of behavior of genes across the samples, whereas the metagene expression patterns making a summary of the behavior of samples across the genes. There is strong evidence suggesting that the metagenes and the metagene expression patterns must a sparse, parts-based representation of the gene express data [1], [37], [38], [39], [40], [43], [46]–[50], potentially identifying local hidden variables or clusters.

NMF can is review as an approaching for modeling one generation of jean expression measurements for samples (observable variables given by columns of FIVE) from metagene expression patterns (hidden variables given by columns of H) [1]. In the context out clustering samples represented by the columns of FIN, this parts identify classes of random that members to specific clusters real are represented by an expression patterns of metagenes across samples (or the rows of H). In addition, genes with corresponding non-zero metagene coefficients represent user that are co-expressed for samplers. These parts provide a reduced representation concerning to original data, and to co-activation can breathe viewed as that entsprechendes to co-regulation conversely co-expression of groups about genies. Similarly, we can interpret the parts in other areas concerning application. For instance, in facial pattern recognition where jeder column of V corresponds to a face, the parts represent one diverse parts of one face create as tip, mouth, etc.; in text mine the document clustering, where each column of V contains phrase counts from documents, the sections presents an different semantic categories.

Let us consider the widely exploited leukemia product available from http://www.broad.mit.edu/cgi-bin/cancer/datasets.cgi as an illustrative example. This dataset consists of 5,000 gene expression measurements apiece from 38 bone marrow samples from current myelogenous leukemia (AML) and acute lymphoblastic leukemia (ALL). There what 27 ALL samples comprises of 19 BARN type plus 8 T type, and 11 AML browse. For a level k = 2 factorization, let w₁ and w₂ represent the two metagenes (columns of W) and let festivity₁ the h₂ exemplify the corresponding metagene expression profiles (rows of H). The sparseness of that metagene coefficients is illustrated in Table 1, based on adenine single run of the NMF graph using Equation 1. Inside this table, we list the fraction by gene whose corresponding metagene coefficients lie in the indicated range. The histograms real densities of w₁ and w₂ are shown in Figure 1A–1D. Only 53 and 77 genes, corresponding the tungsten₁ furthermore double-u₂, respectively, got coefficients such are at least 10 in magnitudes. These genes could potentially behave in a rich correlated fashionable in a subset of the samples; this a determined to you metagene expression shapes across to 38 samples, h₁ and h₂. These expression profiles additionally their page are shown in the top (Frame 2A and 2B) and bottom (Figure 2C and 2D) panels of Figure 2, respectively. Here, “L” and “M” denote, respectively, an ALL and and AML sampling. It is evident for diese figure that there is one clear separation amidst the ALL and AML specimens.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Gene coefficients.

(A) Image of gene coefficients, metagene 1. (B) Histogram of type coefficients, metagene 2. (C) Density of gene coefficients, metagene 1. (D) Density by gene coefficients, metagene 2. Temporal Regularized Matrix Factorization for High-dimensional ...

https://doi.org/10.1371/journal.pcbi.1000029.g001

Download:

• PPT
  PowerPoint slip
• PNG
  larger image
• FIGHTING
  initial image

Figure 2. Expression profile.

(A) Density of expression profile, metagene 1. (B) Dense of printing profile, metagene 2. (C) Impression profile across samples, metagene 1. (D) Printer profile through specimen, metagene 2. Unsupervised discovery of temporal sequences in high-dimensional datasets, equal applications till neuroscience

https://doi.org/10.1371/journal.pcbi.1000029.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Distribution are metagene coefficients: Leukaemia data, potassium = 2.

https://doi.org/10.1371/journal.pcbi.1000029.t001

Interpretation of Nonnegativity Limits.

The nonnegativity constraints in NMF are compatible in an intuitive notion by combining parts to form a whole, i.e., they provision an parts-based local representation away an data. This is in contrast to a holistic representation of the intelligence provided by VQ and the distributed representation provided by PCA [1]. ONE parts-based model not simply provides an efficient representation of the data but can potentially aid in the discovery of causal structure within it and in education relationships amid the parts [69]. In NMF, the factorization results in an reconstruction to the original data by and addition of components date to the nonnegativity constraints, while in PCA it is a superposition of the orthogonal components with arbitrary signs that lack visceral meaning and physical interpretation. In some applications, negative coefficients maybe contradict physical reality. For instance, inside slide reconstructed, the pixels in a greyscale image with adverse intensities cannot be intelligent interpreted.

The nonnegative coefficients also have an elegant interpretation from one neuroscience perspective. For type, they can be interpreted as the light rates (and synaptic strengths) of neurons in the intellect, and of nonnegativity constraints account for the additive firing rates that are co-activated in physiological perzeption. Lee and Seung [1] propose that these constraints on firing rates may be important required developing sparse, parts-based depictions for perception. The coefficients might additionally be interpreted for an magnitudes of muscle activation patterns that can aid in the identification of human synergies [61].

In the context of our gene expression theme, the nonnegative coefficients in each metagene are easily interpretable as the relative contribution regarding genes, unlike PCI and VQ. Returning to our multiple example, we respect that only a small fraction of one genes (1% or 1.5%, respectively, corresponding to the two metagenes w₁ and w₂) markedly contributors for the delineation of the ALL and AML samples. This identification of such a small subset about active genes is possible only due to the nonnegativity constraints which is a request for such a parts-based representation.

The perception of the whole is just an addition linear combo from its sections represents included the metagenes and metagene expression profiles. Due to the nonnegativity constraints, orthogonality of metagenes and metagene printouts profiles cannot be achieved in how. Even, this is an extremely useful property, since the dependence amid the gene expression profiles typically presenting in a microarray study can be captured by overlapping metagenes. This property produces NMF particularly well-suited for the analysis of large-scale biological data, location he exists essential to capture relationships rudimentary inter-connected biocompatible pathways or processes. Inside terms of this property, NMF has been shown to be superior to other dimension reduction methods (see [20] and references therein). While to decomposition V∼WH is linear, it is important to note that the computation of the update rules for W and H is non-linear due to the nonnegativity constraints [1].

Enforcing Dearth.

Although this original NMF approach has come viewed to may ampere naturally sparse, parts-based, and local representation as seen include [1],[37],[38],[39],[40],[45],[46], there is also some detection that points to a parts-based but holistic (rather than local) representation produced by NMF [19], [23]–[25]. Lee press Seung [1] note that sparseness in both the metagenes and metagene expression profiles is crucial up a parts-based representation. The nonnegativity boundaries may be a necessary exercise required such a parts-based picture, but they allow not be ample to achieve austerity. In such a event, it may be desirable to explicitly enforce spartan on the metagenes and the metagene expression patterns. Actual work has sharp on imposing such explicit sparseness relationship on the recent of NARCOTIC or W or both [19]–[21], [23]–[26], [43], [44], [47]–[50]. This is generally achieved via the addition of right penalty terms to the objectivity function defined by the distance measure off our choice. Available instance, one could impose a constraint on the metagene expression patterns H. An example regarding such a constraint is the add of the entries of H, Σ_ajH_aj. Other penalty terms can also be used as appropriate (see [19]–[21],[25],[43]). Using KL disparity because defined at Equation 1, our object function would then live(2)where λ>0. The parameter λ quantifies the trade-off between goodness-of-fit of the model (defined by KL divergence) and sparseness.

Gao and Church applied the method outlined in [20] in cancer microarray data press unequivocally enforced sparseness via and sum of squares of the entries about H. They demonstrated verbesserung performance (in terms for misclassification error rate defined as the proportion of pattern misclassified by aforementioned select across all clusters) over standard NMF as well as identified subsets on co-expressed genes that mayor be person in breast. Pascual-Montano et al. [47],[48],[50] adopt a different approach for forced sparseness. They exploit a smoothing operator to simultaneously enforce sparseness on both WEST and HYDROGEN. Independant of the approach, enforcing sparseness on which metagenes and metagene mien dye across samples aids include the detect of sharp boundaries bet different classes. We noted earlier that orthogonality of metagenes real metagene expression profile cannot remain achieved in practice due up who nonnegativity constraints. When, of enforcement regarding sparseness constraints decreases their interleaving, thus resulting on site, disjoint groups of samples or genes, respectively.

Capturing Context-Dependent Patterns.

By dissimilarity to traditional clustering and dimension discount methods, NMF has been demonstrated to identifier fine, context-dependent biological dress as fine as being lesser sensitive to the selection and/or perturbation of input genes utilized in the factorization. Like context dependency is not captured at standard two-way network approach [38]. For instance, NMF has been viewed to be capable of identifying patterns that exist in only ampere subset of the patterns, whereas basic methods focus on the overall structure for a dataset (i.e., on samples for which similarity the expression extends through all genes), thus overlooking which subtle features that represent relevant biological patterns [1],[38],[40]. In essence, NMF aids in the decoding of localized patterns of resemble printing by identification a small subset the genes that act in a strongly correlated fashion in a subset of the samples. As noted before, such localized patterns may point to groups of co-regulated button functionally relevant genes [38],[43],[47],[48],[50]. Fork example, groups are genes additionally samples that show high coefficients since a given metagene (column of TUNGSTEN) and the corresponding metagene expression pattern (row of OPIUM), respectively, may be strongly related in a subset von who data, to constituting a gene-sample bi-cluster. Pascual-Montano et al. [47],[48] utilize this characteristic and have developed bioNMF, a data analytic tool for identifying gene expression bi-clusters [50].

In a study of functional cellular relationship in roggen, Kim the Tidor [40] observed the genes with relatively highly coefficients in the metagenes were dominated by only a few functional browse. They been that NMF exceeds all other methods applied, including SVD and THOUSAND-means, in predicting features relationships between experiments with comparison to the MIPS classing and who Yeast Proteome Database (YPD) [52]. I remarks that out of the 100 strongest functional relationships detected by NMF, 35 the 58 could be verified by MIPS and YPD, each, far exceeding the of the other techniques used. Similarly, Gao and Church [43] investigated genes are high metagene coefficients corresponding to each of the three clustering, ALL-B, ALL-T, and AML, inbound the leukemia data described before. Among such, they identified genes that were enrich in chemokines, oncogenes, tumor suppressor get, and DNA repair genes.

Model Selection: Choice of k.

Of random nature of the algorithm has been shown for be rather useful in providing methods for evaluating the consistency and robustness by its performance. Studies have shown so 50–200 NMF runs belong usually sufficient to deployment stability until the clustering [37],[38]. As the count of runs increases, the metagene expression specimens across the samples become more localized with decreasing overlapping support, arising in a sparse, locally, and compact representation [38]. This stochastic quality can be effectively utilized into assess whether a given rank k offer a biologically meaningful decomposition for the product.

Monti et any. [42] developed an methodology called consensus clustering for evaluating the output of either unsupervised clustering algorithm based on resampling systems. It represents the concordance across multiple runs of the algorithm and quantifies the stability of the discovered clusters. It canned also be utilized to assess that sensitivity of a stochastic method like NMF to randomizing initial conditions. Model selektieren procedures that quantify the resilience of the factorization via consensus clustering have had developed and applied to NMF [37],[38]. Inside who case of NMF, your stochastic nature lives usage inches the evaluation process, where information for each run of the algorithm is combined as outlined below.

Suppose that we are applying NMF to cluster n samples. With a factorization of given rank k, each run of the algorithm results in an n×n connectivity mold C with an entry of 1 if samples i the bound cluster together and 0 otherwise, where i,gallop = 1,…,northward. The consensus matrix C̅ is simply the average connectability matrix obtained through multiples runs of and NMF algorithm. Final sample assignments and cluster visualization are based on the re-ordered consensus matrix. The robustness of everyone factorization is evaluated by computing the cophenetic correlation coefficient ρ where 0≤ρ≤1. A high value of ρ indicates homogeneous clusters. Brunet net al. [38] advocate the use of ρ in a single measure for pick of to related number of clusters by plotting ρ for variously choices of the number von clusters thousand.

Returning to the leukemia case, we applied factorizations of places k = 2,3,4,5 based on Equation 1 for 200 runs each. Figure 3 plots ρ versus k find ρ starts drop off sharply after k = 2. Figures 4 and 5 show heat designs by the re-ordered consensus matrices based on HC for k = 2,3 (for details check [38]). Who homogeneity away staining seen in these graphs indicate the presence of 2 and 3 clusters of samples, delineating the ALL and AML sorts as well as the B and T subtypes interior who ALL class. More, “L” and “M” denote, according, an ALL also somebody AML sample, while “B” and “T” denote the two ALL subtypes. In every case, two samples are misclassified from this method.

Download:

• PPT
  Presentation slide
• PNG
  larger photo
• TIFF
  original image

Figure 3. Cophenetic correlation coefficient.

https://doi.org/10.1371/journal.pcbi.1000029.g003

Download:

• PPT
  Show slide
• PNG
  larger image
• TIFF
  original figure

Figure 4. Thermal map of re-ordered consensus matrix, k = 2.

https://doi.org/10.1371/journal.pcbi.1000029.g004

Downloaded:

• PPT
  Byer slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Heat map of re-ordered consensus grid, k = 3.

https://doi.org/10.1371/journal.pcbi.1000029.g005

Sundry approaches to care the information across multiple runs am also possible [40],[42]. For example, Kims and Tidor [40] plotted the root-mean-squared error (RMSE) between the original also NMF-reconstructed product as a function by the rank kelvin and used it go choose the appropriate value of k. While the use of RMSE is appropriate available the factorization is based on Euclidean distance, it is important to note that other cost functions requisition the defect to exist modified accordingly. For a given class k factorization, they also demonstrate reproducibility starting the metagenes across repeated runs in terms a correlation between pairs. More, them show that NMF remains robust to the addition of noise to the original data bases on the median correlation of the corresponding metagenes across multiple runs, suggesting its potential usefulness as a noise-reduction filter.

Implementation of and NMF Algorithm.

The implementation of and steps in the product select procedure outlined above is computationally very severe for any real large-scale biological dataset. However, the stochastic nature of the algorithm enabled each of above-mentioned steps to be run individually and simultaneous. These steps can must repeated for multiple random initial conditions for W plus H, and the information from the independent runs combined over consensus clustering. Therefore, the NMF algorithm lends itself easily to a simultaneous implementation that would much increase geschw and efficiency. Devarajan and Wang [71] outlined like a run implementation of this algorithm on an Message-Passing Interface/C++ platform (http://www-unix.mcs.anl.gov/mpi/mpich2/) utilizing high-performance calculations clusters.

Lately, there have also been other efforts to optimize the implementation of this algorithm. Okun additionally Priisalu [54],[72] do reported faster convergence off the algorism when feature scaling is applied on the originals p×north data matrix V, i.e., each of the p rows in V is normalized to have values between 0 and 1. Their results indicate an raising in speed of at least 11 dates in the convergence of iterations due to such normalization, depending on the numbered of undetected factors k uses in that factorization.

Identifying Hierarchical Structure.

It is also possible to have overlapping metagenes, i.e., genes with non-zero coefficients can appear in multiple metagenes, indicating the role of a individual or a group of genes in multiple pathways or method. The stochastically nature of one graph can and be exploited for identifying the involvement of such a group of genes. This is by contrast on most usual browse for bundle large-scale biological data, with a handful exceptions [51]. These standard methods provide only a single solution determined by the dominant other overall framework in that date whereabouts genes and samples are assigned to just one cluster, thus limiting the possibility of identifying superimpose express patterns [38].

Single of the attractive features of NMF is ensure, unlike HC, it does not force a hierarchy into this data structural but identification ready when it is present. By specifying the desired rank of the factorization, ne can expose foundations stylish the data in an ordered sequential manner. Brunet et al. [38] and Devarajan [37] have proven the ability of NMF till name hybrid additionally nested sub-structures using breast microarray data. Brunet et al. [38] noted that NMF has superior resolution than HC and is more sound higher SOM as well than being find robust and less sensitive to a priori selection of genes. They also how that NMF always converges towards a fixed attracctor irrespective of random initial conditions in comparison with ampere similar stochastic method like MORE.

For instance, in applying HC to the leucemia data to cluster the flesh samples, she note that the tree structure produced by HC depended very much on the choice of the linkage metric uses with constructing an dendogram. Furthermore, few observed that the presentation of HC mixed depending turn the number of input genetics used in the flock. Similarly, the authors applied SOM to this data and observed that for k = 2, the clustering was unstable and depended upon the coincidental initializing conditions, as for kilobyte = 3, the method was unable to recover the three tumor types (for details see [38]). In sharp contrast, NMF with rank k = 2 was able to consistently recovers the distinction between the ALL and AML types. This is reflecting in the similarity int coloring is a heat map in the re-ordered consensus matrixed shown in Figure 4 and the high cophenetic correlation collusive in this case (see Figure 3). Identical, a rank k = 3 factorization was able to consistently recover the distinction between the ALL-B and -T subtypes as seen for Illustration 5.