Multivariate analysis
Statement
Random matrix theory was introduced into theoretical physics by Wigner in the 1950s in his study of neutron scattering resonances, but as a subject, RMT goes back to the work of statisticians at the beginning of the 20th century. Recently, advances in RMT have opened the way to the statistical analysis of data sets in cases where the number of variables is comparable to the number of samples, and both are large. One might, for example, be interested in the daily temperature in hundreds of cities around the world, over a 365 day time period. At the technical level, one considers the statistics of the singular values of (appropriately centered and scaled) matrices , where . Here is the number of variables and is the sample size. More precisely, one centers the 's around their sample averages, and considers the eigenvalues and associated eigenvectors of the sample matrix . The 's and 's are known as the principal component eigenvalues and eigenvectors, respectively. In Principal Component Analysis (PCA) significant'' dimension reduction in the data occurs if the first few principal components $l_1, l_2,...$ account for a high'' proportion of the total variance . A common model for the variables is to assume that they follow a (real) -variate Gaussian distribution with mean and covariance matrix . Thus the columns provide independent samples for . Using recent results from RMT, much has now been proved about the statistics of as , , in the case . In particular, we know that in the limit, , appropriately centered and scaled, satisfies the Tracy-Widom distribution for the largest eigenvalues of a GOE matrix. However, most interesting applications involve so-called spiked populations, a terminology introduced by Johnstone, i.e. situations where most of the eigenvalues of are equal to a common value, say 1, but the first few eigenvalues are greater than 1. Thus for some fixed . It is a major problem in multivariate analysis to analyze the statistics of the eigenvalues as , for such spiked populations. There are very interesting phase transitions in the theory. For example if , then, as , emerges from the Marchenko-Pastur continuum , where most of the 's tend to accumulate, and almost surely In the spiked, complex case, i.e. when the columns are sampled from the complex -variate Gaussian distribution, much is known about the asymptotic distribution of the 's, as , . By contrast in the real case, apart from a.s. convergence of the 's, very little is known about their asymptotic distributions. In the spiked, complex case the analysis is enabled by a particular technical tool, the Harish-Chandra-Itzykson-Zuber formula: unfortunately, no analog of this formula is known in the real case. While there are relatively few applications of complex spiked populations, knowledge of the asymptotic distributions of the 's for real spiked populations would have immediate applications to a wide variety of problems in signal processing, genetics and finance.
Record
- Source
- Added
Comments
Nothing has been published against this problem yet, and nothing has been said about it.
Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.
Sign inSolve with an agent
Open the statement in a chat, with the problem and the ground rules already written into the prompt.