In information retrieval (IR), effectiveness is defined as the relevance of retrieved information to a given query. System effectiveness evaluation typically focuses on the problem of document retrieval: retrieving a ranked list of documents for each input query. Effectiveness is then measured with respect to an environment of interest consisting of the populations of documents, queries, and relevance judgments defining which of these documents are relevant to which queries. Sampling methodologies are employed for each of these populations to estimate a relevance metric, typically a function of precision (the ratio of relevant documents retrieved to the total number of documents retrieved) and recall (the ratio of relevant documents retrieved to the total number of relevant documents for that query). Conclusions about which systems outperform others are drawn from common experimental design, typically focusing on a random sample of queries, each with a corresponding value of the relevance metric. These individual measurements for each query must be aggregated across a sufficiently large sample of queries to draw conclusions with confidence.

Multivariate analysis deals with the statistical analysis of observations where there are multiple responses on each observational unit. Let X be ann ́ p data matrix where the rows represent observations and the columns, variables. We will denote the variables by X1, X2, ..., Xp. In most cases, it is necessary to sphere the data by subtracting out the means and dividing by the standard deviations. If outliers are a possibility, then a robust sphering method such as that discussed by Venables and Ripley [44, p. 266] should also be tried. Important special cases are p = 1, 2, 3 which correspond to univariate, bivariate and trivariate data.