In statistics, the likelihood-ratio test is a hypothesis test that involves comparing the goodness of fit of two competing statistical models, typically one found by maximization over the entire parameter space and another found after imposing some constraint, based on the ratio of their likelihoods. If the more constrained model (i.e., the null hypothesis) is supported by the observed data, the two likelihoods should not differ by more than sampling error. Thus the likelihood-ratio test tests whether this ratio is significantly different from one, or equivalently whether its natural logarithm is significantly different from zero.

The likelihood-ratio test, also known as Wilks test, is the oldest of the three classical approaches to hypothesis testing, together with the Lagrange multiplier test and the Wald test. In fact, the latter two can be conceptualized as approximations to the likelihood-ratio test, and are asymptotically equivalent. In the case of comparing two models each of which has no unknown parameters, use of the likelihood-ratio test can be justified by the Neyman–Pearson lemma. The lemma demonstrates that the test has the highest power among all competitors.

Definition

General

Suppose that we have a statistical model with parameter space <math>\Theta</math>. A null hypothesis is often stated by saying that the parameter <math>\theta</math> lies in a specified subset <math>\Theta_0</math> of <math>\Theta</math>. The alternative hypothesis is thus that <math>\theta</math> lies in the complement of <math>\Theta_0</math>, i.e. in <math>\Theta ~ \backslash ~ \Theta_0</math>, which is denoted by <math>\Theta_0^\text{c}</math>. The likelihood ratio test statistic for the null hypothesis <math>H_0 \, : \, \theta \in \Theta_0</math> is given by:

<math display="block">\lambda_\text{LR} = -2 \ln \left[ \frac{~ \sup_{\theta \in \Theta_0} \mathcal{L}(\theta) ~}{~ \sup_{\theta \in \Theta} \mathcal{L}(\theta) ~} \right]</math>

where the quantity inside the brackets is called the likelihood ratio. Here, the <math>\sup</math> notation refers to the supremum. As all likelihoods are positive, and as the constrained maximum cannot exceed the unconstrained maximum, the likelihood ratio is bounded between zero and one and the likelihood ratio test statistic between 0 and infinity.

Often the likelihood-ratio test statistic is expressed as a difference between the log-likelihoods

<math display="block">\lambda_\text{LR} = -2 \left[ \ell( \theta_0 ) - \ell( \hat{\theta} ) \right]</math>

where

<math display="block">\ell( \hat{\theta} ) \equiv \ln \left[\,\sup_{\theta \in \Theta} \mathcal{L}(\theta) \,\right]</math>

is the logarithm of the maximized likelihood function <math>\mathcal{L}</math>, and <math>\ell(\theta_0)</math> is the maximal value in the special case that the null hypothesis is true (but not necessarily a value that maximizes <math>\mathcal{L}</math> for the sampled data) and

<math display="block"> \theta_0 \in \Theta_0 \qquad \text{ and } \qquad \hat{\theta} \in \Theta~</math>

denote the respective arguments of the maxima and the allowed ranges they're embedded in. Multiplying by &minus;2 ensures mathematically that (by Wilks' theorem) <math>\lambda_\text{LR}</math> converges asymptotically to being ²-distributed if the null hypothesis happens to be true. The finite-sample distributions of likelihood-ratio statistics are generally unknown.

The likelihood-ratio test requires that the models be nested &ndash; i.e. the more complex model can be transformed into the simpler model by imposing constraints on the former's parameters. Many common test statistics are tests for nested models and can be phrased as log-likelihood ratios or approximations thereof: e.g. the Z-test, the F-test, the G-test, and Pearson's chi-squared test; for an illustration with the one-sample t-test, see below.

If the models are not nested, then instead of the likelihood-ratio test, there is a generalization of the test that can usually be used: for details, see relative likelihood.

Case of simple hypotheses

A simple-vs.-simple hypothesis test has completely specified models under both the null hypothesis and the alternative hypothesis, which for convenience are written in terms of fixed values of a notional parameter <math>\theta</math>:

<math display="block">

\begin{align}

H_0 &:& \theta=\theta_0 ,\\

H_1 &:& \theta=\theta_1 .

\end{align}

</math>

In this case, under either hypothesis, the distribution of the data is fully specified: there are no unknown parameters to estimate. For this case, a variant of the likelihood-ratio test is available:

<math display="block">

\Lambda(x) = \frac{~\mathcal{L}(\theta_0\mid x) ~}{~\mathcal{L}(\theta_1\mid x) ~}.

</math>

Some older references may use the reciprocal of the function above as the definition. Thus, the likelihood ratio is small if the alternative model is better than the null model.

The likelihood-ratio test provides the decision rule as follows:

The values <math>c</math> and <math>q</math> are usually chosen to obtain a specified significance level <math>\alpha</math>, via the relation

<math display="block"> q \Pr(\Lambda=c \mid H_0) ~ + ~ \Pr(\Lambda < c \mid H_0)~=~\alpha~. </math>

The Neyman–Pearson lemma states that this likelihood-ratio test is the most powerful among all level <math>\alpha</math> tests for this case. This implies that for a great variety of hypotheses, we can calculate the likelihood ratio <math>\lambda</math> for the data and then compare the observed <math>\lambda_\text{LR}</math> to the <math>\chi^2</math> value corresponding to a desired statistical significance as an approximate statistical test. Other extensions exist.

See also

  • Akaike information criterion
  • Bayes factor
  • Johansen test
  • Model selection
  • Vuong's closeness test
  • Sup-LR test
  • Error exponents in hypothesis testing

References

Further reading

  • Practical application of likelihood ratio test described
  • R Package: Wald's Sequential Probability Ratio Test
  • Richard Lowry's Predictive Values and Likelihood Ratios Online Clinical Calculator