“Lagrange multiplier test”的意思、由来-开放百科全书

In statistics, the Lagrange multiplier (LM) test, also known as the score test, is one of three classical approaches to hypothesis testing, together with the Wald test and the likelihood-ratio test, for testing a null hypothesis

for a parameter of interest

.^[1]

The basic idea behind the LM test is that if the restricted estimator{{definition needed|date=March 2019}} is near the maximum of the likelihood function, the gradient of the likelihood function—known as the score function—evaluated at the restricted estimator should be close to zero, and can be shown to asymptotically follow a normal distribution with mean zero and known variance. This result has first been proved by C. R. Rao in 1948^[2], leading to the original name of score test.

An alternative and numerically identical version of the test was derived by S. D. Silvey in 1959 using the vector of Lagrange multipliers in the Lagrangian expression of the constrained likelihood function,^[3] which led to the name that has become more commonly used, particularly in econometrics, since Breusch and Pagan's much-cited 1980 paper.^[1] If the constraint is non-binding at the maximum likelihood, the vector of Lagrange multipliers should be zero.

The main advantage of the LM test over the Wald test and likelihood ratio test is that the LM test only requires the computation of the restricted estimator. This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.{{cn|date=March 2019}} Further, because the LM test only requires the estimation of the likelihood function under the null hypothesis, it is less specific than the other two tests about the precise nature of the alternative hypothesis.^[4]

Single parameter test

The statistic

Let

be the likelihood function which depends on a univariate parameter

and let

be the data. The score

is defined as

which has an asymptotic distribution of

, when

is true. While asymptotically identical, calculating the LM statistic using the outer-gradient-product estimator of the Fisher information matrix can lead to bias in small samples.^[6]

Note on notation

Note that some texts use an alternative notation, in which the statistic

is tested against a normal distribution. This approach is equivalent and gives identical results.

Justification

The case of a likelihood with nuisance parameters

As most powerful test for small deviations

where

is the likelihood function,

is the value of the parameter of interest under the null hypothesis, and

is a constant set depending on the size of the test desired (i.e. the probability of rejecting

is true; see Type I error).

The score test is the most powerful test for small deviations from

. To see this, consider testing

versus

. By the Neyman–Pearson lemma, the most powerful test has the form

Relationship with other hypothesis tests

The likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses.^[7]^[8] When testing nested models, the statistics for each test converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models.

Multiple parameters

A more general score test can be derived when there is more than one parameter. Suppose that

is the maximum likelihood estimate of

under the null hypothesis

while

and

are respectively, the score and the Fisher information matrices under the alternative hypothesis. Then

asymptotically under

, where

is the number of constraints imposed by the null hypothesis and

Special cases

In many situations, the score statistic reduces to another commonly used statistic.^[9]

In linear regression, the Lagrange multiplier test can be expressed as a function of the F-test.^[10]

When the data follows a normal distribution, the score statistic is the same as the t statistic.{{clarify|reason=this can't always be true ... eg when null hypothesis is on the variance|date=March 2011}}

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

When the data consists of failure time data in two groups, the score statistic for the Cox partial likelihood is the same as the log-rank statistic in the log-rank test. Hence the log-rank test for difference in survival between two groups is most powerful when the proportional hazards assumption holds.

See also

References

1. ^¹{{cite journal |first=T. S. |last=Breusch |authorlink=Trevor S. Breusch |first2=A. R. |last2=Pagan |authorlink2=Adrian Pagan |title=The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics |journal=Review of Economic Studies |volume=47 |issue=1 |year=1980 |pages=239–253 |jstor=2297111 }}
2. ^{{cite journal |first=C. Radhakrishna |last=Rao |title=Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation |journal=Mathematical Proceedings of the Cambridge Philosophical Society |volume=44 |issue=1 |year=1948 |pages=50–57 |doi=10.1017/S0305004100023987 }}
3. ^{{cite journal |first=S. D. |last=Silvey |title=The Lagrangian Multiplier Test |journal=Annals of Mathematical Statistics |volume=30 |issue=2 |year=1959 |pages=389–407 |jstor=2237089 }}
4. ^{{cite book |first=Peter |last=Kennedy |title=A Guide to Econometrics |location=Cambridge |publisher=MIT Press |edition=Fourth |year=1998 |isbn=0-262-11235-3 |page=68 }}
5. ^Lehmann and Casella, eq. (2.5.16).
6. ^{{cite journal |first=Russel |last=Davidson |first2=James G. |last2=MacKinnon |title=Small sample properties of alternative forms of the Lagrange Multiplier test |journal=Economics Letters |volume=12 |issue=3–4 |year=1983 |pages=269–275 |doi=10.1016/0165-1765(83)90048-4 }}
7. ^{{cite book |title=Handbook of Econometrics |last=Engle |first=Robert F. |editor=Intriligator, M. D. |editor2=Griliches, Z. |publisher=Elsevier |year=1983 |volume=II |pages=796–801 |chapter=Wald, Likelihood Ratio, and Lagrange Multiplier Tests in Econometrics |isbn=978-0-444-86185-6 }}
8. ^{{cite book|last1=Burzykowski|first1=Andrzej Gałecki, Tomasz|title=Linear mixed-effects models using R : a step-by-step approach|date=2013|publisher=Springer|location=New York, NY|isbn=1461438993}}
9. ^{{cite book |editor-last=Cook |editor-first=T. D. |editor2-last=DeMets |editor2-first=D. L. |year=2007 |title=Introduction to Statistical Methods for Clinical Trials |publisher=Chapman and Hall |isbn=1-58488-027-9 |pages=296–297 }}
10. ^{{cite journal |first=Walter |last=Vandaele |title=Wald, likelihood ratio, and Lagrange multiplier tests as an F test |journal=Economics Letters |year=1981 |volume=8 |issue=4 |pages=361–365 |doi=10.1016/0165-1765(81)90026-4 }}