“Big O in probability notation”的意思、由来-开放百科全书

词条

Big O in probability notation

释义

Definitions
Small O: convergence in probability Big O: stochastic boundedness Comparison of the two definitions
Example
References

The order in probability notation is used in probability theory and statistical theory in direct parallel to the big-O notation that is standard in mathematics. Where the big-O notation deals with the convergence of sequences or sets of ordinary numbers, the order in probability notation deals with convergence of sets of random variables, where convergence is in the sense of convergence in probability.^[1]

Definitions

Small O: convergence in probability

For a set of random variables X_n and a corresponding set of constants a_n (both indexed by n, which need not be discrete), the notation

means that the set of values X_n/a_n converges to zero in probability as n approaches an appropriate limit.

Equivalently, X_n = o_p(a_n) can be written as X_n/a_n = o_p(1),

where X_n = o_p(1) is defined as,

for every positive ε.^[2]

Big O: stochastic boundedness

The notation,

means that the set of values X_n/a_n is stochastically bounded. That is, for any ε > 0, there exists a finite M > 0 and a finite N > 0 such that,

Comparison of the two definitions

The difference between the definition is subtle. If one uses the definition of the limit, one gets:

Big O_p(1):
Small o_p(1):

The difference lies in the δ: for stochastic boundedness, it suffices that there exists one (arbitrary large) δ to satisfy the inequality, and δ is allowed to be dependent on ε (hence the δ_ε). On the other side, for convergence, the statement has to hold not only for one, but for any (arbitrary small) δ. In a sense, this means that the sequence must be bounded, with a bound that gets smaller as the sample size increases.

This suggests that if a sequence is o_p(1), then it is O_p(1), i.e. convergence in probability implies stochastic boundedness. But the reverse does not hold.

Example

If is a stochastic sequence such that each element has finite variance, then

(see Theorem 14.4-1 in Bishop et al.)

If, moreover, is a null sequence for a sequence of real numbers, then converges to zero in probability by Chebyshev's inequality, so

.

References

1. ^Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, OUP. {{ISBN|0-19-920613-9}}
2. ^Yvonne M. Bishop, Stephen E.Fienberg, Paul W. Holland. (1975,2007) Discrete multivariate analysis, Springer. {{ISBN|0-387-72805-8}}, {{ISBN|978-0-387-72805-6}}

4 : Mathematical notation|Probability theory|Statistical theory|Convergence (mathematics)

随便看

开放百科全书收录14589846条英语、德语、日语等多语种百科知识，基本涵盖了大多数领域的百科知识，是一部内容自由、开放的电子版国际百科全书。