Estimating a mean can be an arduous task. Observe, for instance, the “sudden jumps” in trajectories of the Monte Carlo estimate of , where .
Let be a sequence of i.i.d. and integrable random variables, and let . For all we have that
is or according to or .
If , then converges towards at a rate almost surely slower than :
Proof of the proposition.
We can suppose that , by the scale invariance of the problem. Note also that
from which we find
since almost surely. From the Borel-Cantelli lemma and
we obtain that in this case.
When , we let and note that . Therefore
Since the sum over of the right hand side probabilities converges (again using the fact that ), the Borel-Cantelli lemma yields the result.
The result can be refined knowing that for some small : implies . The almost sure lower bound on the rate of convergence can be correspondingly adjusted.
Many thanks to Xi’an who wondered about the occurence of these jumps.