← Notes

Central Limit Theorem

Suppose we estimate an expectation with a Monte Carlo sample average:

\[ \hat I = \frac{1}{N}\sum_{i=1}^{N} f(x_i), \qquad x_i\sim p(x). \]

The estimator is centered on the expectation we are trying to estimate,

\[ \mathbb{E}[\hat I] = \mathbb{E}_{x\sim p}[f(x)], \]

and its variance is

\[ \operatorname{Var}(\hat I) = \frac{1}{N}\operatorname{Var}_{x\sim p}(f(x)). \]

By the central limit theorem, for sufficiently large \(N\), the sampling distribution of the average is approximately normal:

\[ \hat I \approx \mathcal{N}\!\left( \mu, \sigma^2 \right), \qquad \mu=\mathbb{E}_{x\sim p}[f(x)], \qquad \sigma^2=\frac{1}{N}\operatorname{Var}_{x\sim p}(f(x)). \]