Central Limit Theorem
Suppose we estimate an expectation with a Monte Carlo sample average:
\[
\hat I
=
\frac{1}{N}\sum_{i=1}^{N} f(x_i),
\qquad x_i\sim p(x).
\]
The estimator is centered on the expectation we are trying to estimate,
\[
\mathbb{E}[\hat I]
=
\mathbb{E}_{x\sim p}[f(x)],
\]
and its variance is
\[
\operatorname{Var}(\hat I)
=
\frac{1}{N}\operatorname{Var}_{x\sim p}(f(x)).
\]
By the central limit theorem, for sufficiently large \(N\), the sampling distribution of the average is approximately normal:
\[
\hat I
\approx
\mathcal{N}\!\left(
\mu,
\sigma^2
\right),
\qquad
\mu=\mathbb{E}_{x\sim p}[f(x)],
\qquad
\sigma^2=\frac{1}{N}\operatorname{Var}_{x\sim p}(f(x)).
\]