Taylor Expansion
Why Taylor Expansion
Many functions we care about are too complicated to work with directly, but can be approximated locally by a polynomial. A Taylor expansion builds that polynomial from the value and derivatives of the function at a chosen expansion point.
Single-Variable Taylor Expansion
For a scalar function \(f(x)\), expanded around \(x=a\),
Each derivative captures another aspect of the local shape:
- \(f(a)\): the value at the expansion point.
- \(f'(a)\): local slope.
- \(f''(a)\): local curvature.
- \(f'''(a)\) and higher: how lower-order shape itself changes.
The approximation is local: it assumes the derivatives measured at the expansion point remain informative nearby.
Multivariate Taylor Expansion
For a scalar function
choose an expansion point \(x_0\) and write a nearby point as
The vector \(h\) describes both the direction and distance from the expansion point. The second-order approximation is
The three visible pieces are the starting value, the linear response, and the curvature response.
Concrete Example
Consider
Approximate the function at \(x=(1,1.1)\) using an expansion around \(x_0=(1,1)\). The displacement is
The starting value is
The gradient is
Hence the first-order response is
giving the first-order approximation
The Hessian is
The second-order term is
Therefore,