The Hessian
Definition
The Hessian is the matrix of all second partial derivatives of a scalar-valued function. For
the Hessian is
The gradient tells us the local slope. The Hessian tells us how that slope changes as we move: it describes local curvature.
Second-Order Response
In a second-order Taylor expansion, curvature contributes through the quadratic form
The direction \(h\) matters: the same Hessian can imply very different curvature along different directions.
Example
For
the Hessian is
With displacement
first apply the Hessian:
Then
so the second-order correction is
Why the Hessian Matters for Policy Optimization
When KL divergence is expanded locally around the current policy, its first-order term vanishes and its second-order term controls the local change in distribution:
Here the Hessian of the local KL is the Fisher information matrix. This is the bridge from ordinary curvature to the geometry used by natural-gradient and trust-region methods.