Jensen's inequality: Nonlinear Function
Created: June 25, 2022
Modified: June 25, 2022

Jensen's inequality

This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.

For any convex function f(x)f(x) and probability distribution p(x)p(x), Jensen's inequality states that

Exp[f(x)]f(Exp[x])\mathbb{E}_{x\sim p}\left[f(x)\right] \ge f\left(\mathbb{E}_{x\sim p}\left[x\right]\right)

The special case of a distribution over two elements with probabilities p(x0)=tp(x_0) = t and p(x1)=1tp(x_1) = 1-t is just the definition of convexity; the general case follows by an inductive argument.