Created:
Modified:
Modified:
Lipschitz
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.A function is -Lipschitz continuous if changing the input by a little bit can't change the output by more than a proportional amount,
for all in its domain. In particular, choosing implies that the gradient is bounded by , but the definition doesn't require that be differentiable.
Sometimes people talk about "Lipschitz smoothness" or an "-smooth" function to mean a function whose gradient is Lipschitz continuous with constant . Interestingly, a function is Lipschitz smooth if its convex dual is strongly convex (see, e.g., https://xingyuzhou.org/blog/notes/Lipschitz-gradient).