Created: January 16, 2022
Modified: January 16, 2022
Modified: January 16, 2022
inductive bias
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.- Ways to specify inductive bias:
- Feature engineering
- Prior distribution
- acts as regularizer in MAP estimates
- Graphical model (constraint on independence structure)
- relies on feature engineering: choosing variables to model
- Probabilistic program (joint prior distribution)
- Neural net architecture (i.e., differentiable program)
- Kernel
- implies (or enriches) feature engineering
- special case of a similarity function
- Similarity function (as in k-nearest-neighbors)
- Regularization --- explicit (e.g., L2) or implicit (from dropout, SGD, etc.)