See https://emtiyaz.github.io/papers/learning_from_bayes.pdf Suppose we have a learning problem For some choice of exponential-family…
Modified: July 18, 2021.
The Bayesian approach to statistics is to 'just use probability theory'. You write down a joint probability distribution over observed and…
Modified: April 08, 2023.
https://arxiv.org/abs/2106.10314 In Sequential Monte Carlo, we can resample with any set of weights, as long as we then initialize the new…
Modified: June 28, 2021.
I'm trying to build my understanding. These are fragments of intuitions. Bayesian inference starts with a prior P and a likelihood. Given…
Modified: January 15, 2021.
Relevant papers: DIfferentiable compositional kernel learning for Gaussian Processes (Sun et al., 2018) Differentiable Architecture Search…
Modified: March 07, 2020.
Like quantum mechanics! We build up a distribution over variables defined so far. When we need to use a value, we sample from this…
Modified: May 16, 2020.
Measures uncertainty, disorder, or randomness. The (Shannon) entropy of a probability distribution is: The quantity inside the…
Modified: April 15, 2022.
(note: this is dancing around the issues around why I think [ probabilistic programming is not AI research ], even if it will be a…
Modified: February 10, 2022.
Exponential Families, Conjugacy, Convexity, and Variational Inference Any parameterized family of probability densities that can be written…
Modified: May 21, 2022.
Importance sampling allows us to compute expectations under a distribution using samples from a different distribution , by weighting the…
Modified: July 05, 2022.
Much of statistical practice is concerned with distinguishing signal from noise. For example, significance tests quantify the likelihood…
Modified: May 19, 2022.
Short descriptions of things, when they exist, must capture some kind of structure. The principle of [ Occam's razor ] posits that we should…
Modified: April 12, 2022.
I have a [ strong opinion weakly held ] that doesn't seem to be wildly shared in the [ approximate Bayesian inference ] community: reverse…
Modified: March 14, 2022.
[ thoughts on multivariate causalimpact ]
Modified: February 15, 2022.
Many [ probabilistic programming ] researchers frame their work as part of the broader problem of [ artificial intelligence ]. Artificial…
Modified: December 01, 2023.
(aka, why frequentists will always make more money) In the "real" (corporate/governmental) world, most high-level decision making is…
Modified: March 04, 2022.
Reservoir samplers solve the following task: sample items without replacement from a stream of unknown length . Because the length is…
Modified: May 10, 2022.
Andrew Gelman believes that in certain areas of research , like the social sciences, everything is connected. "I’m not expressing…
Modified: June 08, 2021.
SSC link: How general is this phenomenon? You have a belief Your belief colors your perception of something that doesn't inherently…
Modified: March 12, 2021.
References: Jacob Eisner, High-Level Explanation of Variational Inference (2011) https://www.cs.jhu.edu/~jason/tutorials/variational.html…
Modified: April 26, 2022.
Note: these are personal notes, taken as I was refreshing myself on this material. They're mostly stream of consciousness and probably not…
Modified: March 16, 2022.