See https://emtiyaz.github.io/papers/learning_from_bayes.pdf Suppose we have a learning problem For some choice of exponential-family…
The Bayesian approach to statistics is to 'just use probability theory'. You write down a joint probability distribution over observed and…
https://arxiv.org/abs/2106.10314 In Sequential Monte Carlo, we can resample with any set of weights, as long as we then initialize the new…
I'm trying to build my understanding. These are fragments of intuitions. Bayesian inference starts with a prior P and a likelihood. Given…
Relevant papers: DIfferentiable compositional kernel learning for Gaussian Processes (Sun et al., 2018) Differentiable Architecture Search…
Like quantum mechanics! We build up a distribution over variables defined so far. When we need to use a value, we sample from this…
Measures uncertainty, disorder, or randomness. The (Shannon) entropy of a probability distribution is: The quantity inside the…
(note: this is dancing around the issues around why I think [ probabilistic programming is not AI research ], even if it will be a…
Exponential Families, Conjugacy, Convexity, and Variational Inference Any parameterized family of probability densities that can be written…
Importance sampling allows us to compute expectations under a distribution using samples from a different distribution , by weighting the…
Much of statistical practice is concerned with distinguishing signal from noise. For example, significance tests quantify the likelihood…
Short descriptions of things, when they exist, must capture some kind of structure. The principle of [ Occam's razor ] posits that we should…
I have a [ strong opinion weakly held ] that doesn't seem to be wildly shared in the [ approximate Bayesian inference ] community: reverse…
[ thoughts on multivariate causalimpact ]
Many [ probabilistic programming ] researchers frame their work as part of the broader problem of [ artificial intelligence ]. Artificial…
(aka, why frequentists will always make more money) In the "real" (corporate/governmental) world, most high-level decision making is…
Reservoir samplers solve the following task: sample items without replacement from a stream of unknown length . Because the length is…
Andrew Gelman believes that in certain areas of research , like the social sciences, everything is connected. "I’m not expressing…
SSC link: How general is this phenomenon? You have a belief Your belief colors your perception of something that doesn't inherently…
How should people do VI? One ultimate goal is that you write a Stan model (or better, a model with discrete variables, but one step at a…
Note: these are personal notes, taken as I was refreshing myself on this material. They're mostly stream of consciousness and probably not…