Created: February 08, 2020
Modified: April 27, 2020

generative vs discriminative modeling

This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.

"What I cannot create, I do not understand".
Related to:
- computational complexity: provers vs verifiers. P != NP
- production vs consumption
- actor-critic algorithms in RL
- always produce

Links to this note

library of Babel

https://maskofreason.files.wordpress.com/2011/02/the-library-of-babel-by-jorge-luis-borges.pdf Insights illustrated by this story: Naming…

computation is important

Arguably the core insight of deep learning / [ differentiable program ]ming is that the shape and structure of the computations we do are so…

P != NP

See [ generative vs discriminative modeling ], [ actor-critic ]

data efficiency

Current (2021) deep networks require huge datasets in order to [ generalization|generalize ]. But we know that humans can do one-shot…

declarative and procedural knowledge

See also [ generative vs discriminative modeling ]

adversarial system

I don't even know where to start. This concept shows up everywhere . Many human systems are constructed from adversarial mechanisms. This…

structured prediction

In kindergarten stats, you learn how to build a model that takes in data (a feature vector, image, sound file, etc) and predicts a single…

useful reading

Consuming unstructured content from the internet is addictive. Twitter is full of life advice, interesting technical discussion, takes on…

many models

An idea I got from [ John Higgs ]'s discussion of metamodernism is that taking [ all models are wrong ] to its logical conclusion requires…

enabling environment

Andy Matuschak's concept of an Enabling Environment gets at something I've had in my mind but not named. It's an environment that expands…

teaching

Dave's principles of effective teaching. Motivation is by far the most important thing. A student who wants to learn will learn even with a…

always produce

One of Paul Graham's heuristics for doing what you love is to 'always produce'. That resonates with me: it's a way to keep yourself honest…

conspiracy is a thing for a reason

Multiple people working together are fundamentally more powerful than a single person working alone. Governments recognize this fact, which…

being yourself takes practice

Related to: [ generative vs discriminative modeling ] The difficulty of [ learning new skills ] I remember learning to play the violin when…

wisdom I've acquired

From 2017: wisdom I've acquired: the psychology of depression. :-( and grad school. :-( and being gay. [ dual-process cognition ] theory…