Modified: .
Modified: .
Warren Buffett suggests the exercise of writing down [ 25 career goals ]. Then choose the five most important. Then avoid at all costs…
Modified: October 02, 2020.
How to consume, from reddit :
Modified: February 25, 2022.
Link: A Universal Law of Robustness via Isoperimetry | OpenReview This paper purports to explain (and quantify) the observed fact that…
Modified: March 03, 2022.
Companies that are on my radar as doing cool stuff, or where I might consider reaching out to people or applying for jobs. Big labs / AGI…
Modified: February 16, 2023.
Companies that are on my radar as doing cool stuff, or where I might consider reaching out to people or applying for jobs. Big labs / AGI…
Modified: February 16, 2023.
Doing [ math ] seems like a really promising area for AI. And by 'math' I mean math research (not arithmetic, which computers are already…
Modified: April 26, 2022.
Most 'AI' in business domains doesn't have any intellectual relationship to AGI research. In the vast majority of problem spaces, what you…
Modified: March 02, 2022.
In the spirit of [ prediction as a model-building exercise ]. Language modeling: system writes publishable poetry: debatably already…
Modified: April 07, 2022.
This page is a general jumping-off point for organizing my thoughts about the [ AI research landscape ], where the field is, where it is…
Modified: November 27, 2023.
As of April 2021: Giant [ transformer ]s work better than anyone has a right to expect. GPT3 is fucking amazing. [ DALL-E ] clearly has some…
Modified: April 23, 2021.
AI safety, as a term, is sterile and hard to get excited about. Preventing catastrophe is important, but doesn't motivate me, since [ the…
Modified: January 24, 2022.
An AI that thought 'like a human' would be able to engage in [ meditation ]. What does this imply? What capabilities are missing from…
Modified: February 26, 2022.
One of three types of [ ionotropic receptors|ionotropic ] [ glutamate ] receptor, along with [ NMDA receptor ]s and kainate receptors. It is…
Modified: February 13, 2022.
Modified: July 31, 2021.
Modified: .
What do I think of the Antitrust Case Against Google ? What are the issues? What principles are in question? What remedies are being…
Modified: June 11, 2020.
[ research idea ]s The ASVI paper proposes adding a learnable constant to the mean (a canonical parameter) of each Gaussian distribution…
Modified: June 03, 2020.
Whale watching trip Computer history museum Climb twin peaks SFMOMA Sutro baths Nude beach
Modified: August 31, 2020.
The Bayesian approach to statistics is to 'just use probability theory'. You write down a joint probability distribution over observed and…
Modified: April 08, 2023.
See https://emtiyaz.github.io/papers/learning_from_bayes.pdf Suppose we have a learning problem For some choice of exponential-family…
Modified: July 18, 2021.
Modified: .
Modified: .
A model of [ option ] prices that assumes: The existence of a risk-free asset paying some interest rate, for example, US Treasury bonds…
Modified: October 26, 2021.
For any strictly [ convex ] function , define the Bregman divergence: Examples: (Squared) Euclidean distance : choose the squared norm…
Modified: March 07, 2022.
The Buddha really liked lists. For example: the three refuges (the buddha, the dharma, and the sangha) the [ three characteristics ] the…
Modified: July 08, 2022.
Modified: .
Christianity's best ideas: [ grace ]: you're never too far gone to be redeemed. You may have done things that are so terrible you can…
Modified: February 15, 2022.
paper 2015 Existing systems for extracting features ('fingerprints') from molecules look a lot like convnets. They repeatedly apply a hash…
Modified: June 06, 2020.
Related to [ natural gradient ] and the [ Fisher information ] matrix. Let's say we have a parametric model of some data. The Cramer-Rao…
Modified: July 05, 2022.
Aka computational trinitarianism . Churchill, My Early Life : I have noticed in my life deep resemblances between many different kinds of…
Modified: April 26, 2022.
Generates images from captions by combining [ CLIP ]-style representation learning with a [ diffusion model ] model to construct images from…
Modified: April 30, 2022.
The problem of [ exposure bias ] (where an autoregressive sequence model goes off the rails of its training distribution) comes up as a…
Modified: October 13, 2022.
Chemically, (n,n-dimethyl)-tryptamine is a [ substituted tryptamines|tryptamine ] molecule with two methyl groups hanging off of the…
Modified: May 22, 2022.
Modified: .
A pointwise maximum of [ convex ] functions Specifically, we require that is convex in for every . is itself convex in , and when…
Modified: August 13, 2022.
https://arxiv.org/abs/2106.10314 In Sequential Monte Carlo, we can resample with any set of weights, as long as we then initialize the new…
Modified: June 28, 2021.
Any reasonable 'adapted' and 'integrable' [ stochastic process ] can be written as the sum of a [ martingale ] and a [ predictable process…
Modified: August 27, 2022.
Modified: .
Modified: .
Modified: .
Emmett Shear - AGI as "Another Kind of Cell" in the Tissue of Life (Worthy Successor, Episode 11) July 2025 https://www.youtube.com/watch…
Modified: December 05, 2025.
Emmett Shear - Why NATURE Holds the Answer To AI Alignment Sept 2025 https://www.youtube.com/watch?v=Nkhp-mb6FRc (YouTube automated…
Modified: December 05, 2025.
Emmett Shear – SF, Power, AI Alignment, Meditation, Softmax Sept 2025 https://www.youtube.com/watch?v=3SSelpjkNpA (YouTube automated…
Modified: December 05, 2025.
There's no such thing as aligned AI, with Emmett Shear (co-founder Twitch) Sept 2025 https://www.youtube.com/watch?v=H1QV9XbeAoI (YouTube…
Modified: December 05, 2025.
Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering Nov 2025 https://www.youtube.com/watch?v=Ua8nPJ1_yk8 (YouTube…
Modified: December 05, 2025.
Notes on understanding modern innovations in blockchains: smart contracts [ proof of stake ] What are the advantages of blockchains? They…
Modified: March 28, 2021.
One-particle system Let be the [ Lagrangian ] for a system with time-varying position and velocity , with forces defined by a potential…
Modified: July 14, 2022.
Modified: .
Given a [ diffusion process ] specified by the [ stochastic differential equation ] the [ Fokker-Planck ] equation aka Kolmogorov forward…
Modified: August 28, 2022.
Modified: .
Talk from Tom Minka: https://tminka.github.io/papers/acmll2019/ Part 1: Autodiff. Recommended reading: Evaluating Derivatives. Can think of…
Modified: November 01, 2020.
(from [ Principles of Neural Design ]): a G protein skates along the inside of a cell membrane, bumping into the undersides of…
Modified: May 22, 2021.
Modified: August 08, 2021.
A major inhibitory [ neurotransmitter ] in the brain. Related to the action of [ ethanol ] and other depressants and anesthetics. TODO…
Modified: August 08, 2021.
Modified: .
Modified: .
GHB is gamma-hydroxybutyrate (or gamma-hydroxybutyric acid): It is produced endogenously in the body, and agonizes a couple of receptor…
Modified: October 12, 2022.
Modified: February 13, 2023.
Weight-Space View Recall standard linear regression. We suppose and where , where can be augmented with an implicit 1 term to allow a…
Modified: March 16, 2022.
The law says that: when a measure becomes a target, it ceases to be a good measure . One can distinguish four types of Goodhart problems…
Modified: April 08, 2023.
Gatsby's life has been consumed by the single purpose of winning Daisy. He becomes wealthy, buys a giant house, and throws massive parties…
Modified: February 07, 2022.
What are Hamiltonian dynamics? A system has [ phase space ] coordinates (p, q) , representing momenta and positions respectively. How do…
Modified: July 14, 2022.
Sometimes caricatured as 'thesis', 'antithesis', 'synthesis', but Hegel didn't use those words! His three stages were: Abstract Negative…
Modified: .
References: Dayan, Hinton, Neal, Zemel (1994) https://www.cs.toronto.edu/~hinton/absps/helmholtz.pdf This paper is one of the first to…
Modified: March 24, 2024.
Modified: .
References: J.J. Hopfield. Neural networks and physical systems with emergent collective computational abilities (1982). https://www.ncbi…
Modified: March 29, 2024.
Normally I think it's valuable to know who you are. Having good models of your future self's actions and desires puts you in a position to…
Modified: May 22, 2020.
We think of creating money as the government's job. But actually, anyone can create money by writing an IOU. In fact most of the [ money…
Modified: .
advice: really get to know yourself. ask "what do I want?", or better, from a bundle-of-conflicting-impulses frame, "what do we want?"
Modified: .
This is the technical formulation that makes it meaningful to write [ stochastic differential equation ]s 'driven by' a Weiner process…
Modified: August 30, 2022.
A Itô process is a [ stochastic process ] satisfying a [ stochastic differential equation ] of the form where is Brownian motion. This…
Modified: August 27, 2022.
The partial derivatives of a multivariate function form its Jacobian matrix The convention here (matching Wikipedia, and I believe also…
Modified: August 13, 2022.
For any [ convex ] function and probability distribution , Jensen's inequality states that The special case of a distribution over two…
Modified: June 25, 2022.
Modified: August 03, 2022.
Modified: .
References: The KLF: Chaos, Magic and the Band who Burned a Million Pounds (2013), by [ John Higgs ].
Modified: August 03, 2022.
Given a [ constrained optimization ] problem over a [ convex ] function , we consider the [ Lagrangian ] function introducing variables…
Modified: July 07, 2022.
This is a block [ Kronecker product ]. Given block matrices their product is Unlike the Kronecker product, the Khatri-Rao product is…
Modified: February 25, 2022.
Modified: .
The Kraft inequality in information theory states (roughly?) that, for any probability distribution , there is a prefix code C under which…
Modified: April 12, 2022.
Given two matrices , the Kronecker product is the block matrix that 'tiles' across the entries of : It can be generalized to block…
Modified: October 20, 2022.
LSD is essentially rigidified [ tryptamine ] — whereas the tryptamine side chain swings loose in other psychedelic tryptamines, in LSD (and…
Modified: .
Good tutorial: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Modified: October 17, 2022.
We're given a [ constrained optimization ] problem Note that the standard formulation of Lagrange multipliers handles only equality…
Modified: April 29, 2023.
Modified: .
Modified: July 06, 2022.
The Laplacian or Laplace operator computes the [ vector divergence ] of the gradient of a function , given by the sum of 'unmixed…
Modified: June 08, 2024.
Yann Lecun's famous cake analogy is that: "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake…
Modified: June 12, 2021.
References: Jess Riedel on the Legendre transform in physics Stack Overflow discussion Prof. V. Balakrishnan on Hamiltonian dynamics…
Modified: July 15, 2022.
A function is -Lipschitz continuous if changing the input by a little bit can't change the output by more than a proportional amount…
Modified: .
A function is -Lipschitz continuous if changing the input by a little bit can't change the output by more than a proportional amount…
Modified: .
References: Wikipedia: https://en.wikipedia.org/wiki/London_dispersion_force Khan academy: https://www.khanacademy.org/science/ap…
Modified: September 13, 2023.
Some ideas from this paper. Binary positional encodings . Memory index is represented by a binary vector . This has the property that…
Modified: February 13, 2023.
Used in analyzing the stability of an equilibrium of a dynamical system. A Lyapunov function is a scalar-valued function of the state space…
Modified: January 04, 2024.
Methylene-dioxy-[ methamphetamine ]
Modified: July 24, 2023.
There are some suggested supplement regiments for taking [ MDMA ] safely: https://rollsafe.org/mdma-supplements/ https://matznerd.com/how-to…
Modified: May 23, 2022.
Modified: .
A [ stochastic process ] in which the past is independent of the future, conditioned on the current value. Striking point made by https…
Modified: August 29, 2022.
References: https://photonlines.substack.com/p/an-intuitive-guide-to-maxwells-equations Electric and magnetic fields : the electromagnetic…
Modified: June 08, 2024.
Modified: .
Modified: .
Minerva : they basically fine-tuned a language model on ArXiv papers and web pages containing LaTeX, so that it can produce latex. This is…
Modified: August 03, 2022.
Modified: .
Nicotinamide adenine dinucluotide hydride. Consists of two [ nucleotide ]s---one with an adenine nucleobase, the other with nicotinamide…
Modified: July 31, 2021.
One of three types of [ ionotropic receptors|ionotropic ] [ glutamate ] receptor, along with [ AMPA receptor ]s and kainate receptors. It is…
Modified: February 13, 2022.
Gilmer at al. paper 2017 Experiments on QM9. Unlike SMILES strings, includes molecular geometry . General formulation of message passing…
Modified: March 21, 2022.
Modified: .
Michael Nielson wrote some nice thoughts about existential risk from artificial superintelligence (ASI): https://michaelnotebook.com/xrisk…
Modified: September 29, 2023.
Notes from Charles Margossian's talk on pharmacometrics models. Types of ODEs: Linear: can be solved by [ matrix exponential ]ials nonlinear…
Modified: August 19, 2021.
It seems like all of my Roam problems are solved. I can write real posts, with paragraphs. I can use latex inline: ! I can…
Modified: February 07, 2022.
If two hypotheses are equally consistent with the data, the simpler is more likely to be 'true'. Formally, it is more likely to generalize…
Modified: March 03, 2022.
Here's a formal argument for [ Occam's razor ], adapted from Theorem 2.2 of Kearns and Vazirani's "An Introduction to Computational Learning…
Modified: April 20, 2022.
Useful and non-obvious #life-advice, from https://www.theguardian.com/lifeandstyle/2020/sep/04/oliver-burkemans-last-column-the-eight…
Modified: March 08, 2021.
Most important body parts are eyes, hips, feet. Eyes are most important: don't look at your feet, look down the mountain (this dovetails…
Modified: February 12, 2023.
[ explicit models of uncertainty ] (8/28/20) Standing up for myself (8/28/20): It's much easier to feel righteous about standing up for my…
Modified: January 23, 2022.
See [ generative vs discriminative modeling ], [ actor-critic ]
Modified: May 23, 2021.
Modified: .
I'm trying to build my understanding. These are fragments of intuitions. Bayesian inference starts with a prior P and a likelihood. Given…
Modified: January 15, 2021.
This page contains my notes from the Pointing Out the Great Way Level 1 [ mahamudra ] [ meditation ] [ retreats|retreat ] course I took…
Modified: October 07, 2022.
Attempt to outline [ Dan Brown ]'s version of the [ mahamudra ] path to awakening: 'Ordinary' emptiness practices: Emptiness of self. Search…
Modified: September 13, 2022.
Predictions for: April 23ish (one month from now): Cases in the bay area will have inflected, and at a relatively lower level than NYC or…
Modified: August 28, 2020.
Chapter 2: why an animal needs a brain. Brains allow for control and adaptation within a single lifetime. They make more sense for bigger…
Modified: November 28, 2021.
[ Project Management Checklist ] {{TODO}} Define project Objectives:: Deliverables:: Key performance indicators:: Scope:: Budget…
Modified: January 23, 2022.
Following the pattern of [ state values, then action values ], the one-step [ temporal difference ] update for action values is called…
Modified: April 23, 2022.
Fell victim to (or helped commit) a spiritual fraud, Joya Santanya. If Ram Dass can't tell if spiritual teachings are legit, does that…
Modified: October 04, 2021.
Modified: .
Modified: .
Modified: .
Modified: .
Very simple brainstorming Say we want to search over STS architectures. An STS architecture here is really just a list of components, with…
Modified: March 07, 2020.
Basic of quantum mechanics from Wikipedia : The quantum state of a system is a ray (equivalence class of vectors) in a separable (has a…
Modified: May 16, 2020.
The theme of the book is that top-down control necessarily involves abstraction, and often, changing existing systems so that they are more…
Modified: February 20, 2020.
Modified: .
The 'universal prior' gives probability to all sequences that can be generated by a -bit program. Generally a sequence is generated by…
Modified: April 08, 2023.
I just saw a paper by Tom rainforth L on steca stick conditioning and I wanna try to articulate what you could mean by that and what I think…
Modified: May 09, 2021.
The stuff in marijuana is delta-9-THC, where the '9' refers (somehow?) to the position of the double bond in the cyclohexane ring?
Modified: August 24, 2022.
On the Robot Brains podcast , Andrej Karpathy explained to Pieter Abbeel why he thinks Tesla has the right approach to self-driving. Tesla…
Modified: January 24, 2022.
Turning and turning in the widening gyre. The falcon cannot hear the falconer. Things fall apart; the center cannot hold. Mere anarchy is…
Modified: February 23, 2020.
Notes from working through Kevin Buzzard's Natural number game (imperial.ac.uk) using the Lean theorem prover. We know from the [ Curry…
Modified: June 09, 2021.
In a bandit setting, in each round we see a context , choose an action , and receive a reward sampled from a distribution with some…
Modified: March 26, 2025.
Modified: February 25, 2022.
Massive list here: https://github.com/cedrickchee/awesome-bert-nlp Bahdanau, Cho, Bengio. Neural Machine Translation by Jointly Learning to…
Modified: January 24, 2022.
1:30pm: 240ug of LSD (plus a third phenibut capsule) intents for the day: generally feel my feelings and my body what gets in the way of…
Modified: .
things to maybe consider followup on giving myself as much attention as I need/can ideal parenting, again and again do I want therapy / NET…
Modified: .
Transcript of dictation (over the course of an ~hour with a few hits on the vape): Infinitely bad things might happen and I might not be…
Modified: .
Frustration how do I get better fucking emotions that might let me actually fucking do things the part that doesn't believe I can write…
Modified: .
Modified: .
Modified: .
Alan Turing proposed that we might consider a machine to think if it could succeed in an 'imitation game'. In one variant of this game, a…
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
Modified: .
The -Wasserstein distance between probability distributions is defined as where the infimum is over all joint distributions having…
Modified: August 04, 2022.
Modified: .
Who are you? Hi, I'm Dave. I'm interested in the secrets of intelligence, consciousness, the good life, and other easy questions. I have a…
Modified: November 30, 2022.
Abstraction is lossy [ compression ]. A good abstraction throws away everything not relevant to a particular problem, while preserving a…
Modified: February 20, 2022.
Modified: February 23, 2020.
Modified: .
An acetyl group is - (C = O) - CH3 Attaching an OH gives acetic acid.
Modified: May 14, 2021.
A single big goal is very difficult and risky to achieve and, in the time you're working on it, you will not have achieved any goals at all…
Modified: February 14, 2022.
Modified: .
This is really the only advice I know of for [ how to make a friend ]. And it works. Invite someone to social events, as you would a…
Modified: July 18, 2022.
How does an active inference agent work? I'll spell out a concrete story that we can plausibly implement. As an example, suppose the agent…
Modified: April 14, 2025.
Modified: .
Modified: .
(references: https://julien-vitay.net/deeprl/ActorCritic.html ) Advantage actor-critic The advantage function is a 'centered' version of…
Modified: June 22, 2022.
Modified: .
Imagine you want to make decisions in the world. One way is to learn how the world works, consider possible approaches, and think through…
Modified: April 26, 2018.
In reinforcement learning, the advantage of a state-action pair under a policy is the improvement in value from taking action a (and…
Modified: July 05, 2022.
Here are some high-quality ways to have fun: Things I do now and should do more of: Ski trips with friends Traveling athletic events (races…
Modified: October 02, 2020.
Modified: .
I don't even know where to start. This concept shows up everywhere . Many human systems are constructed from adversarial mechanisms. This…
Modified: July 10, 2020.
Take notes in a personal knowledge manager like [ Roam Research ] or [ Obsidian ] that makes it easy to jot down fragments of ideas and…
Modified: February 07, 2022.
It's natural to fault someone who fails to follow advice, especially when the results are predictably bad. It can be helpful to notice and…
Modified: March 19, 2024.
see also [ agent ] agency has two parts: [ yin and yang ]: yang agency: take steps. break apart a goal into subgoals until one is tractable…
Modified: April 13, 2025.
There is a similarity in kind between the negative effects of: rape slavery feeling forced to work on someone else's projects, by a boss…
Modified: March 02, 2022.
see also [ agency ]
Modified: February 25, 2022.
[ ethanol|Ethyl alcohol ] is by far the most harmful drug that most people regularly consume. It is almost certainly way more harmful than…
Modified: August 01, 2023.
Modified: .
Fundamentally an algorithm is any computational procedure: something that takes in data and spits out some function of that data. Computer…
Modified: November 29, 2022.
Modified: .
Modified: .
Fundamental observation is that [ the map is not the territory ]. BUT---there is a real and important distinction between models that are…
Modified: February 06, 2023.
This was one of the beliefs that SuccessfulFriend and I shared (and still share). This seems like a cynical thing to say. But it's not meant…
Modified: January 17, 2021.
One of Paul Graham's heuristics for doing what you love is to 'always produce'. That resonates with me: it's a way to keep yourself honest…
Modified: February 25, 2022.
Modified: .
Churchill, 'My Early Life': Come on now all you young men, all over the world. You are needed more than ever now to fill the gap of a…
Modified: April 25, 2022.
Modified: May 22, 2022.
Structure: NH2 - C - COOH (amine - C - acid/carboxyl), usually with a side chain hanging from the central (alpha) carbon. Roughly speaking…
Modified: May 14, 2021.
Modified: .
Shorthand for alpha-methyl[ phenethylamine ]. Adding another methyl group to the (n)itrogen gives n-methyl-amphetamine aka…
Modified: August 27, 2022.
I used to play board games with ExBoyfriend and his friend Will. Will was literally getting a PhD in game studies, and almost always beat…
Modified: January 18, 2021.