Warren Buffett suggests the exercise of writing down [ 25 career goals ]. Then choose the five most important. Then avoid at all costs…
How to consume, from reddit :
Link: A Universal Law of Robustness via Isoperimetry | OpenReview This paper purports to explain (and quantify) the observed fact that…
Companies that are on my radar as doing cool stuff, or where I might consider reaching out to people or applying for jobs. Big labs / AGI…
Doing [ math ] seems like a really promising area for AI. And by 'math' I mean math research (not arithmetic, which computers are already…
Most 'AI' in business domains doesn't have any intellectual relationship to AGI research. In the vast majority of problem spaces, what you…
In the spirit of [ prediction as a model-building exercise ]. Language modeling: system writes publishable poetry: debatably already…
This page is a general jumping-off point for organizing my thoughts about the [ AI research landscape ], where the field is, where it is…
As of April 2021: Giant [ transformer ]s work better than anyone has a right to expect. GPT3 is fucking amazing. [ DALL-E ] clearly has some…
AI safety, as a term, is sterile and hard to get excited about. Preventing catastrophe is important, but doesn't motivate me, since [ the…
An AI that thought 'like a human' would be able to engage in [ meditation ]. What does this imply? What capabilities are missing from…
One of three types of [ ionotropic receptors|ionotropic ] [ glutamate ] receptor, along with [ NMDA receptor ]s and kainate receptors. It is…
What do I think of the Antitrust Case Against Google ? What are the issues? What principles are in question? What remedies are being…
[ research idea ]s The ASVI paper proposes adding a learnable constant to the mean (a canonical parameter) of each Gaussian distribution…
Whale watching trip Computer history museum Climb twin peaks SFMOMA Sutro baths Nude beach
The Bayesian approach to statistics is to 'just use probability theory'. You write down a joint probability distribution over observed and…
See https://emtiyaz.github.io/papers/learning_from_bayes.pdf Suppose we have a learning problem For some choice of exponential-family…
A model of [ option ] prices that assumes: The existence of a risk-free asset paying some interest rate, for example, US Treasury bonds…
For any strictly [ convex ] function , define the Bregman divergence: Examples: (Squared) Euclidean distance : choose the squared norm…
My feelings about Buddhism are linked to my feelings about [ psychedelic ]s and the mind more generally. I am pretty ignorant on all of…
The Buddha really liked lists. For example: the three refuges (the buddha, the dharma, and the sangha) the [ three characteristics ] the…
Christianity's best ideas: [ grace ]: you're never too far gone to be redeemed. You may have done things that are so terrible you can…
paper 2015 Existing systems for extracting features ('fingerprints') from molecules look a lot like convnets. They repeatedly apply a hash…
Related to [ natural gradient ] and the [ Fisher information ] matrix. Let's say we have a parametric model of some data. The Cramer-Rao…
Aka computational trinitarianism . Churchill, My Early Life : I have noticed in my life deep resemblances between many different kinds of…
Generates images from captions by combining [ CLIP ]-style representation learning with a [ diffusion model ] model to construct images from…
The problem of [ exposure bias ] (where an autoregressive sequence model goes off the rails of its training distribution) comes up as a…
Chemically, (n,n-dimethyl)-tryptamine is a [ substituted tryptamines|tryptamine ] molecule with two methyl groups hanging off of the…
A pointwise maximum of [ convex ] functions Specifically, we require that is convex in for every . is itself convex in , and when…
https://arxiv.org/abs/2106.10314 In Sequential Monte Carlo, we can resample with any set of weights, as long as we then initialize the new…
Any reasonable 'adapted' and 'integrable' [ stochastic process ] can be written as the sum of a [ martingale ] and a [ predictable process…
Notes on understanding modern innovations in blockchains: smart contracts [ proof of stake ] What are the advantages of blockchains? They…
One-particle system Let be the [ Lagrangian ] for a system with time-varying position and velocity , with forces defined by a potential…
Given a [ diffusion process ] specified by the [ stochastic differential equation ] the [ Fokker-Planck ] equation aka Kolmogorov forward…
Talk from Tom Minka: https://tminka.github.io/papers/acmll2019/ Part 1: Autodiff. Recommended reading: Evaluating Derivatives. Can think of…
(from [ Principles of Neural Design ]): a G protein skates along the inside of a cell membrane, bumping into the undersides of…
A major inhibitory [ neurotransmitter ] in the brain. Related to the action of [ ethanol ] and other depressants and anesthetics. TODO…
GHB is gamma-hydroxybutyrate (or gamma-hydroxybutyric acid): It is produced endogenously in the body, and agonizes a couple of receptor…
Weight-Space View Recall standard linear regression. We suppose and where , where can be augmented with an implicit 1 term to allow a…
The law says that: when a measure becomes a target, it ceases to be a good measure . One can distinguish four types of Goodhart problems…
Gatsby's life has been consumed by the single purpose of winning Daisy. He becomes wealthy, buys a giant house, and throws massive parties…
What are Hamiltonian dynamics? A system has [ phase space ] coordinates (p, q) , representing momenta and positions respectively. How do…
Sometimes caricatured as 'thesis', 'antithesis', 'synthesis', but Hegel didn't use those words! His three stages were: Abstract Negative…
Normally I think it's valuable to know who you are. Having good models of your future self's actions and desires puts you in a position to…
We think of creating money as the government's job. But actually, anyone can create money by writing an IOU. In fact most of the [ money…
advice: really get to know yourself. ask "what do I want?", or better, from a bundle-of-conflicting-impulses frame, "what do we want?"
This is the technical formulation that makes it meaningful to write [ stochastic differential equation ]s 'driven by' a Weiner process…
A Itô process is a [ stochastic process ] satisfying a [ stochastic differential equation ] of the form where is Brownian motion. This…
The partial derivatives of a multivariate function form its Jacobian matrix The convention here (matching Wikipedia, and I believe also…
For any [ convex ] function and probability distribution , Jensen's inequality states that The special case of a distribution over two…
References: The KLF: Chaos, Magic and the Band who Burned a Million Pounds (2013), by [ John Higgs ].
Given a [ constrained optimization ] problem over a [ convex ] function , we consider the [ Lagrangian ] function introducing variables…
This is a block [ Kronecker product ]. Given block matrices their product is Unlike the Kronecker product, the Khatri-Rao product is…
The Kraft inequality in information theory states (roughly?) that, for any probability distribution , there is a prefix code C under which…
Given two matrices , the Kronecker product is the block matrix that 'tiles' across the entries of : It can be generalized to block…
LSD is essentially rigidified [ tryptamine ] — whereas the tryptamine side chain swings loose in other psychedelic tryptamines, in LSD (and…
Good tutorial: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
We're given a [ constrained optimization ] problem Note that the standard formulation of Lagrange multipliers handles only equality…
Yann Lecun's famous cake analogy is that: "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake…
References: Jess Riedel on the Legendre transform in physics Stack Overflow discussion Prof. V. Balakrishnan on Hamiltonian dynamics…
A function is -Lipschitz continuous if changing the input by a little bit can't change the output by more than a proportional amount…
References: Wikipedia: https://en.wikipedia.org/wiki/London_dispersion_force Khan academy: https://www.khanacademy.org/science/ap…
Some ideas from this paper. Binary positional encodings . Memory index is represented by a binary vector . This has the property that…
Used in analyzing the stability of an equilibrium of a dynamical system. A Lyapunov function is a scalar-valued function of the state space…
Methylene-dioxy-[ methamphetamine ]
There are some suggested supplement regiments for taking [ MDMA ] safely: https://rollsafe.org/mdma-supplements/ https://matznerd.com/how-to…
A [ stochastic process ] in which the past is independent of the future, conditioned on the current value. Striking point made by https…
Minerva : they basically fine-tuned a language model on ArXiv papers and web pages containing LaTeX, so that it can produce latex. This is…
Nicotinamide adenine dinucluotide hydride. Consists of two [ nucleotide ]s---one with an adenine nucleobase, the other with nicotinamide…
One of three types of [ ionotropic receptors|ionotropic ] [ glutamate ] receptor, along with [ AMPA receptor ]s and kainate receptors. It is…
Gilmer at al. paper 2017 Experiments on QM9. Unlike SMILES strings, includes molecular geometry . General formulation of message passing…
NameRedacted wrote some nice thoughts about existential risk from artificial superintelligence (ASI): https://michaelnotebook.com/xrisk…
Notes from Charles Margossian's talk on pharmacometrics models. Types of ODEs: Linear: can be solved by [ matrix exponential ]ials nonlinear…
It seems like all of my Roam problems are solved. I can write real posts, with paragraphs. I can use latex inline: ! I can…
If two hypotheses are equally consistent with the data, the simpler is more likely to be 'true'. Formally, it is more likely to generalize…
Here's a formal argument for [ Occam's razor ], adapted from Theorem 2.2 of Kearns and Vazirani's "An Introduction to Computational Learning…
Useful and non-obvious #life-advice, from https://www.theguardian.com/lifeandstyle/2020/sep/04/oliver-burkemans-last-column-the-eight…
Most important body parts are eyes, hips, feet. Eyes are most important: don't look at your feet, look down the mountain (this dovetails…
[ explicit models of uncertainty ] (8/28/20) Standing up for myself (8/28/20): It's much easier to feel righteous about standing up for my…
See [ generative vs discriminative modeling ], [ actor-critic ]
I'm trying to build my understanding. These are fragments of intuitions. Bayesian inference starts with a prior P and a likelihood. Given…
This page contains my notes from the Pointing Out the Great Way Level 1 [ mahamudra ] [ meditation ] [ retreats|retreat ] course I took…
Attempt to outline [ Dan Brown ]'s version of the [ mahamudra ] path to awakening: 'Ordinary' emptiness practices: Emptiness of self. Search…
Predictions for: April 23ish (one month from now): Cases in the bay area will have inflected, and at a relatively lower level than NYC or…
Chapter 2: why an animal needs a brain. Brains allow for control and adaptation within a single lifetime. They make more sense for bigger…
[ Project Management Checklist ] {{TODO}} Define project Objectives:: Deliverables:: Key performance indicators:: Scope:: Budget…
Following the pattern of [ state values, then action values ], the one-step [ temporal difference ] update for action values is called…
Fell victim to (or helped commit) a spiritual fraud, Joya Santanya. If Ram Dass can't tell if spiritual teachings are legit, does that…
Very simple brainstorming Say we want to search over STS architectures. An STS architecture here is really just a list of components, with…
Basic of quantum mechanics from Wikipedia : The quantum state of a system is a ray (equivalence class of vectors) in a separable (has a…
The theme of the book is that top-down control necessarily involves abstraction, and often, changing existing systems so that they are more…
The 'universal prior' gives probability to all sequences that can be generated by a -bit program. Generally a sequence is generated by…
I just saw a paper by Tom rainforth L on steca stick conditioning and I wanna try to articulate what you could mean by that and what I think…
The stuff in marijuana is delta-9-THC, where the '9' refers (somehow?) to the position of the double bond in the cyclohexane ring?
On the Robot Brains podcast , Andrej Karpathy explained to Pieter Abbeel why he thinks Tesla has the right approach to self-driving. Tesla…
Turning and turning in the widening gyre. The falcon cannot hear the falconer. Things fall apart; the center cannot hold. Mere anarchy is…
Notes from working through Kevin Buzzard's Natural number game (imperial.ac.uk) using the Lean theorem prover. We know from the [ Curry…
Massive list here: https://github.com/cedrickchee/awesome-bert-nlp Bahdanau, Cho, Bengio. Neural Machine Translation by Jointly Learning to…
Alan Turing proposed that we might consider a machine to think if it could succeed in an 'imitation game'. In one variant of this game, a…
The -Wasserstein distance between probability distributions is defined as where the infimum is over all joint distributions having…
Who are you? Hi, I'm Dave. I'm interested in the secrets of intelligence, consciousness, the good life, and other easy questions. I have a…
Abstraction is lossy [ compression ]. A good abstraction throws away everything not relevant to a particular problem, while preserving a…
An acetyl group is - (C = O) - CH3 Attaching an OH gives acetic acid.
A single big goal is very difficult and risky to achieve and, in the time you're working on it, you will not have achieved any goals at all…
This is really the only advice I know of for [ how to make a friend ]. And it works. Invite someone to social events, as you would a…
(references: https://julien-vitay.net/deeprl/ActorCritic.html ) Advantage actor-critic The advantage function is a 'centered' version of…
Imagine you want to make decisions in the world. One way is to learn how the world works, consider possible approaches, and think through…
In reinforcement learning, the advantage of a state-action pair under a policy is the improvement in value from taking action a (and…
Here are some high-quality ways to have fun: Things I do now and should do more of: Ski trips with friends Traveling athletic events (races…
I don't even know where to start. This concept shows up everywhere . Many human systems are constructed from adversarial mechanisms. This…
Take notes in a personal knowledge manager like [ Roam Research ] or [ Obsidian ] that makes it easy to jot down fragments of ideas and…
It's natural to fault someone who fails to follow advice, especially when the results are predictably bad. It can be helpful to notice and…
see also [ agent ]
There is a similarity in kind between the negative effects of: rape slavery feeling forced to work on someone else's projects, by a boss…
see also [ agency ]
[ ethanol|Ethyl alcohol ] is by far the most harmful drug that most people regularly consume. It is almost certainly way more harmful than…
Fundamentally an algorithm is any computational procedure: something that takes in data and spits out some function of that data. Computer…
Fundamental observation is that [ the map is not the territory ]. BUT---there is a real and important distinction between models that are…
This was one of the beliefs that SuccessfulFriend and I shared (and still share). This seems like a cynical thing to say. But it's not meant…
One of Paul Graham's heuristics for doing what you love is to 'always produce'. That resonates with me: it's a way to keep yourself honest…
Churchill, 'My Early Life': Come on now all you young men, all over the world. You are needed more than ever now to fill the gap of a…
Structure: NH2 - C - COOH (amine - C - acid/carboxyl), usually with a side chain hanging from the central (alpha) carbon. Roughly speaking…
Shorthand for alpha-methyl[ phenethylamine ]. Adding another methyl group to the (n)itrogen gives n-methyl-amphetamine aka…
I used to play board games with ExBoyfriend and his friend Will. Will was literally getting a PhD in game studies, and almost always beat…
In the published version of these notes, names are anonymized to protect the privacy of friends and colleagues.
Most [ argument ] that people engage in isn't formally persuasive. It's a back and forth of giving reasons: I give a reason to do a thing…
see also [ communication is processing ] Developing an idea with someone else applies regularization to the idea. It guarantees that the…
Intelligence is what makes humans special. PhdAdvisor defines it as the ability to make high-quality decisions. I think we can distinguish…
References: https://en.wikipedia.org/wiki/Alcohol_flush_reaction There seem to be three genes associated with poor alcohol tolerance…
To watch: The Untamed: https://www.youtube.com/watch?v=SlpEVsKiYCU HIStory (Taiwanese boy love) HisTory 2: right or wrong: https://www…
In a work context: Consider a junior engineer and a senior engineer. The junior engineer says, 'I think doing X will be better than doing Y…
One of the best ideas in machine learning. (I even thought so in 2011!) There are two common mechanisms: 'soft' and 'hard'. In both cases…
Thesis: [ attention ] is the dominant factor in our utility. It follows that work that shifts our attention can genuinely improve global…
A paradox maybe worth remembering: if I want to build a conscious identity around something, then I have to be willing to feel inauthentic…
This is my stab at explaining automatic differentiation, specifically backprop and applications to neural nets. A few dimensions to think…
[ Dan Brown ] uses this term a lot. Loosely, I think it means [ awareness ] that is aware of itself, without [ fabrication ]. Like…
Here's a perspective I like on spiritual awakening, elaborating on a metaphor used by NameRedacted. Suppose you awaken from a dream: a…
As investigated in [ nondual ] or [ mahamudra ] meditation practice, awareness is both the 'field' in which all of experience occurs, and…
Matt Levine points out a basic fact of derivatives theory: a basket of options is worth more than an option on a basket. If five stocks…
I've sometimes felt a scarcity mindset with respect to friendship. Like: friends and especially close friends and relationships are things…
What would I say to someone I met up with on Tindr? How do I think about the encounter? I like almost everyone, so in particular I like you…
just in case I'm ever a professor/teacher and forget the personal side of what people might be going through: as a freshman: I started out…