SuccessfulFriend highlighted this distinction which I should really read more about. At a high level it's about the distinction between…
Language is a really natural way to tell AI systems what we want them to do. Some current examples: [ GPT ]-3 and successors (InstructGPT…
In chemistry, a salt is a neutral-ish (not too acidic nor basic) compound held together by an [ ionic bond ]. Salts can be formed by [ acid…
Scheduled sampling is a training procedure for sequence models that attempts to mitigate [ exposure bias ] - the problem in which generation…
The score function is the gradient of a log-density with respect to its parameters: It is the direction that we would move the parameters…
Aapo Hyvärinen: Estimation of Non-Normalized Statistical Models by Score Matching (2005) https://jmlr.org/papers/volume6/hyvarinen05a…
[ unearned confidence ] [ agency and confidence ]
Sometimes it's necessary and right to prioritize my own interests, even if [ global utility ] is ultimately the only metric. Developing…
I need to genuinely care about other people and want the best for them, both in general, and for specific people in my life. Why? Obviously…
Traditional Buddhism describes six "sense bases" or gates: the eye, ear, nose, tongue, body, and mind. Western science usually omits the…
When we talk about "the self", or having a "sense of self", what do we mean? There is an interpretation in terms of [ consciousness ] - that…
Take the statement 'human-level AI is possible'. As a kid, I saw this as obviously true. We can simulate physics, and brains are physical…
There are 14 kinds of serotonin receptors; most (but not all) are [ G protein ]-coupled. The central nervous system has almost all of them…
Shadow work means, roughly speaking, a practice of noticing, loving, and integrating the parts of yourself that you've repressed (your…
Shard theory's basic ontology of RL holds that shards are contextually activated, behavior-steering computations in neural networks…
References: https://generative.ink/posts/simulators/ It seems pretty clear that the intelligence emerging from [ language model ]s is not…
The performance of an investment can be modeled as where the 'market return' is that of some sufficiently broad index such as the S&P 50…
In Korea (and maybe also Japan?) it's common for young guys to bond through physical touch and affection: hugging, holding hands, sitting in…
It's weird that we lie down every day to cease our consciousness, and sometimes to hallucinate. There are physiological benefits to sleep…
It's not a terrible summation of [ depression ] that it starts from seeing no way to achieve your goals. Sometimes that's because it's…
I've been really unhappy about how TFP is developed. It's felt pedantic. I waste a lot of effort thinking about things I don't want to think…
This is the advice I wish I'd had. It's catered to my preferences; caveat emptor. Packing: (for men) bring one pair of long pants, for…
paper 1: http://redwood.berkeley.edu/bruno/papers/VR.pdf basic idea: find a basis such that any given image (or whatever signal) can be…
References: Jacobs, Jordan, Nowlan, Hinton. Adaptive Mixtures of Local Experts (1991) Shazeer et al. Outrageously Large Neural Networks…
I used to be able to say 'superintelligent AI is possible'. Now in industry the notion of 'possible' is 'something I can myself do': by…
What would true wireheading feel like? People have this impression that it'd be thin, exhausting, artificial, ultimately isolating and not…
A stablecoin is a dollar-denominated liability registered on a [ blockchain ]. It can be backed by USD reserves, as Tether allegedly is…
Julian Shapiro recommends keeping a set of six 'Starting principles' that you use to make decisions. That's about all that you can…
A common pattern in [ reinforcement learning ] pedagogy is to develop some idea first in the context of estimating state values , and then…
A [ stochastic process ] is (strictly) stationary if all of its joint distributions are invariant under time displacement. It is wide…
I've always been an evening person more than a [ morning person ]. I often stay up until 1 or 2am, and in the absence of hard constraints it…
Getting language models to align their output with human preferences would be highly useful for [ computational life coach ]ing. What's the…
SDEs are typically written in terms of the differential of a Weiner process (Brownian motion), e.g., Although Weiner processes are nowhere…
A stochastic process is a collection of [ random variable ]s defined on a common [ probability space ] . Equivalently, it is a joint…
Pasting a quote from Adam Smith by way of HN (source http://www.econlib.org/library/Smith/smMS7.html , I should read the whole thing…) that…
A stopping time for a stochastic process is a time-valued That is, integer-valued for discrete-time processes and real-valued for…
A lot of confused discussion around large organizations comes from conflating individual motivations with larger-scale 'structural…
In kindergarten stats, you learn how to build a model that takes in data (a feature vector, image, sound file, etc) and predicts a single…
Revolutionary ideas must live in the blind spots of the current intellectual conversation; otherwise people would already be using them…
Note naming The general goal is to minimize the use of aliasing in links. In case where these guidelines suggest an unnatural or uncommon…
substantive questions I've had these are things I've wondered about that were never answered properly in the classes in which I learned them…
Substituted tryptamines - PsychonautWiki Tryptamine consists of an [ indole ] moeity plus a two-carbon (ethyl) chain with an amine group. We…
A sugar is any molecule with the empirical formula C(N)H(2N)O(N). These are like alkanes, which are C(N)H(2N + 2), except that each carbon…
A -dimensional vector can represent distinct orthogonal features, but due to the weirdness of [ high-dimension ]al geometry, it can…
What have I learned in 2.5 years at Google? What did I not realize? The model of research. How low the expectations are. How fake it felt to…
Refs: https://opentheory.net/Qualia_Formalism_and_a_Symmetry_Theory_of_Valence.pdf
My Supernote A5X syncs through Dropbox, but unfortunately Dropbox doesn't support Windows ARM64 machines like the Surface Pro X. Here's my…
Buddhist (Pali) term referring to craving, longing, desire for the world to be other than as it is. This includes craving good things and…
There's a famous quote attributed to Eleanor Roosevelt: "great minds discuss ideas, average ones events, mediocre ones discuss people". This…
A set of methods for maintaining an " attitude of spacious passion ". The particular methods are contingent; if you could maintain the…
A general issue with [ temporal difference ] learning methods, which 'update a guess towards a guess', is that they can end up 'chasing…
Something that confused for me for a while is that people in certain communities talk about 'teacher forcing' as though it's a trick or a…
Dave's principles of effective teaching. Motivation is by far the most important thing. A student who wants to learn will learn even with a…
As a researcher, I wonder if there's a 'critical point' of growing an idea when it's important to be [ teaching ] it, whether formally or…
working with Sinclair, Klein, Abbeel, they’ve all got great experience and advice especially for large classes You don’t have to give the…
Rob wants to firm up his foundations. He wants to understand relevant stats, probabilistic models, inference, and maybe work our way up to…
Epistemic status: either this is true or TV is maybe one of the greatest contributions to human utility ever. Unclear. The average American…
From David Silver's slides : TD-learning 'updates a guess towards a guess'. Sutton and Barto define the temporal difference error as the…
This page (first brainstormed in an Otter note) is for issues where I feel pulled in several directions. Different principles seem to yield…
Every in machine learning talks about tensors, but no one really understands what they are. This page collects several definitions and…
The tensor product of two vector spaces (defined on the same scalar field, we'll assume ) is the vector space of formal sums of…
This post gives a nice, mathematically clear development of basic terms in statistical mechanics. Highlights: Think of a physical system as…
thai holy basil / hot basil thai basil turmeric rice noodles: thin (pad thai) or wide (pad see ew / pad kee mao) oyster sauce, fish sauce…
Status: in conflict with [ negative utility ] ? See also: the [ hedonic treadmill ]. Evolutionary, 'pain' exists to motivate you to get out…
I used to think that there was a 'best' way to motivate an area. For example, in VI, the ELBO is derived from the KL divergence between a…
NameRedacted often mentions this as a thing Sharon Salzberg would say. What does it mean? I don't know - I should ask Tucker to clarify…
It's almost never worth worrying about whether an individual action is the right thing to do. It's like trying to dance while worrying at…
In order for a group of people, like an academic field, or a political elite, to meaningfully converse about a complex topic, they have to…
Metaphor connected to the observation that [ all models are wrong ]. Borges, On Exactitude in Science : ...In that Empire, the Art of…
A point made by NameRedacted in various talks, e.g. The World is Inside You (also the '[ emptiness ] of perception' described by [ Dan…
Comparing myself to SuccessfulFriend, I might be tempered to think that because he is interested in antitrust law, zoning reform, political…
Andrew Gelman believes that in certain areas of research , like the social sciences, everything is connected. "I’m not expressing…
In every field, there is a store of 'standard' advice that is handed down from mentors to ambitious youngsters. In computer science grad…
The Feynmannian/Sagan/Tyson "scientific" view is that the [ purpose ] of life is understanding : the world is a giant mystery, with layers…
It exists, but is [ empty ], insubstantial, a [ fabrication ]. Foregrounding this view is an important part of [ awakening ] or…
[ things are deeply wrong ]
If I'm managing someone, I want them to be coming up with their own ideas and owning them. Owning their ideas means they will themselves…
tl;dr : the ideas we need to build intelligent systems may be different from those we need to understand them. Both are important, but…
Several ideas here: When I try to tell a story about what I'd like to change about my life, at a high level, I can come at it from different…
For several reasons: multiple object-level causes a telescoping tower of causes at increasing levels of generality or abstraction 'because…
If two statements that both seem true conflict with each other, then it seems like you have a paradox. But the world itself is just as it is…
Pointed out in this tweet: https://twitter.com/AmandaAskell/status/1311776280128479238 but also in many other places over the years…
AI is going to work. Obviously lots of people believe this. But most 'AI' companies and 'AI' investors are hyping applications of current…
No matter what other priorities or any incredibly important goals arise in my life, whether through work, family, or other circumstances…
Write Write regularly: under routine circumstances, at least a few minutes per day. This could be filling in nodes of this graph, blogging…
See also: [ the system is bad ] I find it hard to be okay with a 'normal' life, because that would imply some level of acceptance of the…
These might not be the best thing to do at any point, but they're better than doing nothing. And doing them can create a sense of progress…
See also [ writing inbox ] See also [ ongoing projects ] See, first and foremost, the backlinks below. Crypto trading model. Write a system…
This is one of those things that sounds cliche but is still profound and #fundamental: this is all. There's no great reward in the future…
I want kids, eventually. I want to be able to talk with them, to build a relationship, to see the world through someone else's eyes. I want…
The [ agent ] model of intelligence imposes a sharp distinction between the agent and its environment, where the agent 'chooses' actions…
let's say the signal we see after the intervention is modeled as the combination of the counterfactual forecast and an intervention effect…
[ impermanence ] [ dukkha ] (unsatisfactoriness) [ no-self|annita ] (no-self) Daniel Ingram's summary: things "come and go, don't satisfy…
(fellow student) Smitha has these post-its on her desk: what are you doing? why is it important? are you making progress? I think these…
This is personal mental image for [ emptiness ] that has been really resonant for me, arising from an experience taking [ MDMA ] with a…
It helps a lot to write down the things you think someone should know about working in a new environment. Even if a new person would figure…
I feel an obligation to try to do big things with my life, because I've had access to rare opportunities. If ten thousand randomly selected…
Television: Arcane ken burns on the vietnam war WandaVision For All Mankind Severance: https://m.imdb.com/title/tt11280740/ Borgen Diplomat…
How should a machine learning model represent text? Word-level and character-level features are obvious options, but both have drawbacks…
Sometimes mentioned as a potential approach to [ AI safety ]. Gwern: Why Tool AIs want to be Agent AIs (roughly: because treating…
Notes on Toolformer: Language Models Can Teach Themselves to Use Tools The basic method is: "Given just a handful of human-written examples…
Trace of a Linear Operator We define the trace as the sum of diagonal elements of a matrix: Lemma : If and are square, then . Proof…
There are three main approaches to moral philosophy: [ utilitarian ]ism: you should feed a starving person because it will increase 'global…
These days we think a lot about using data to train large [ language model ]s. But there's only so much data in the world; eventually we'll…
I didn't have a good intuitive understanding of the social landscape of being a researcher (and joining a [ research community ]). When…
If you and I agree of our own volition to exchange X for Y, this implies that we both believe we are gaining value in the trade. If one of…
The core of the transformer architecture is multi-headed [ attention ]. The transformer block consists of a multi-headed attention layer…
What does the computational profile of a transformer vs a similar RNN look like? First, the transformer. Let's take the LLama 6.7B model…
In developing intuition about [ transformer ]s it's useful to think about specific primitive operations that can be implemented by a small…
Incorporating explicit memory and retrieval seems pretty clearly like the next frontier in language modeling and AI more broadly. We have…
According to this reddit post , one of the main takeaways of functional analysis is that the right way to interpret the 'transpose' of a…
SSC link: How general is this phenomenon? You have a belief Your belief colors your perception of something that doesn't inherently…
A pitfall with relying too heavily on rational deduction is that lots of logically 'true' conclusions are unimportant, or worse yet…
(notes loosely based on the Berkeley deep RL course lecture ) Setup: RL with policy gradients The basic setup is that we want to optimize…
Language is an incredible bottleneck. There are infinitely many true facts about the world, even just in pure math, and yet we communicate…
The reason to try new things is not really because the new things themselves are more exciting than the old ones. The reason is that it…
From Jeff Bezos' 1997 shareholder letter : Some decisions are consequential and irreversible or nearly irreversible – one-way doors – and…
Inspired by Kevin Buzzard's overview of the state of automatic theorem provers. Type theory is like set theory in that sets and types are…
nostalgebraist argues that unconditional love can't and shouldn't exist : A parent might love their child "unconditionally," in the well…
Confidence is a positive [ feedback loop ]. You need a certain amount of confidence to try something hard. Success breeds more confidence…
It's a basic law of probability that, given two events A and B, the probability that at least one of them occurs is given by This counts the…
(this note expresses a tendency that I notice in myself. I don't necessarily endorse this tendency but I think it's interesting to…
From a review by [ Oliver Burkeman ] of Jordan Peterson's "Beyond Order" ( https://www.theguardian.com/books/2021/mar/02/beyond-order-by…
like a 'useful perpective', but 'lens' implies focus or distortion whereas 'perspective' implies linear projection. Related to [ many models…
Consuming unstructured content from the internet is addictive. Twitter is full of life advice, interesting technical discussion, takes on…
Suppose I have an agent that generates text. I want it to generate text that is [ value alignment|aligned ] with human values. Approaches…
Notes on the Alignment Forum's Value Learning sequence curated by Rohin Shah. ambitious value learning : the idea of learning 'the human…
The standard [ Markov decision process ] formalism includes a reward function ; the total (discounted) reward across a trajectory is its…
How should people do VI? One ultimate goal is that you write a Stan model (or better, a model with discrete variables, but one step at a…
Holy shit. In December on Galiano I was brainstorming about [ continuous structure learning ] and thought of the general trick, for…
Inspired by NameRedacted, I'm considering going 'mostly' vegetarian. What would that mean for me? I don't myself buy meats or dairy products…
Why am I doing all of this? If I carve aside hours or days or months to 'fill in' my graph of notes, what am I hoping to get from it? Why is…
Ref: https://arxiv.org/abs/2010.11929 We start by chunking an image into patches, and concatenating each patch with a position embedding…
Telling people about your failures, your fears, your self-doubt, your insecurities can be a path towards deeper connection. Understanding…
How to be warm: https://www.youtube.com/watch?v=1MolmoFuXu4&t=123s
The "strength of weak ties": most good things in life come from people you barely know. This is because your close, regular connections are…
Thinking through: Why the toughest capitalists should root for a wealth tax ( https://www.ft.com/content/e1adf707-b95a-4422-9211-1841cd7ce…
Moxie Marlinspike on web3: https://moxie.org/2022/01/07/web3-first-impressions.html We know that people do not want to run their own…
[ weekly review ] • Plus: What went well? • Minus: What didn't go so well? • Next: What will I focus on next week?
Reference: Mahmood et al., 2014. Weighted importance sampling for off-policy learning with linear function approximation Here's a situation…
I suspect many of these are evergreen. I'm not [ writing ] enough. I'm not keeping up a regular journaling practice.
Just like norms in the Trump administration, there are mental habits, rhythms of life, attitudes towards the world, that are powerfully…
In the course of any person's life, you take in a vast amount of information. You have your own personal experiences, of course, and you…
See also: [ if ever a prof ], [ advice for college students ] Things not directly related to course material that I wish I'd learned earlier…
What will I do when I don't have a job? I don't feel that I have a clear direction. I want to learn and explore. There are lots of [ my…
A story from [ Dan Brown ]: A group of psychologists came to interview the Dalai Lama, the spiritual leader of Tibet. One of the Americans…
As a kid, we learned about https://en.wikipedia.org/wiki/The_Game_(mind_game) : if you think of the game, you lose. (and have to say "I…
From 2017: wisdom I've acquired: the psychology of depression. :-( and grad school. :-( and being gay. [ dual-process cognition ] theory…
“It was true that I didn’t have much ambition, but there ought to be a place for people without ambition, I mean a better place than the one…
This may be a central point of confusion: how do we define AI systems that have preferences about the real world , so that their goals and…
In software: a library is a collection of tools. You can use some or all of them, in combination with other tools. A framework , on the…
your writing needs to be at the edge of your knowledge, it needs to address the most fascinating people you know or can imagine. That is…
Quote I like from Manuel Blum's advice to grad students , connecting writing to the power of [ Turing machine ]s: STUDYING: You are all…
What is the philosophy of the project? What principles is it betting on? Example from Ben's Ads doc: iterating on an end-to-end pipeline…
Regular writing practices that would be valuable. [ prediction as a model-building exercise ]
The models we use in AI are [ all models are wrong|wrong ] (if maybe still useful). How? Agency The [ agent ] model assumes a separation of…
status: a theory that feels true for my personal trajectory. Totally uncritiqued and unverified that anyone else shares this experience…
Sometimes it's daunting how much knowledge there is in the world. For any given area, there are a thousand specialties and subspecialties…
Something that SuccessfulFriend said today: It's rare that someone totally independent comes up with a really good idea. The best ideas come…
A zero-knowledge proof allows a prover to demonstrate that it possesses certain information, without revealing that information to the…
A zk-SNARK, or zero knowledge Succinct Non-interactive Argument of Knowledge, is a [ zero knowledge ] proof system that is non-interactive…