The law says that: when a measure becomes a target, it ceases to be a good measure . One can distinguish four types of Goodhart problems…
Modified: April 08, 2023.
Abstraction is lossy [ compression ]. A good abstraction throws away everything not relevant to a particular problem, while preserving a…
Modified: February 20, 2022.
It's natural to fault someone who fails to follow advice, especially when the results are predictably bad. It can be helpful to notice and…
Modified: March 19, 2024.
I used to play board games with ExBoyfriend and his friend Will. Will was literally getting a PhD in game studies, and almost always beat…
Modified: January 18, 2021.
Most [ argument ] that people engage in isn't formally persuasive. It's a back and forth of giving reasons: I give a reason to do a thing…
Modified: January 23, 2022.
see also [ communication is processing ] Developing an idea with someone else applies regularization to the idea. It guarantees that the…
Modified: March 04, 2022.
In a work context: Consider a junior engineer and a senior engineer. The junior engineer says, 'I think doing X will be better than doing Y…
Modified: February 10, 2022.
I got this concept from Matt Johnson. People overestimate what they can do in a week, but underestimate what they can do in ten years. To do…
Modified: January 24, 2022.
Modified: January 07, 2022.
Modified: January 09, 2021.
Talking and writing are not nearly as much about communication as we think. That's part of it, of course. But a significant portion, often…
Modified: March 04, 2022.
(below was originally an email to SuccessfulFriend, copying here for posterity) learning as compression seems like kind of a folk idea with…
Modified: September 06, 2021.
Related: Learning increasingly complex ideas may amount to forming larger effective chunk sizes Expertise requires increasingly…
Modified: May 04, 2020.
Multiple people working together are fundamentally more powerful than a single person working alone. Governments recognize this fact, which…
Modified: March 04, 2022.
All else equal, constraints prevent you from doing what you'd have otherwise wanted to do, which is bad. But. Constraints prevent [ analysis…
Modified: January 09, 2021.
Okay so there’s a lot of research on what conversations are, what the goals are (of course I don’t know most of this research…). It seems as…
Modified: February 13, 2022.
Which is more useful: reading the New York Times every day, or reading John Stuart Mill? listening to a current-events podcast, vs listening…
Modified: January 10, 2021.
[ Otter notes ] August 2020: When somebody says that X is good---here X could be love relationships, money, peace, or whatever---it is never…
Modified: September 11, 2021.
Elephant and rider Asking "what should I value?" is asking the rider. it demands solving impossible moral questions. Asking "what do I value…
Modified: July 08, 2020.
Events that seem really terrible---closing off good outcomes and potentially leading to bad outcomes---often refine into a fine path that…
Modified: February 25, 2022.
Modified: February 10, 2022.
I grew up in the 2000s reading the New Atheists, where 'faith' was considered a dirty word. Faith was the opposite of reason; it meant…
Modified: November 14, 2022.
[ Andre ] points out the strong free will theorem . This says that electrons sometimes have 'choices': situations where their behavior is…
Modified: October 04, 2021.
Why would you be 'proud' of something you had no control over? The core revelation for me was that pride is the opposite of shame . Most…
Modified: January 20, 2022.
these include things like: posing interesting questions for discussion useful [ generative questions ] like: what have you been reading…
Modified: February 25, 2022.
Opposite of a fixed or 'scarcity' mindset. It's important to recognize that the world is nonzero-sum and that improvements are possible. We…
Modified: January 09, 2021.
I'm sometimes tempted to look back and find patterns in my life, and identify those as "who I really am". For example: maybe I want to be a…
Modified: February 22, 2022.
To achieve final goals, we have to break them down into a hierarchy of instrumental goals, and then get to work on achieving those. And for…
Modified: February 25, 2022.
A lot of discussion around [ artificial intelligence ] implicitly conflates intelligence with [ consciousness ]. It assumes that as we…
Modified: January 18, 2023.
Interfaces enable modularity. In general, standardizing an interface can yield quadratic benefit at linear cost. Suppose we have people…
Modified: May 16, 2022.
I don't want to be led. I want to be creative and do things that are dramatically new. Telling other people what to do feels almost evil to…
Modified: February 25, 2022.
https://maskofreason.files.wordpress.com/2011/02/the-library-of-babel-by-jorge-luis-borges.pdf Insights illustrated by this story: Naming…
Modified: October 17, 2022.
There's a tendency to focus on things that we have the (conceptual/mathematical/societal) tools to understand, even when we know this is…
Modified: February 10, 2022.
I saw this phrase on Twitter somewhere and it really resonates as a description of the ideal approach to science. There is no real…
Modified: May 16, 2022.
An idea I got from [ John Higgs ]'s discussion of metamodernism is that taking [ all models are wrong ] to its logical conclusion requires…
Modified: January 06, 2023.
Like democracy , meritocracy is the worst form of social organization, except for all the others that have been tried. Of course it is good…
Modified: December 01, 2022.
One last thought mental models are so, so important. When I think about computer modeling. It's actually great computers are powerful they…
Modified: July 25, 2020.
This is one of the big problems with the world. Not the only one, and not the only way to look at it. But it's everywhere. status : a…
Modified: January 25, 2022.
(see David Graeber https://www.strike.coop/bullshit-jobs/ ) Most work is oriented towards achieving [ instrumental goal ]s. But most…
Modified: February 25, 2022.
One way to model real-world [ causality ] is a bunch of forces working with and against each other. In this view, no individual force…
Modified: July 14, 2023.
Something can be true but not 'true enough'. That is, you have a compelling causal theory for why X should increase Y. It might be that the…
Modified: August 21, 2020.
As Josh Marshall said , at the beginning of the Trump presidency: "Optimism is not primarily a prediction but an ethic, a philosophy, a way…
Modified: June 08, 2021.
A really valuable exercise that I should consider building into my routine is to regularly try to make and write down explicit predictions…
Modified: January 24, 2021.
Illegible privilege We often talk about the 'privilege' associated with certain categories: being born white, straight, male, rich, in a…
Modified: April 04, 2024.
The idea of 'projection' in psychology means to assume that someone else has the same flaws, or foibles, or motivations as you do. It struck…
Modified: February 25, 2022.
One model you could have of reading a book is that the book contains information, and once you've read it, you now possess that information…
Modified: February 23, 2020.
Why do I want to write more? Because: writing forces thoughts to crystallize. It forces me to draw conclusions about what I believe and who…
Modified: May 16, 2022.
When I was younger---in college or in grad school---I was sometimes conflicted about whether I should prioritize trying to get to correct…
Modified: February 11, 2022.
SuccessfulFriend highlighted this distinction which I should really read more about. At a high level it's about the distinction between…
Modified: July 13, 2020.
It's not a terrible summation of [ depression ] that it starts from seeing no way to achieve your goals. Sometimes that's because it's…
Modified: May 16, 2022.
A lot of confused discussion around large organizations comes from conflating individual motivations with larger-scale 'structural…
Modified: December 14, 2022.
Revolutionary ideas must live in the blind spots of the current intellectual conversation; otherwise people would already be using them…
Modified: January 24, 2022.
Modified: October 03, 2021.
This page (first brainstormed in an Otter note) is for issues where I feel pulled in several directions. Different principles seem to yield…
Modified: February 10, 2022.
I used to think that there was a 'best' way to motivate an area. For example, in VI, the ELBO is derived from the KL divergence between a…
Modified: January 23, 2022.
It's almost never worth worrying about whether an individual action is the right thing to do. It's like trying to dance while worrying at…
Modified: May 16, 2022.
In order for a group of people, like an academic field, or a political elite, to meaningfully converse about a complex topic, they have to…
Modified: February 25, 2022.
The Feynmannian/Sagan/Tyson "scientific" view is that the [ purpose ] of life is understanding : the world is a giant mystery, with layers…
Modified: February 25, 2022.
tl;dr : the ideas we need to build intelligent systems may be different from those we need to understand them. Both are important, but…
Modified: February 26, 2022.
If two statements that both seem true conflict with each other, then it seems like you have a paradox. But the world itself is just as it is…
Modified: January 23, 2022.
For several reasons: multiple object-level causes a telescoping tower of causes at increasing levels of generality or abstraction 'because…
Modified: February 07, 2022.
The [ agent ] model of intelligence imposes a sharp distinction between the agent and its environment, where the agent 'chooses' actions…
Modified: June 27, 2021.
These days we think a lot about using data to train large [ language model ]s. But there's only so much data in the world; eventually we'll…
Modified: October 27, 2022.
A pitfall with relying too heavily on rational deduction is that lots of logically 'true' conclusions are unimportant, or worse yet…
Modified: February 15, 2022.
Language is an incredible bottleneck. There are infinitely many true facts about the world, even just in pure math, and yet we communicate…
Modified: January 23, 2022.
From Jeff Bezos' 1997 shareholder letter : Some decisions are consequential and irreversible or nearly irreversible – one-way doors – and…
Modified: January 18, 2021.
update April 2024: I'm going to leave this here, but I now think about confidence in less of an information-theoretic belief way, and more…
Modified: April 16, 2024.
like a 'useful perpective', but 'lens' implies focus or distortion whereas 'perspective' implies linear projection. Related to [ many models…
Modified: January 17, 2021.
In the course of any person's life, you take in a vast amount of information. You have your own personal experiences, of course, and you…
Modified: June 12, 2021.
What is the philosophy of the project? What principles is it betting on? Example from Ben's Ads doc: iterating on an end-to-end pipeline…
Modified: June 25, 2022.
Quote I like from Manuel Blum's advice to grad students , connecting writing to the power of [ Turing machine ]s: STUDYING: You are all…
Modified: September 13, 2022.
Sometimes it's daunting how much knowledge there is in the world. For any given area, there are a thousand specialties and subspecialties…
Modified: April 17, 2022.