Most [ argument ] that people engage in isn't formally persuasive. It's a back and forth of giving reasons: I give a reason to do a thing…
Modified: January 23, 2022.
see also [ communication is processing ] Developing an idea with someone else applies regularization to the idea. It guarantees that the…
Modified: March 04, 2022.
Intelligence is what makes humans special. PhdAdvisor defines it as the ability to make high-quality decisions. I think we can distinguish…
Modified: March 02, 2022.
References: https://en.wikipedia.org/wiki/Alcohol_flush_reaction There seem to be three genes associated with poor alcohol tolerance…
Modified: January 21, 2023.
To watch: The Untamed: https://www.youtube.com/watch?v=SlpEVsKiYCU HIStory (Taiwanese boy love) HisTory 2: right or wrong: https://www…
Modified: May 16, 2021.
In a work context: Consider a junior engineer and a senior engineer. The junior engineer says, 'I think doing X will be better than doing Y…
Modified: February 10, 2022.
[ Hopfield network ] [ sparse distributed memory ] Milledge et al. (2022), Universal Hopfield Networks: A General Framework for Single-Shot…
Modified: March 29, 2024.
Modified: February 25, 2022.
One of the best ideas in machine learning. (I even thought so in 2011!) There are two common mechanisms: 'soft' and 'hard'. In both cases…
Modified: January 24, 2022.
Thesis: [ attention ] is the dominant factor in our utility. It follows that work that shifts our attention can genuinely improve global…
Modified: February 25, 2022.
A paradox maybe worth remembering: if I want to build a conscious identity around something, then I have to be willing to feel inauthentic…
Modified: September 27, 2021.
This is my stab at explaining automatic differentiation, specifically backprop and applications to neural nets. A few dimensions to think…
Modified: August 23, 2022.
Modified: .
The most important thing to know about aversion (avoidance / fear / ...) as an emotional response is that it's, at best, exponentially less…
Modified: April 20, 2024.
[ Dan Brown ] uses this term a lot. Loosely, I think it means [ awareness ] that is aware of itself, without [ fabrication ]. Like…
Modified: March 23, 2023.
Here's a perspective I like on spiritual awakening, elaborating on a metaphor used by [ Michael Taft ]. Suppose you awaken from a dream: a…
Modified: March 23, 2023.
As investigated in [ nondual ] or [ mahamudra ] meditation practice, awareness is both the 'field' in which all of experience occurs, and…
Modified: March 23, 2023.
Matt Levine points out a basic fact of derivatives theory: a basket of options is worth more than an option on a basket. If five stocks…
Modified: December 04, 2023.
I've sometimes felt a scarcity mindset with respect to friendship. Like: friends and especially close friends and relationships are things…
Modified: February 10, 2022.
Modified: .
What would I say to someone I met up with on Tindr? How do I think about the encounter? I like almost everyone, so in particular I like you…
Modified: November 28, 2021.
Modified: .
just in case I'm ever a professor/teacher and forget the personal side of what people might be going through: as a freshman: I started out…
Modified: February 10, 2022.
Imagine a really good person: someone whom everyone likes, is warm and friendly towards everyone they meet, gives freely of themselves, has…
Modified: April 01, 2022.
Related to: [ generative vs discriminative modeling ] The difficulty of [ learning new skills ] I remember learning to play the violin when…
Modified: May 01, 2023.
I used to believe in AI and machine learning. It was obvious to me as a student that the ability to create intelligent machines will be a…
Modified: September 18, 2020.
Modified: .
This year (2014-15) I served as a student reader on the Berkeley EECS PhD admissions committee. Berkeley typically gets about N…
Modified: March 06, 2022.
lagrange multipliers: www.eecs.berkeley.edu/~klein/papers/lagrange-multipliers.pdf legendre transfors / convex duality: http://student…
Modified: February 10, 2022.
Modified: .
I think of "variance" as the error in a statistical estimate that comes from not having enough data (assuming an [ identifiable ] model…
Modified: September 28, 2023.
The Atlantic has this article on "Masters of Love" http://www.theatlantic.com/health/archive/2014/06/happily-ever-after/372573/ presenting…
Modified: February 10, 2022.
I think this might be the wrong way to frame this. The field is CS, and the biggest problem in CS is AI. Within AI, the problems are…
Modified: September 07, 2020.
http://www.incompleteideas.net/IncIdeas/BitterLesson.html The bitter lesson is based on the historical observations that 1) AI researchers…
Modified: July 25, 2023.
(followup q: is 'blackpill' the best name? or is that a broader concept, there are blackpills other than the thing I'm describing, so maybe…
Modified: .
Modified: .
[ writing inbox ] the privilege of having correct advice work out for you ([ the privilege of advice working out ]) Let's stipulate that any…
Modified: February 10, 2022.
Jitendra Malik would often tell us that some idea or explanation makes sense if you 'blur your eyes'. This seems counterintuitive, but it's…
Modified: January 24, 2022.
Possible resources on tai chi, qi gong Paul Lam beginner video: https://www.youtube.com/watch?v=rbFzbS2lLT0 if it's good, can pay for his…
Modified: .
[ meditation ] and dharma: The Buddha's Own Words Mastering the Core Teachings of the Buddha Science of Enlightenment Seeing that Frees…
Modified: May 20, 2022.
Fiction memoirs of hadrian shogun red plenty name of the wind Cloud Atlas Lord of Light Tomorrow and tomorrow and tomorrow gay: Giovanni's…
Modified: July 05, 2022.
Modified: .
The four qualities of mind that it is impossible to have too much of, aka the four 'divine abodes' or immeasurables: [ loving-kindness…
Modified: May 19, 2022.
Sources: wikipedia GPT-4 https://ocw.mit.edu/courses/9-13-the-human-brain-spring-2019/video_galleries/lecture-videos/ Major parts of the…
Modified: October 27, 2023.
Antidepressent that acts as a [ norepinephrine ]-[ dopamine ] reuptake inhibitor. In the class of substituted [ phenethylamine ]s and (more…
Modified: August 27, 2022.
A nice paper that gets at some subtleties of calibration: Daniel D. Johnson, Daniel Tarlow, David Duvenaud, Chris J. Maddison. Experts Don't…
Modified: March 26, 2024.
Endocannabinoids are retrograde [ neurotransmitter ]s, meaning that they pass 'backwards' (from dendrite to axon) through the synaptic cleft…
Modified: August 24, 2022.
In the discourse around [ AI safety ] you sometimes see the claim that research on AI capabilities is harmful to the extent that it outpaces…
Modified: February 26, 2022.
Modified: February 10, 2022.
Modified: August 02, 2020.
The problem with work that's 'just a job' is that you'll never be as good at it as at work that really excites you, where you really want…
Modified: January 25, 2022.
Modified: .
Modified: January 23, 2022.
aka 'little love' or 'big love' or various other things Saved from https://www.carsieblanton.com/blog/post/82149148832/casual-love This…
Modified: February 22, 2022.
Modified: .
Modified: .
Modified: .
How do we infer [ causality ] from observational data? This question is important in science and is closely related to the progress of [ off…
Modified: July 05, 2022.
see [ causal inference ] Great Causality & ML Papers and Researchers – Blog (logangraham.xyz)
Modified: August 02, 2021.
We can view cellular respiration (and combustion more generally) from a high level as the transfer of electrons from carbon to oxygen atoms…
Modified: July 31, 2021.
Modified: March 13, 2022.
Modified: .
see: https://www.abolitionist.com/ https://qualiacomputing.com/2018/11/07/anti-tolerance-drugs/ [ suffering ] may seem inevitable. [ karma…
Modified: June 14, 2023.
Other things held equal. Abstract reasoning is good for coming to 'ceteris paribus' conclusions. It's easy to identify one force acting in…
Modified: May 23, 2021.
Modified: .
There are two major 'chain rules' relevant to machine learning: the chain rule of probability theory and the chain rule from calculus…
Modified: August 13, 2022.
Say you want to make a big personal change: to stop smoking, or to stop eating meat, or to meditate every day, or introduce yourself to a…
Modified: June 07, 2021.
I got this concept from Matt Johnson. People overestimate what they can do in a week, but underestimate what they can do in ten years. To do…
Modified: January 24, 2022.
Basic classes: NC (Nick's class) : Boolean circuits with fan-in 2 AC (alternating circuits) : Boolean circuits with unlimited fan-in for AND…
Modified: .
Also known as the Krebs cycle. Is the final common pathway for oxidation ('burning') of fuel molecules: carbs, fats, proteins. The fuel…
Modified: July 31, 2021.
More Is Different (kit.edu) On proof and progress in mathematics (Thurston, 1994) On Being the Right Size (Haldane, 1928))
Modified: March 02, 2022.
The [ distinction ] between classification and regression is, from one point of view, arbitrary: it's all just function approximation, and…
Modified: March 12, 2021.
Dan Klein once said I don't remember the context, but I took his NLP class in spring 2011 so it was probably around then. that he thought…
Modified: .
[ conceptual scaffolding ]
Modified: April 29, 2020.
Modified: January 07, 2022.
"find the thing that makes you come alive, and do that." I find myself thinking about 'giving up' in the sense of dreams of being an…
Modified: February 10, 2022.
Modified: February 07, 2022.
A commitment scheme allows for one party to publicly commit to some value without revealing that value. For example, Alice wants to bet on…
Modified: October 23, 2022.
Modified: January 09, 2021.
Talking and writing are not nearly as much about communication as we think. That's part of it, of course. But a significant portion, often…
Modified: March 04, 2022.
Modified: February 07, 2022.
Multivariate Completion of Squares A useful trick: if is a symmetric, nonsingular matrix, then This is easy to see just by expanding out…
Modified: March 16, 2022.
TO READ: Kronecker factored Approximate Curvature : K-FAC Practical Gauss-Newton : gives recursions for computing Hessian blocks, and…
Modified: October 30, 2020.
(below was originally an email to SuccessfulFriend, copying here for posterity) learning as compression seems like kind of a folk idea with…
Modified: September 06, 2021.
Arguably the core insight of deep learning / [ differentiable program ]ming is that the shape and structure of the computations we do are so…
Modified: September 13, 2022.
Modified: .
The view that performing the "right kind of computation" is necessary and sufficient for [ consciousness ]. Chalmers (1995): if a person's…
Modified: September 21, 2023.
Modified: .
**from “theory of computation as a lens on the sciences” ehud kalai talks about structural robustness in games: an equilibrium is robust if…
Modified: February 07, 2022.
How do you start building and selling [ computational therapy ]? It can't just be a medical product, because that's a hugely regulated and…
Modified: February 10, 2022.
See also: [ computational life coach ] A recurring dream I have is to use AI to solve mental health. It is simultaneously one of the most…
Modified: May 16, 2022.
Concentration is something of a misnomer for the meditative practice and states of samādhi . The word "concentration" carries connotations…
Modified: March 23, 2023.
Modified: .
Modified: July 08, 2020.
Related: Learning increasingly complex ideas may amount to forming larger effective chunk sizes Expertise requires increasingly…
Modified: May 04, 2020.
Nate Soares wrote an essay a while ago on how he experiences [ self-confidence ] when opining on difficult topics: On reflection, I've…
Modified: July 25, 2023.
https://markmanson.net/how-to-be-confident see also [ imposter syndrome ] and the thesis that a lack of confidence is often an aversion to…
Modified: July 25, 2023.
Modified: .
Modified: .
Philosophical views on consciousness: Buddhist and meditative traditions focus on [ awareness ]. They claim that consciousness has nothing…
Modified: July 14, 2023.
Multiple people working together are fundamentally more powerful than a single person working alone. Governments recognize this fact, which…
Modified: March 04, 2022.
Suppose we want to optimize an objective under some equality and/or inequality constraints, Some general classes of approach we can use are…
Modified: July 07, 2022.
All else equal, constraints prevent you from doing what you'd have otherwise wanted to do, which is bad. But. Constraints prevent [ analysis…
Modified: January 09, 2021.
it seems clear that forcing [ transformer ]s to produce discrete tokens is a significant constraint on their reasoning ability. tokens…
Modified: .
Relevant papers: DIfferentiable compositional kernel learning for Gaussian Processes (Sun et al., 2018) Differentiable Architecture Search…
Modified: March 07, 2020.
A contraction mapping on a metric space is a function such that for all and for some , called the [ Lipschitz ] constant of the map…
Modified: August 13, 2022.
A method for fitting an unnormalized probability density (aka [ energy-based model ]) to data. Note that this is a different and harder…
Modified: May 15, 2021.
A technique for [ representation ] learning in which semantically similar datapoints are encouraged to have similar representations, and…
Modified: April 30, 2022.
Modified: January 16, 2022.
Okay so there’s a lot of research on what conversations are, what the goals are (of course I don’t know most of this research…). It seems as…
Modified: February 13, 2022.
A convex function satisfies the property that a line between any two points on its graph is on or above the graph: for any . It is…
Modified: July 09, 2022.
See also: https://www2.sonycsl.co.jp/person/nielsen/Note-LegendreTransformation.pdf Jess Riedel on the Legendre transform in physics looks…
Modified: June 25, 2022.
Modified: .
Order of operations When should you add oil when preparing to sauté? Generally the answer is "after preheating the pan, unless it's nonstick…
Modified: March 30, 2022.
References: https://en.wikipedia.org/wiki/Cooperative_game_theory https://en.wikipedia.org/wiki/Shapley_value conversations with GPT-4 A…
Modified: September 12, 2023.
References: Cooperative Inverse Reinforcement Learning The Off-Switch Game Incorrigibility in the CIRL Framework The CIRL setting models…
Modified: April 05, 2023.
Modified: February 25, 2022.
Level 3 of Pearl's [ causal inference ] hierarchy: questions of the form 'given that (X, Y) happened, what would have happened if (X', Y…
Modified: August 02, 2021.
USA (45/50 states: all but N Dakota, S Dakota, Minnesota, Lousiana, Mississippi) Canada Ecuador Costa Rica Argentina England Scotland France…
Modified: April 06, 2022.
In the cell, creatine is stored as phosphocreatine. It acts like a 'backup' adenosine: Phosphocreatine can can donate its [ phosphate…
Modified: January 19, 2022.
Modified: .
"Crispr" == "Clustered regularly interspaced palindromic repeats". These are DNA sequences in bacteria that represent a genetic 'memory' of…
Modified: February 25, 2022.
easy introduction: https://developer.nvidia.com/blog/even-easier-introduction-cuda/ modern GPU (not so modern now): https://moderngpu…
Modified: .
Modified: .
Modified: .
This is an index of pages that reflect things I'd like to do, or that should otherwise should stay updated to reflect my current thinking…
Modified: September 11, 2021.
Someone on tinder had this phrase in their bio. As a life motto I think it's pretty powerful. It's a short pneumonic for 'fuck the world…
Modified: March 02, 2022.
Current (2021) deep networks require huge datasets in order to [ generalization|generalize ]. But we know that humans can do one-shot…
Modified: May 23, 2021.
How much compute does a typical Google data center have? Where would it fall on the supercomputer rankings? The top computer on the TOP50…
Modified: May 02, 2021.
The idea is that a [ mesa optimizer|mesa-optimizing ] policy with access to sufficient information about the world (e.g., web search) might…
Modified: March 31, 2023.
paper: Chen, Lu, et al. 2021, https://arxiv.org/abs/2106.01345 Trajectories are represented as sequences: where is the return-to-go, i.e…
Modified: April 15, 2022.
See also [ generative vs discriminative modeling ]
Modified: July 19, 2020.
References: Holtzman et al. (2020), The Curious Case of Neural Text Degeneration https://arxiv.org/abs/1904.09751 How should we actually…
Modified: July 02, 2022.
A simple lens on meditative insight progress, from Michael Taft : Start with some sensory object - the sight of a tree, or the felt…
Modified: September 06, 2022.
Notes from John Schulman's Berkeley course on deep [ reinforcement learning ], Spring 2016. Value vs Policy-based learning Value-based…
Modified: February 22, 2022.
Deep deterministic policy gradient (DDPG) is an interesting RL algorithm with a somewhat misleading name. Although its name indicates that…
Modified: July 27, 2022.
see also: [ differentiable program ]
Modified: February 10, 2022.
Which is more useful: reading the New York Times every day, or reading John Stuart Mill? listening to a current-events podcast, vs listening…
Modified: January 10, 2021.
A set of connected brain regions that are active when you're 'at rest', not focused on the external world. This includes mental states such…
Modified: February 25, 2022.
Vitalik Buterin argues for defensive accelerationism (d/acc): One frame to think about the macro consequences of technology is to look at…
Modified: December 01, 2023.
Like quantum mechanics! We build up a distribution over variables defined so far. When we need to use a value, we sample from this…
Modified: May 16, 2020.
Depression is the worst thing . Why? Ultimately we care about global utility. Depression is literally the state of finding it difficult or…
Modified: January 24, 2022.
Google-internal equivalents: Dev tools: The ex-Googler guide (sourcegraph.com) GitHub - jhuangtw/xg2xg: by ex-googlers, for ex-googlers - a…
Modified: May 15, 2022.
The [ hedonic treadmill ] manifests in taste for things like wine, beer, coffee, fine cuisine. I've never spent effort refining my taste in…
Modified: July 24, 2020.
Maybe a stupid idea, but I wonder if the idea behind differentiable physics simulators (like Brax) can be extended more broadly to rich…
Modified: July 04, 2022.
Fast differentiable sorting and ranking: https://arxiv.org/abs/2002.08871 What are differentiable analogues of 'standard' programming…
Modified: March 07, 2020.
Diffusion models for image generation were independently invented at least twice: in a discrete-time variational inference framework…
Modified: August 31, 2022.
References: http://www0.cs.ucl.ac.uk/staff/C.Archambeau/SDE_web/figs_files/ca07_RgIto_text.pdf https://www.ma.imperial.ac.uk/~pavl/lec_diff…
Modified: August 29, 2022.
Modified: .
References: Direct Preference Optimization: Your Language Model is Secretly a Reward Model This seems like a compelling reframing of…
Modified: May 31, 2023.
Modified: .
Modified: May 27, 2021.
Modified: May 03, 2024.
[ Otter notes ] August 2020: When somebody says that X is good---here X could be love relationships, money, peace, or whatever---it is never…
Modified: September 11, 2021.
There are two forms of uncertainty in value-based [ reinforcement learning ]. Let be the return from trajectory , and be the expected…
Modified: August 04, 2022.
Perhaps the only free lunch in [ finance ]. Given N investments all with the same expected value and level of risk (variance), whose…
Modified: May 15, 2021.
References: ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus , Causal Inference 2: Illustrating Interventions via a Toy…
Modified: August 06, 2021.
What does it mean to 'be yourself'? Selves aren't a well defined thing. Ultimately everything about your self is shaped by your surroundings…
Modified: May 01, 2023.
Chemically, dopamine is a benzene ring (aka phenyl group), with two hydroxyl (OH) groups at adjacent sites, and a two-carbon (ethyl) chain…
Modified: December 21, 2022.
Empirically, as model capacity increases past the memorization threshold ( ), [ generalization|generalization ] error starts decreasing…
Modified: March 02, 2022.
[ nootropics ] [ psychedelic ]
Modified: May 14, 2021.
https://twitter.com/getjonwithit/status/1832842063768867176 Duality" is a deep concept in mathematics, but an intuitive way to think about…
Modified: September 09, 2024.
TODO: flesh out theory, understand ADMM (e.g., https://www.cis.upenn.edu/~cis515/ws-book-IIb.pdf )
Modified: July 06, 2022.
From a conversation about [ attention ], [ multiplicative interaction ], and [ meta-reasoning ]: at some level, a lot of the AI problem…
Modified: October 17, 2022.
Elephant and rider Asking "what should I value?" is asking the rider. it demands solving impossible moral questions. Asking "what do I value…
Modified: July 08, 2020.
Modified: .
In Pāli, dukkha refers to the unsatisfactory parts of existence. Literally it refers to a wheel in which the axel hole is not centered…
Modified: October 25, 2022.
The fifth Buddhist precept is to refrain from intoxicants. Historically this means no alcohol. But a modern interpretation From [ Tucker…
Modified: .
Modified: January 26, 2022.
an idea that takes on a life of its own in the collective [ consciousness ] similar to Harari's concept of a shared myth. but Harari's myths…
Modified: .
Academic components: Right view (right understanding): seeing reality as it really is and understanding the [ four noble truths ]. This…
Modified: February 25, 2022.
sources: wikipedia claude materials deform following a stress-strain curve. for many materials, this curve starts out approximately…
Modified: September 11, 2024.
Modified: May 14, 2021.
Modified: October 29, 2021.
A few ways to think about eligibility traces: an explicit accounting of credit assignment a [ sufficient statistic ] for the history of the…
Modified: March 29, 2022.
Notes on Abram Demski and Scott Garrabrant's sequence on Embedded Agency Embedded Agents : Classic models of rational [ agency ], such as…
Modified: April 07, 2023.
Modified: .
A consequence of [ phase transition ]s in [ large models ] is that models may end up having capabilities we didn't expect. For example…
Modified: April 07, 2022.
Relationships and community and mental health are not automatically maintained, and maintaining them doesn't come for free. Comforting…
Modified: July 17, 2020.
I think [ Dan Brown ] said somewhere that a good synonym for 'empty' in meditative contexts is 'mere construction'. For example, practicing…
Modified: March 23, 2023.
See [ emptiness ].
Modified: March 23, 2023.
Andy Matuschak's concept of an Enabling Environment gets at something I've had in my mind but not named. It's an environment that expands…
Modified: January 24, 2022.
Modified: February 10, 2022.
David Chapman suggests that enlightenment in Buddhism is not a single defined thing, 'the word is hopelessly confused': https://vividness…
Modified: March 23, 2023.
Very smart people tend to disbelieve in [ enlightenment ] because they hold up unrealistic notions of what it is or what it entails. There…
Modified: May 30, 2023.
Often we think of ensembles in the context of supervised learning: we have some algorithm that learns X -> y mappings, and by running it…
Modified: February 10, 2022.
Measures uncertainty, disorder, or randomness. The (Shannon) entropy of a probability distribution is: The quantity inside the…
Modified: April 15, 2022.
Modified: .
Modified: .
Modified: .
Innerring (C. S. Lewis). In any institution there are unofficial circles of influence: people who are 'on the inside' and people who are…
Modified: March 29, 2024.
Probably the simplest and smallest molecule used as a psychoactive drug: It is neurotoxic, carcinogenic, and addictive, but of course…
Modified: August 01, 2023.
The currency of [ Ethereum ]. Why does Ether have value? It represents computing time on a shared global computer. The more Ether you have…
Modified: October 08, 2021.
Modified: January 25, 2022.
Foreign banks can create dollar-denominated liabilities much larger than their reserve of actual dollars, without the need to adhere to US…
Modified: March 13, 2022.
[ evergreen notes ] are a concept from Andy Matuschak. They're a framework for thinking about writing, note-taking, and intellectual…
Modified: May 01, 2020.
Events that seem really terrible---closing off good outcomes and potentially leading to bad outcomes---often refine into a fine path that…
Modified: February 25, 2022.
"In other words, there's a method of pursuing or following a devotion to the Guru, but ultimately every method is a trap, and you've got to…
Modified: October 03, 2021.
The state transitions we observe in [ reinforcement learning ] are typically correlated over time, both within a trajectory (obviously) and…
Modified: September 04, 2022.
Modified: February 10, 2022.