Created: May 29, 2020
Modified: May 29, 2020
Modified: May 29, 2020
useful reading
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.- Consuming unstructured content from the internet is addictive. Twitter is full of life advice, interesting technical discussion, takes on the daily news, the firehose of new ML papers. There's so much to learn! I can fill pretty much every moment with thoughtful writing or listening in on highbrow discussion. Compared to what was available even ten or fifteen years ago, it feels like you can't lose.
- But no amount of reading ML researchers on twitter will add up to the experience of asking and exploring a question yourself. No amount of reading Derek Lowe columns on drug discovery, or Matt Levine financial columns, will build a structured knowledge base in those areas. No quantity of news stories or takes in high-end magazines will add up to the experience of living in different places, meeting different people, and building a coherent worldview.
- Breadth is valuable, but only when accompanied by some measure of depth. Being competent but not expert in two scientific fields, or two languages, or the experience of living in two different countries, is valuable, but knowing trivia about two scientific fields, languages, or countries, is much less valuable. The important aspect is generative capacity, in the sense of (generative vs discriminative modeling): being able to do two different sciences, or speak two different languages, is inherently useful, and requires much deeper and more coherent knowledge than simply being able to describe the scientific practice, or the language, etc., which is not useful.
- Because useful, generative knowledge is inherently deep, you have to focus to acquire it. You have to move step by step, each step building the cognitive structure on which the next steps can hang. A firehose of content will not do this for you; you need an organized curriculum. (also one of the main flaws of sgd!!)
- Time is exponentially more valuable when spent on activities that give compounding returns. Focused reading and practice in a particular area makes it easier for you to learn new and bigger things in that area. Reading internet takes does not.
- There's nothing wrong with reading Twitter or other media as entertainment: as a substitute for watching TV or playing a game. It's probably as valuable as either of those. But it's not a substitute for structured learning and purpose-driven exploration.
- The best way to learn something new is to challenge yourself to achieve a goal, and let that goal drive your learning. Building a molecular dynamics simulator is driving learning a lot of physics and chemistry for me right now. Most people can't just choose a goal, though; it has to be enforced in some external way. The best is to find someone to pursue the goal with, because conspiracy is a thing for a reason.
- You can also choose external circumstances to challenge you. For example, moving to a new country will force you (to some extent) to learn the culture and language. Taking a new job will force you to get up to speed in an area. Joining a club or other community can also be challenging.
- Not all reading has to be goal-directed. But it's still often useful to give time to content that has been deliberately structured to convey something deep, even if that structure is not currently motivated by a goal. For example, for the past few months I've been listening to Edward Gibbon's History of the Decline and Fall of the Roman Empire. It's been light entertainment, to listen to while I run or work out. I haven't internalized most of the details, and sometimes I drift off entirely, but over time it's still added up in a way that most podcasts don't. I don't miss the time spent listening to discussion of current events.