Modified: April 05, 2020
growing up means becoming wrong
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.(related: communication is processing)
A big part of growing up is communicating to your future self. Your future self isn't going to understand all the complexities that you currently do, because they're going to be thinking about other things. To benefit from your experience, they need abstractions---big lessons, general rules, useful categories and models.
If you look at senior people and think "they don't really understand what's going on", you might be tempted to never become one of those people. But what you don't realize is that you yourself only understand a tiny fraction of what's going on---just one piece of the ground. People with more experience see from higher up in the air. They really don't know what's going on on any given patch of ground, and that's a major blind spot. But they can see a lot further.
By resolving to never become one of the people who doesn't understand the details, you're committing to never leaving the ground, never seeing further. That's fine---we need people of all sorts---but understand that's the tradeoff.
Of course, the best people can see from high up while still maintaining regular enough contact with some patch of ground that they at least have some sense of the dynamics on the ground---even though no one can be in touch with every patch at once.
I mean all of this in a metaphorical way, but also in a very technical way. These are points about abstraction, processing, and compression. As you move away from the ground, your vision of lower-level details 'blurs': literally, information is lost. Only larger patterns are visible. Some new types of patterns might become visible that you could never see at ground level---road and rail networks, land usage, density of lighting, and so on. By compressing the low-level details---throwing out 'unnecessary' information---you can represent more context with the same amount of space. This is almost literally what the upper levels in many neural networks do.
- Of course the tricky thing is to understand what information is necessary and what's not. With aerial vision, you don't get to choose; the physics of (refraction?) do the blurring for you. In information processing, we often do get to choose what information to preserve. Sometimes there's redundancy that can be thrown out at no cost (lossless compression). Sometimes the choices are natural and general (all neural nets learn edge detectors). But sometimes you learn from end-to-end training that some features are important to preserve.