the discourse is wrong

In order for a group of people, like an academic field, or a political elite, to meaningfully converse about a complex topic, they have to agree on a conceptual scaffolding: a set of terms, definitions built on definitions, and mental models. We can't express raw thought directly into each others' minds. Language requires compression and abstraction.

This may work fine in well-developed and well-understood fields. But almost by definition the most interesting discourse is about things that we don't yet understand. For example, we don't know how to build general AI, or how to build a just society, or by what principles government can best optimize human welfare. This means that the conceptual scaffolding chosen by elite discourse is almost certainly not the right one! It's not the set of concepts we'd have if and when we finally understand these problems.One might hold that there is no right set of concepts: concepts are models, and all models are wrong. This is true and implies that the discourse will always be wrong, in the sense of being incomplete and missing important truths. It will always be a (set of) reality tunnels, not reality itself. But not all tunnels are created equal: within some complexity budget, we can still think about the representations that are optimal for achieving particular goals. And in immature fields we (almost by definition) haven't found those representations.

This is a difficulty for anyone who tries to carve the world at the joints. If the current discourse doesn't do this, then recognizing and thinking in terms of concepts that really do this means shutting yourself out of the discourse. It's a paradox: you might think the goal of research is to figure out how the world actually works, but this is useless if you can't express what you've discovered. The real goal is to improve the discourse. So you must force yourself to think and express yourself within the conceptual scaffolding of the current discourse even if it's wrong. You have to live within the same reality tunnel as everyone else, even though you know it's artificial.

This makes it seem like it's impossible to improve anything, but there are no paradoxes, just bad models. The way forward is to develop the skill of translating unconventional thoughts into the existing discourse. For thinking differently to matter, you have to explain your thinking so that others can follow. This might make it hard to think very unconventional things. But deep learning has shown that even unconventional things can be quickly adopted if they are effective.

You also need to be careful not to take this too far. Coming up with a separate discourse isn't useful if you're just reinventing the same things with different names. If your conceptual scaffolding gives you no extra 'bite' on the world, then who cares? Unfortunately there is a long way between internal 'bite' and something you can objectively justify. We might have unique experiences that lead us to vividly understand some insight that's hard to put into the general discourse, because the discourse itself is lacking. But it may also be that our experiences are not universal---others have conflicting experiences---so there's a reason that the concepts to describe our experience are missing from the discourse.

• Related: Paul Graham's warnings against groupthink
  • https://twitter.com/paulg/status/1281480690060726273
  • http://www.paulgraham.com/say.html