projection is unavoidable

The idea of 'projection' in psychology means to assume that someone else has the same flaws, or foibles, or motivations as you do. It struck me as a weird psychobabble term until I found a way to understand it that makes it seem #fundamental to everything.

Here's the thing: how does an intelligent system build models of other agents? How do we reason about: what would person X do in this situation? By necessity, we reuse most of the same mental circuitry we use for thinking about what we ourselves would do in that situation. The brain doesn't have enough capacity to simulate many other people as part of making our own decisions. I don't have a special sub-brain for 'what would Alice do' or 'what would Bob do', I just have the one brain.

In particular, we have learned planning algorithms, learned proposal mechanisms for 'what actions would make sense here', and learned attention mechanisms for 'what do we even pay attention to in this situation'. These connect to learned conceptual scaffolding and reality tunnels. In reality, other people will have a wildly different conceptual hierarchy than we do (obviously not entirely different, since it's learned from the same world, but it will be more different than we can imagine, since our imaginations are limited by our own concepts). All of these will introduce leakage from our own minds into our expectations of what other people might do.

An interesting question in this light is whether we see projection as valid. If we think 'at the core, I am like other people', then we tend to trust our intuitions about what other people will think and feel, and we feel comfortable relating to people. If we think 'at the core, I am different from other people', then we will distrust our projective models, and have a hard time relating. This is alienation, of the sort discussed by VGR in keys and locks.