Created: January 16, 2021
Modified: January 16, 2021
Modified: January 16, 2021
researchers don't always know best
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.- People who do research have a very ground-level, zoomed-in view of their field. They know where the current obstacles are, how incredibly difficult it will be to overcome them, and that there are likely very many additional obstacles beyond those.
- They see themselves making slow, painful progress, and it becomes important to their egos to believe that the work they're doing is genuinely hard. And it is.
- As adults, their sense of the possible tends to be limited by concrete execution capability. They're naturally conservative and afraid to promise results they don't know how to deliver.
- Of course, researchers generally know vastly more about their fields than outsiders, and they're often right about things. But in AI in particular the expert consensus has been too pessimistic recently.
- When I took graduate computer vision in 2010, no one was predicting or would have believed that ImageNet would be effectively solved within a few years.
- I don't think NLP researchers from a few years ago would have believed GPT-3 results.
- No one expected us to solve Go in this decade. Even this year, no one expected us to solve protein folding, until it happened.
- Sometimes, looking at larger trends and thinking at the meta level gives a clearer and more accurate picture than is available to researchers on the ground. You have to blur your eyes to see properly.