Skip to main content

The Bitter Lesson

· 3 min read
Steven Normore
Steven Normore
Author

In 2019, Richard Sutton, one of the pioneers of reinforcement learning, wrote a short but impactful essay titled "The Bitter Lesson". The core argument is simple: history has repeatedly shown that leveraging more computation trumps human-designed domain knowledge in AI. Despite decades of attempts to encode expert rules and handcrafted heuristics into AI systems, raw computational power combined with scalable methods has consistently outperformed these approaches over time.

This lesson is "bitter" because it goes against the instincts of many AI researchers. We like to believe that human insight, careful engineering, and domain-specific tricks will give us an edge. But the evidence tells a different story: general-purpose learning methods that scale with compute outperform carefully engineered solutions in the long run.

The History of Compute vs. Human Ingenuity​

Sutton’s essay outlines several historical examples:

  1. Chess & Go: Early chess engines relied on handcrafted rules. But once Deep Blue (1997) and later AlphaZero (2017) used massive compute and self-play learning, they obliterated human-crafted chess engines. AlphaZero even surpassed the strongest Go players by training from scratch with deep reinforcement learning.
  2. Speech Recognition: Early systems relied on linguistic rules and expert phoneme modeling. Eventually, deep learning models trained on massive datasets, leveraging more compute, dominated the field.
  3. Computer Vision: Feature engineering was once a major focus in image recognition. Then, deep convolutional neural networks (CNNs) trained with huge datasets and computational power crushed all previous handcrafted approaches (e.g., AlexNet in 2012).

Why Does The Bitter Lesson Hold?​

  1. Computation Scales, Human Design Doesn’t: Engineering heuristics requires domain expertise and doesn’t generalize well. Computation, on the other hand, follows Moore’s Law (or similar trends) and keeps getting cheaper.
  2. Learning Outperforms Handcrafted Rules: Whether in board games, natural language processing, or robotics, general learning algorithms fed with data tend to outperform manually designed rules over time.
  3. The Nature of Intelligence: Intelligence in nature evolved through massive amounts of computation (e.g., biological evolution, neural activity), not through handcrafted rules. The same principle applies to AI.

The Implications for AI Research​

  • Favor Scalable Methods: Invest in approaches that improve with more compute and data, rather than those that rely on fragile heuristics.
  • Expect Compute-Heavy Breakthroughs: The biggest advances will likely come from scaling existing learning methods rather than fine-tuning domain-specific tweaks.
  • Embrace the Shift in Research Priorities: As AI matures, the role of human intuition in designing solutions may shift toward selecting architectures and optimizing computational efficiency rather than encoding domain-specific knowledge.

The Future: AI and the Compute Race​

The Bitter Lesson suggests that the key to continued AI progress is not better heuristics but more compute and better scaling laws. This is why AI research has moved toward massive transformer models, large-scale reinforcement learning, and self-supervised learning across diverse domains. The arms race in AI is increasingly becoming a battle of compute, where companies and nations invest in massive training clusters to push the limits of what’s possible.

In the end, the lesson remains the same: Bet on compute. It may be bitter for some, but it is the reality that continues to shape the future of AI.