The Bitter Lesson: Theoretical Foundations
· 2 min read
Sutton's Bitter Lesson is usually presented as an empirical observation - compute beats cleverness historically. But there are theoretical reasons to expect this.
Sutton's Bitter Lesson is usually presented as an empirical observation - compute beats cleverness historically. But there are theoretical reasons to expect this.
Richard Sutton's 2019 essay "The Bitter Lesson" makes a simple argument: in AI, more compute beats clever engineering. Every time.