Test Pyramid Criticized: 'Test Deciderata 2.0' Offers a New Framework for Software Quality

The long-standing Test Pyramid, introduced by Mike Cohn in 2009 and popularized in agile circles, has been a foundational model for guiding software testing investment. Initially delineating unit, service, and UI tests, Martin Fowler later enhanced the model by explicitly adding axes for speed and cost—fast/cheap for unit tests, slow/expensive for UI tests, and service tests in between. However, Emily B., a software developer and creator at Saman Coaching, highlights significant criticisms: the pyramid’s assumption of a direct correlation between test granularity and speed/cost is often untrue, and its terminology (e.g., ‘unit test,’ ‘service test’) is confusing and inconsistent with modern agile practices, frequently conflating concepts like UI tests with end-to-end system validation. These ambiguities, she argues, render the pyramid’s advice less helpful for contemporary software development.

In response, Emily B. proposes the ‘Test Deciderata 2.0’ model, shifting the focus from specific test types to four macro properties a test suite should possess: fast, cheap, predictive of deployment success, and supportive of ongoing code design changes. This model emphasizes that the kind of test is less critical than the desirable properties it contributes. ‘Predictive of deployment success’ ensures tests find critical bugs before production, addressing scenarios where only a fully running system can expose issues. The fourth property, ‘supportive of ongoing code design changes,’ underscores the role of tests in driving good design through practices like Test-Driven Development (TDD) or Behavior-Driven Development (BDD), leading to more usable APIs and testable code, thereby indirectly supporting the other three deciderata. This contrasts with many AI-generated tests, which often excel at speed, cheapness, and predictiveness but struggle to embed the ‘why’ or customer intent crucial for design guidance and meaningful documentation. The Deciderata 2.0 aims to redefine test strategy by prioritizing these benefits over rigid categorizations, fostering a more effective and adaptable approach to software quality.