From Agile to AI: Anniversary workshop says test-driven development ideal for AI coding

25 years after the Agile Manifesto, a group of experts hosted by one its signatories met to consider the impact of AI on software development, concluding among other things that test-driven development has never been more important.

"Test-driven development (TDD) produces dramatically better results from AI coding agents," states the report on the workshop. "TDD prevents a failure mode where agents write tests that verify broken behavior. When the tests exist before the code, agents cannot cheat by writing a test that simply confirms whatever incorrect implementation they produced."

TDD is an approach to software development where automated tests are written before the production code. The tests are expected to fail until the code is written. TDD is valuable for iterative development and refactoring – improving code without changing its behavior – since it verifies that the code continues to behave as expected. 

The Agile Manifesto was composed by a group of IT professionals in February 2001, and influential in its promotion of a collaborative, flexible and iterative approach to software development. One of its signatories was Martin Fowler, chief scientist at software development company Thoughtworks. Fowler and Thoughtworks hosted the workshop 25 years on, to "rethink how software is built" in the light of "a new inflection point: the shift to AI-native software development."

The workshop identified engineering discipline as a key topic surrounding AI coding. If AI writes the code, the rigor that used to live in coding "does not disappear, it moves elsewhere," the report states. The question that "surfaced in nearly every session" is how to retain engineering discipline in the context of AI-driven development.

When a team is given AI tools, the bottleneck shifts from engineering capacity to other elements of software development, according to the report, including cross-team dependencies, architecture reviews, and project decisions. "The result is not faster delivery. It is the same speed with more frustration," the workshop concluded.

An issue with AI development is that, as with human teams, agents working on different parts of a project will use divergent patterns and preferences. AI accelerates this drift, and there is a question about whether standardization should be attempted, or divergence embraced.

Although experienced developers are more effective at supervising AI agents, thanks to their understanding of system architecture, junior developers also have increased value, the report suggests. "They are better at AI tools than senior engineers, having never developed the habits and assumptions that slow adoption."

Security is "dangerously behind" according to the report, because it tends to be treated as something to solve later.

A consistent pattern is that established practices, tools and organizational structures are now "breaking in predictable ways under the weight of AI-assisted work," the report states. People are experiencing "an identity shift in their professional lives" according to participants. There is an unsolved problem of "how to build trust in systems that are inherently non-deterministic."

In the light of this AI shift, should there now be a new manifesto?" No, "it’s way too early” said Fowler. "People are still experimenting with ideas, still trying stuff.” He added that, “I don’t have a lot of time for manifestos. Most of them are wisely ignored by most people," though he acknowledged that the Agile Manifesto was an exception.