The Five Feedback Loops That Turn AI Coding Agents From Liability Into Leverage

In Brief

The AI Codebase Maturity Model (ACMM), published in April 2026 by Andy Anderson of IBM Research, proposes a five-level progression for codebases working with AI coding agents. The central finding: agent intelligence resides in the surrounding infrastructure of tests, instructions, and feedback loops, not in the model itself. For Bulgarian engineering teams evaluating AI coding tools, this reframes the investment question from which model? to what does our codebase need to measure? The framework has been validated through a four-month experience report on KubeStellar Console, a CNCF Sandbox project that achieved 91% test coverage and 81% PR acceptance rate with AI agents.

The ISTA 2026 organisers, including Experian, Accenture, Infragistics, Avenga, SAP, and Merkle, will be hosting exactly these conversations on stage this September.

Pattern Name: AI Codebase Maturity Model (ACMM)

Why This Is Important: Adoptable. Teams using AI coding agents without structured feedback loops are experiencing negative productivity returns. The ACMM provides the first systematic framework for diagnosing where a codebase sits and what investment unlocks the next level of agent autonomy.

Domain: DevOps, Architecture

Who Should Care: DevOps/SRE, Backend, Frontend, Manager, Student

Level: Intermediate

Evidence Type: Case Study

The Problem

AI coding agents produce code faster than developers can review it. The initial productivity gains are real: features that would take days appear in hours. Then the failure modes compound. Builds break in ways that are hard to trace. Architectural choices from the previous day get silently overwritten. The agent touches files it was not directed toward. Fix one thing, three others break.

The standard industry advice, give the agent more autonomy, tends to make the failure mode worse. Teams end up spending more time reverting than reviewing, and the promised 10x productivity becomes a net negative.

What Changed

The AI Codebase Maturity Model, published on arXiv in April 2026, provides a five-level framework for understanding why this happens and what to do about it. The model is inspired by CMMI (Capability Maturity Model Integration), but each level is defined by its feedback loop topology: the specific mechanisms that must exist before the next level becomes possible.

The five levels are:

• Assisted: Developers use AI for code completion and generation. Outputs require manual review. No structured feedback.
• Instructed: Preferences are externalized through explicit documentation. CLAUDE.md files, copilot-instructions.md, and structured prompts define conventions and boundaries. The agent only knows what the codebase tells it.
• Measured: Tests become the trust layer. Coverage thresholds, test reliability, and CI/CD workflows provide the feedback signals that let agents self-correct. A flaky test in a human workflow is tolerable; in an autonomous one, it erodes trust.
• Adaptive: Automation follows measurement. The codebase can respond to agent outputs with automated gates, rollbacks, and escalations. Automation without measurement produces confident drift.
• Self-Sustaining: The codebase becomes the operating manual. Agents can onboard new functionality from the codebase state alone, without extensive human instruction for each task.

Why Now

Three converging pressures make this framework timely.

First, AI coding agent adoption has crossed the threshold where most teams have tried them. According to JetBrains AI Pulse data from January 2026, AI tool usage in software development exceeds 90%. The question is no longer whether to adopt, but how to make adoption productive.

Second, the gap between demo performance and real-world results is widening. BlueOptima’s analysis of 30,000+ developers found only a 5.4% productivity uplift in real-world conditions for the median developer, compared to the 55% faster task completion reported in controlled experiments. The difference comes down to infrastructure, not model capability.

Third, the cost structure has changed. Teams using agentic tools now spend $200 to $2,000+ per engineer per month in token costs on top of seat licenses, according to Larridin’s 2026 developer productivity benchmarks. At this scale, the difference between top-quartile and bottom-quartile performance is a financial governance question, not just an engineering management one.

The New Practice

The ACMM validation comes from a four-month experience report on KubeStellar Console, a Kubernetes multi-cluster management dashboard in the CNCF Sandbox. The project was built from scratch by a single maintainer using Claude Code (Opus) and GitHub Copilot running in parallel terminal sessions.

The infrastructure that emerged: 63 CI/CD workflows, 32 nightly test suites, 91% code coverage across twelve shards. Across 82 days, PR acceptance settled around 81%. Community bug reports moved to merged fixes in roughly thirty minutes. Feature requests landed as pull requests in about an hour.

The intelligence of an AI-driven development system resides not in the AI model itself, but in the infrastructure of instructions, tests, metrics, and feedback loops that surround it.

The levels cannot be skipped. At each level, the thing that unlocks the next one is another feedback mechanism. Testing, specifically the volume of test cases, the coverage thresholds, and the reliability of test execution, proved to be the single most important investment in the entire journey.

Tooling Implications

The ACMM has direct implications for how teams should evaluate and invest in AI coding infrastructure.

Level 1 to Level 2 (Assisted to Instructed): The cheapest intervention with the highest return is externalizing preferences. This means creating CLAUDE.md files at the repository root, .github/copilot-instructions.md for PR conventions, and structured documentation that defines what the agent should and should not do. No new tooling required; just documentation discipline.

Level 2 to Level 3 (Instructed to Measured): Test infrastructure becomes the critical investment. The KubeStellar Console experience suggests that 90%+ coverage is not a target but a precondition for agent autonomy. Flaky tests must be eliminated, not tolerated. Test sharding and parallel execution become essential for feedback loop speed.

Level 3 to Level 4 (Measured to Adaptive): Automation gates require measurement to exist first. This means CI/CD workflows that can block, rollback, or escalate based on test results, coverage deltas, and other quality signals. The automation layer interprets the measurement layer.

Level 4 to Level 5 (Adaptive to Self-Sustaining): At this level, the codebase itself becomes the source of truth for agent behavior. The agent can read the codebase state and determine what to do next without explicit human instruction for each task.

Evidence

The primary evidence comes from the arXiv paper itself, which includes a transparency note: This paper was written with AI assistance. The author was interviewed by Claude Code (Anthropic’s Claude Opus), and the content was organized with AI support. The 4-month experience described herein is entirely the author’s.

The metrics reported:

• 63 CI/CD workflows
• 32 nightly test suites
• 91% code coverage across twelve shards
• 81% PR acceptance rate over 82 days
• Bug-to-fix time under 30 minutes
• Feature request to PR time approximately one hour

The author, Andy Anderson, holds a Ph.D. in Technology Adoption and has been with IBM Research for nine years, the last four with IBM Research. He has served as chief maintainer of the KubeStellar organization for four years, stewarding it through three years as a CNCF Sandbox project.

An OpenSSF Scorecard integration proposal has been submitted to add ACMM criteria as a source in the OpenSSF dashboard, which would allow teams to see their security and AI-readiness posture in one place.

Failure Modes

The ACMM paper identifies several failure modes that teams encounter when trying to skip levels:

Automation without measurement: Teams that add CI/CD automation before establishing reliable test coverage end up with confident drift. The automation runs, but the feedback signals are unreliable, so the automation makes things worse.

Instruction without enforcement: Documentation that is not enforced through tests or gates becomes stale. The agent reads the instructions but the codebase does not verify compliance.

Measurement without action: High test coverage that does not gate deployments or trigger rollbacks provides visibility without leverage. The team knows something is wrong but the system does not respond.

Autonomy without trust: Giving agents more autonomy before the feedback loops are in place amplifies the cascade problem. The agent makes more changes, more things break, and the team spends more time reverting.

Metrics

Teams can assess their ACMM level using the following indicators:

• Assisted: AI tools in use, no structured documentation, manual review of all outputs
• Instructed: CLAUDE.md or equivalent exists, agent preferences documented, conventions defined
• Measured: 80%+ test coverage, flaky test rate below 1%, CI/CD provides reliable feedback
• Adaptive: Automated gates block bad changes, rollbacks triggered by test failures, escalation paths defined
• Self-Sustaining: Agent can onboard new functionality from codebase state, bug-to-fix time under 1 hour

The KubeStellar Console metrics suggest that Level 4 (Adaptive) is achievable within four months for a greenfield project with a single maintainer. Existing codebases with technical debt will require longer timelines.

Do This Next Sprint

• Audit your current level. Check whether your repository has a CLAUDE.md or equivalent instruction file. If not, you are at Level 1. Creating one is a single-sprint task.
• Measure your test reliability. Calculate your flaky test rate over the past 30 days. If it exceeds 1%, prioritize flaky test elimination before adding more agent autonomy.
• Check your coverage gates. Determine whether your CI/CD pipeline blocks merges when coverage drops. If not, add a coverage gate this sprint.
• Document one agent boundary. Pick one area where the agent should not make changes (e.g., security-critical code, database migrations) and add it to your instruction file.
• Track bug-to-fix time. Start measuring how long it takes from bug report to merged fix. This becomes your baseline for measuring agent productivity improvements.

What This Means for Bulgaria

Bulgarian engineering teams working with AI coding agents face the same infrastructure gap as teams elsewhere, but with some specific considerations.

For teams at Bulgarian outsourcing firms: Client expectations around AI-assisted development are rising. The ACMM provides a framework for explaining to clients why agent productivity varies and what infrastructure investments are needed. Ask your technical lead whether your client projects have instruction files and coverage gates.

For teams at Bulgarian product companies: The cost structure of AI coding tools (token costs on top of seat licenses) makes the ROI question urgent. Track your team’s actual token spend and correlate it with productivity metrics. If you are spending €500/month per engineer on AI tools and seeing only 5% productivity uplift, the infrastructure gap is the likely cause.

For Bulgarian engineers interviewing at international companies: Ask about the company’s AI coding infrastructure during interviews. Questions like Do you have instruction files for AI agents? and What’s your test coverage gate threshold? will reveal whether the company has thought systematically about agent productivity.

For students: The ACMM paper is available on arXiv and provides a concrete framework for understanding how AI coding tools work in practice. Reading it before your first job will give you vocabulary that most junior engineers lack.

Dig Deeper

• The AI Codebase Maturity Model: From Assisted Coding to Self-Sustaining Systems (arXiv paper, April 2026)
• AI in DevOps: Why Adoption Lags in CI/CD (JetBrains, April 2026)
• Developer Productivity Benchmarks 2026 (Larridin)
• ACMM OpenSSF Integration Proposal (GitHub issue)