On 8 November 2026, a coalition of researchers from Alibaba and several Asian universities published findings that sharply contrast with the prevailing industry narrative surrounding autonomous artificial intelligence. Their newly developed evaluation framework, Co-Design Bench, reveals that state-of-the-art AI coding agents still require human intervention to clean up nearly three-quarters of real-world software engineering tasks.

Co-Design Bench represents a critical departure from older, simplified tests where frontier models frequently achieved near-perfect scores. By constructing a dataset entirely from authentic user change requests, the researchers created a highly realistic testing environment that mirrors the daily challenges faced by software developers. Under these rigorous conditions, even the most advanced models fully completed only about 25 per cent of assigned programming tasks. This exposes a persistent deficit in the ability of current AI architectures to reason through complex, multi-step codebases without continuous human supervision.

The research team noted that the high success rates achieved on earlier, highly constrained tests have fostered a false sense of security regarding AI capabilities. By shifting the evaluation criteria to reflect the messy, interconnected reality of actual software change requests, the Co-Design Bench effectively resets the baseline for measuring agentic performance.

The findings from the software engineering sector are being mirrored in other domains, indicating a structural limitation in current agentic AI. Also on 8 November 2026, Visa introduced a new industry benchmark designed to evaluate AI in agentic commerce. Visa’s research concluded that artificial intelligence remains highly inconsistent and is not yet capable of reliably executing purchasing decisions on behalf of consumers. When AI systems are tasked with navigating e-commerce platforms, verifying product details, and executing financial transactions autonomously, their decision-making logic frequently breaks down.

For technology sectors across Aotearoa and the wider Asia-Pacific, these parallel benchmarks deliver a necessary dose of reality. The intense commercial pressure to deploy AI agents in customer service, retail, and software development has often outpaced empirical evidence of their reliability. Regional enterprises looking to reduce headcount or accelerate development cycles through autonomous AI must now recalibrate their operational expectations. The data confirms that human-in-the-loop oversight is not merely a transitional safety measure, but an ongoing requirement for the foreseeable future.

The introduction of robust evaluation frameworks like Co-Design Bench signals a maturation in AI research. As the industry moves past the initial hype of generative capabilities, the focus is decisively shifting toward identifying exact failure points in complex workflows. The road to genuine autonomy will require overcoming these fundamental reasoning bottlenecks, rather than simply scaling up existing computational models.

Sources


AI disclosure: Automated systems assisted with research and drafting. Gemini independently peer-reviewed this article before publication. Sources are linked for verification.