OpenAI · Feb 2026
A beta product built by agents
Agents generated the application, tests, documentation, infrastructure and tooling. Human work concentrated on preparing the environment, specifying intent and building feedback. [F06]
What the evidence already supports
Capability, productivity and outcome are three different questions. Learn to keep them apart.
⏱ 5 minChapter 2 / 18
2.1Reorganization is already happening
Two reports from the front line
These are vendor accounts of specific experiences — not forecasts for your organization. Their value is showing that directing agents and redesigning the production system are already engineering work.
Anthropic · Oct 2026
Merged changes per day
May~2,500
September30,000+
The bottleneck moved to development infrastructure, and version control was redesigned. The transferable lesson is to investigate your actual bottleneck — not to replace Git. [F39]
2.2Three questions to keep apart
Capability ≠ productivity ≠ outcome
A demo answers whether an agent can do something. It says nothing about whether your process got more efficient — or whether anything that matters improved. Classify each claim by the question it actually answers.
Three questions to keep apart
1 / 7 · Score 0An agent fixes a bug end to end in a live demo.
2.3What the research supports
Five studies, five kinds of evidence
Every study answers a narrower question than a headline suggests. Open each one to see what it supports — and where its limits are.
4,867
developers in three randomized field experiments (2025)
26.08%
more tasks completed (pooled estimate, standard error 10.3 pp)
~400,000
agentic coding sessions analysed by Anthropic (Oct 2025–Apr 2026)
260
agent-system configurations tested across six benchmarks
2.4The career implication
Where to invest your competence
The evidence converges on a handful of places where human competence makes AI pay off. This course turns that convergence into a curriculum — without claiming any single study validates all of it.
- Mastery of the problem domain
- Accessible context
- Good interfaces
- Verification
- Work organization
Key takeaways
- 1
Real teams already reorganize around agents; the lesson to transfer is finding your actual bottleneck, not copying their tooling.
- 2
Keep three questions apart: can it? (capability), is it more efficient? (process), did it improve what matters? (outcome).
- 3
Every study has a scope: read the limits as carefully as the headline numbers.
- 4
The evidence points to investing in domain, context, interfaces, verification and work organization.
Check your understanding
Question 1 of 3
A vendor shows an agent completing a complex migration in a demo. Which question does this answer?
Read the full section of the original reportThe complete, unabridged text — with every source and nuance.
2.1. There are present-day experiences of deep reorganization
In February 2026, OpenAI described a team that built a beta product using agents to generate the application, tests, documentation, infrastructure, and tooling. Human work concentrated on preparing the environment, specifying intent, and building feedback. This is a technical account from the vendor about a specific experience; its estimated gains are not a forecast for any given organization. F06
In October 2026, Anthropic reported that its volume of merged changes grew from roughly 2,500 per day in May to more than 30,000 in September. The bottleneck moved to development infrastructure, and the company redesigned its version control. The authors stress how exceptional this load is. The transferable lesson is to investigate the actual bottleneck; the experience does not imply that other teams should replace Git. F39
These cases make the manifesto's point concrete: directing agents and redesigning the production system are already engineering activities. The result depends on the ability to turn local speed into a reliable process.
2.2. Research distinguishes capability, productivity, and outcome
| Evidence | What was studied | What it supports | Relevant limitation |
|---|---|---|---|
| DORA 2025 | Nearly 5,000 respondents and more than 100 hours of qualitative research | AI amplifies organizational strengths and difficulties; the working environment matters | Observational survey and self-reports; does not provide a universal causal multiplier. F03 |
| Three field experiments, 2025 | 4,867 developers at three companies; randomized access to a coding assistant | The pooled analysis estimated 26.08% more tasks completed, with a standard error of 10.3 percentage points | Tools and tasks of that time; completed tasks do not measure all of a product's value or quality. F09 |
| DORA, March 2026 | Analysis of 1,110 open-ended responses from Google engineers about Q3 2025 | Faster generation can shift effort toward auditing and verification | One organizational context and qualitative data; not a causal measurement of the full flow. F05 |
| Anthropic, June 2026 | About 400,000 sessions from roughly 235,000 people, October 2025 to April 2026 | Domain expertise is associated with greater success; human decisions concentrate more on planning than on execution | Vendor study with session classification; local success does not prove longitudinal quality in production. F08 |
| Kim et al., April 2026 revision | 260 configurations across six agent benchmarks | Task structure and coordination affect the benefit of using multiple agents | Experimental preprint; its percentages are not estimates of team productivity. F21 |
Three questions need to be kept separate: can the agent do the task?; did the process become more efficient?; did the outcome improve something that matters? A capability demonstration answers the first. To answer the others, we need to observe cost, time, quality, rework, and effect on the user.
2.3. The career implication
The most useful result of this literature for the engineer is identifying where to invest competence. The evidence points to mastery of the problem, accessible context, good interfaces, verification, and work organization. This report turns that convergence into a practical curriculum, without attributing validation of the whole curriculum to any single study.
We also need to update the way we measure. In February 2026, METR explained that participant and task selection, changes in compensation, and simultaneous use of agents made its new round of experiments hard to interpret. This case shows why human time, the duration of a run, and concurrent work need to be distinguished. A result from a previous generation of tools should not serve as a permanent portrait of the profession. F10
The horizon for action, therefore, is current work: experimenting on real tasks, measuring consequences, and changing the process. Research serves to make that adaptation more competent.