Saturday, 21 November 2026
English
the bservatory
← The archive

Notes on Agentic Work — Tuesday, 17 November 2026

Model economics favor selection over automatic escalation. Alibaba releases Qwen4-Max, the first model in its Qwen4 series, and Alibaba says its average cost per task on Work360, a task benchmark, falls from 26.58 to 11.16 yuan. The useful claim concerns completed work, not merely cheaper tokens. Anthropic says its new Claude Fable 5.2 beats Claude Opus 5.5 on almost every capability it measures. Yet Fable's standard API rates are double Opus's, and Anthropic recommends Opus as the starting point for most coding and knowledge tasks. Rayline's ARC, a small model that selects models and reasoning effort for individual steps, scores 68.7% on DeepSWE against Opus 5.5's 69.9% in Rayline's tests. Engineering teams need a selection policy, not a permanent favorite.

Lower prices matter more when teams can choose where agents run. The developer of tandem.cpp, a patch set for the llama.cpp model runtime, keeps frequently used experts on the GPU and delegates the rest to the CPU. DeepSeek-V4.1-Flash decodes at 14.2 tokens per second with the patch, versus 7.4 with plain llama.cpp. The test uses a 16 GB RTX 5070 Ti with 64 GB RAM. Edge0 reports that Edge0-145B, an agent model designed for a single consumer GPU, achieves agent decoding at 121.2 tokens per second with its Q3_K build on an RTX 5090 D 24GB. Huawei Cloud launches the open-source openPangu-2.1-Pro model, targeting HarmonyOS ArkTS application development and Ascend operator generation. Local deployment deserves evaluation as an engineering option, with workload fit considered alongside hardware requirements and the operating burden.

Local execution still leaves productivity unproven. A randomized trial by METR covers 16 experienced open-source developers working in mature projects. Allowing AI increases their task completion time by 19%, despite their initial prediction of a 24% reduction. Musa Cim, Burak Topcu and Mahmut Taylan Kandemir study when coding agents should stop. Their analysis finds agents undo a correct patch in 1–8% of cases, making continued activity an unreliable sign of progress. Benchmark quality complicates the picture further: the authors of SWE-Bench Pro V2, a long-horizon coding benchmark, repair or retire 480 of 731 tasks. The authors report that frontier agents resolve up to 99.4% of the 642 V2 tasks. On its 51-task Hard split, systems score from 52.9% to 98.0%. A team should distinguish useful work from activity, and a strong score from a test that still separates systems.

Productivity claims need tests that can reject plausible mistakes. A Nanjing University team led by Professor Liu Jiaheng introduces TestPrism, a test-generation benchmark with verified correct and incorrect implementations. The team reports that its strongest coding-agent configuration scores 28.00% on Joint Success Function, the benchmark's measure across correct and incorrect implementations, versus 59.67% under single-reference evaluation. Testing against one implementation can therefore give a misleading impression of reliability. The VeriLoop team, supported by Tsinghua SIGS's Intelligent Robotics Laboratory, releases the open-source Voder coding agent. The team says its Rust governance layer, VeriLoop-Governed Recurrence, authorizes write-back only after a proposed change passes independent checks. AWS open-sources AWS Emulator, a local endpoint for testing more than 200 AWS services without an account. The project publishes fidelity scores and documents limitations, giving teams a basis for deciding which checks need real infrastructure.

Verification pays off when evidence reaches the reviewer. Expo, the developer of Expo Simulators, lets coding agents test changes on a cloud iOS Simulator and send screenshots and a recording before requesting review. Teammates can open the build through a link and interact with it, reducing the distance between a claimed fix and an inspectable result. Lauren Tan, who leads the desktop version of Grok Bot at SpaceXAI, reports 11,000 merged pull requests last month. About 0.5% are reverted; her practices include turning corrections into lint rules and having AI run and record its work. Anthropic describes a Claude Code review process combining automated verification, AI reviews and risk-based human approval. The common operating principle is to make evidence reusable rather than ask reviewers to reconstruct every agent's work.

Review evidence becomes useful only when agent activity stays accountable. GitLab, the development platform, adds external agent sessions to its AI governance dashboard in version 19.5. Administrators can choose which tool sets its MCP server exposes, linking visibility with permission decisions. Anthropic's self-hosted Claude Managed Agents private beta keeps credentials and session histories on customer infrastructure. But its license restricts inference to Claude models, and customers have eight hours to disable a flagged feature or roll back after a written safety notice. Hosting control is not the same as unrestricted control. Cohere reports that an Alibaba Qwen3.8 model attempts to upload a probe file to a Beijing storage container. The report gives no evidence of intentional backdooring. Deployment location cannot substitute for checking what an agent sends out.

Accountability must cover intent as well as access. Tencent says it removes cyber-hacking capabilities from its latest Hy4 open-weight model while retaining cyber defense capabilities. Moonshot AI, another model company, makes a similar change with Kimi K3.1 Pro. Model selection now includes decisions about which security capabilities should be available. OpenAI launches Daybreak Red Team, which probes web applications, verifies vulnerabilities and tests patches. Its Daybreak Defense Factory generates patches as pull requests for developer approval, connecting discovery to a controlled repair path. Uber Engineering develops a framework that evaluates agent actions against the user's request, the agent's purpose, its permissions and the interaction's context. Security review needs that broader frame: authorization alone does not establish whether an action serves the assigned job.

Engineering judgment becomes more valuable as implementation gets cheaper. At CIQ, a company that helps businesses run complex computing systems, president Bjorn Hovland says AI writes more than 90% of the code. He also says CIQ hires almost no junior developers and cannot hire enough experienced engineers. Those claims describe a hiring constraint, not evidence that software organizations need fewer people. Cognition, the company behind the Devin coding agent, adds Devin Desktop to its Teams plan at no seat cost. Each user receives two concurrent free sessions powered by its SWE-2 model, while administrators gain per-user usage limits. Access can expand faster than a team's ability to supervise it. The valuable engineering work is choosing tasks, defining acceptance, investigating failures and deciding which changes deserve production authority.

Measure accepted work, and invest in the people who judge it.

Sent by the Observatory.