Agents are still losing the thread

A July 20 update to Long-Horizon Terminal-Bench puts agents through 46 containerized tasks with a 90-minute budget, hidden verifiers, and hundreds of dependent steps. Its current repository snapshot reports that Grok 4.5 averages 0.505 reward and solves 13 of 46 tasks, while 29 tasks remain unsolved by every model. 1 2

That is the clearest signal in this window. Agent systems are gaining more ways to remember, act, and move money, but the hard measurement is still whether they can sustain useful work until an external system says the job is done.

The project is timely because its repository received multiple updates on July 20, including a harness patch and a README update. The benchmark keeps an agent inside a stateful container, lets it continue working until the timeout on 30 of its 46 tasks, and grades artifacts with hidden, rebuild-from-artifact verifiers. A self-reported "done" does not count. 1 2

The current snapshot reports 21 frontier models under the same Terminus-2 harness. Grok 4.5 leads at 0.505 mean reward with 13 tasks at reward >= 0.95. MiniMax M3 scores 0.385 at an estimated $6.13 per task, while Claude Fable 5 scores 0.487 at $73.11 per task. The repository says 29 of 46 tasks are still unsolved by every model. These are benchmark-reported figures, not a general ranking of models in production. 2

There is already version drift that builders should record in their own evaluation metadata. The linked arXiv v2 paper describes a 15-model evaluation and reports 15.2% pass@1 at a partial-reward threshold of 0.95 and 10.9% at a perfect-reward threshold of 1.0. The repository's later July snapshot lists 21 models and 13 of 46 tasks solved by its current leader. The numbers are not interchangeable. 6 2

For teams building agents, the useful design choice is the grading model. Binary pass/fail would turn most of these runs into zeros. Dense reward records whether the agent built part of the system, recovered from a failed experiment, or made progress before timing out. That gives an engineering team a way to compare state management, retry policy, context compression, and model routing without pretending that partial work is a completed task.

What to copy: pin the benchmark commit, harness version, model endpoint, timeout, and reward threshold in every run record. A leaderboard without those fields will age badly as the task suite changes.

Agentic OS released v1.8.15 on July 19 with a narrow but useful correction: every always-loaded rule now carries either a verifiable enforcement tier or an honest label saying that it has none. Its ADR-011 release note describes a 112-row semantic census across four phase-entry surfaces, the removal of two false gate advertisements, and the merging of three duplicates. 3

The distinction matters because a rule file and an enforcement hook have different powers. AGENTS.md, CLAUDE.md, or a similar instruction can tell an agent to run tests, record evidence, and ask for review. It cannot prove that those steps happened. Agentic OS puts checks in git hooks and CI, including credential scanning, test execution, and validation of the work trail. The release also adds a directive-count ratchet that fails only when the current baseline grows. 3

This is a governance release, not a new agent runtime. Its value depends on whether the checks are mandatory in the deployment path. The repository says the local pre-commit hook is opt-in, while required CI checks provide the harder boundary. That makes the installation and branch-protection configuration part of the product, not a footnote. 7

What to copy: separate guidance from gates in your architecture document. For every completion claim, name the machine-checkable evidence, where it is recorded, and which control prevents a skipped check from being merged.

Natural announced a $30 million Series A led by Forerunner on July 20, bringing its total funding to more than $40 million. The company says it was founded in 2025 by Kahlil Lalji, Eric Wang, and Walt Leung and has grown to 17 employees. TechCrunch reports that Natural had been in beta and is positioning itself against Stripe and other companies rebuilding payment rails for autonomous transactions. 4 8

The product scope is broader than adding a card to an agent. Natural says 13 products are planned, with six generally available: Wallets, Vaults, Pay, Request, Transfer, and Connect. It describes the underlying stack as ledgering, money movement, multi-bank settlement, multi-currency support, fraud and compliance, agent identity, and observability. Voice, Accept, and Cards are planned for the coming months; Charge, Credit, Direct, and Billing are planned for Q4. These are company-reported product plans, not evidence of adoption or transaction volume. 4

The technical question for builders is authorization, not checkout design. An agent that can request a payment still needs a bounded identity, a funding source, a policy for recipients and amounts, a settlement receipt, and a way to handle disputes. Natural's product list suggests that the market is assembling those primitives as a dedicated stack. It does not yet answer how portable those controls will be across banks, currencies, agent runtimes, and payment networks.

What to copy: model an agent payment as a state transition with an approval policy and an external receipt. Do not treat a successful API response as proof that funds moved or that the intended party received them.

Braintrust published an independent comparison of Paper MCP and Figma MCP for frontend agents on July 20. Each run gave a coding agent a screenshot and required it to build a single self-contained HTML page through one assigned design MCP server. The evaluation used 40 simple pages and 27 complex designs, with three trials per tool for the complex set. 5

On the simple pages, visual similarity was effectively tied: Paper scored 0.741 and Figma 0.744. On the complex set, Paper scored 0.716 and Figma 0.679, but the paired difference did not reach statistical significance at p = 0.21. The more repeatable differences were operational. Figma's runs varied about 1.9 times more, took 42% longer, and cost 32% more per point of visual quality in this evaluation. 5

The output code also differed. Figma produced semantic HTML tags at more than three times Paper's rate, while Paper's HTML-first canvas gave the agent a more direct path from layout to shipped CSS. That is a real tradeoff: the tool with the more stable visual workflow did not automatically produce the more maintainable or accessible markup. 5

The evaluation also found that agents took screenshots of their own work without being explicitly told to create a self-correction loop. Yet screenshot frequency was almost uncorrelated with final quality in the captured runs. A visible feedback loop is not the same as a useful one. The agent needs a scorer that detects the failure mode it is trying to correct, not just another screenshot.

What to copy: score agent workflows on quality, variance, runtime, cost, and generated artifact quality. A single best run can hide a workflow that fails unpredictably on the next design.

These four updates sit at different layers, but the engineering lesson is consistent. LHTB checks whether a long sequence ends in a replayable artifact. Agentic OS checks whether a coding workflow has evidence outside the model's own report. Natural is building payment primitives around identity, settlement, and compliance. Braintrust shows that tool choice changes variance and code quality even when average visual scores look similar.

The practical stack now needs at least two separate records:

The first record explains why an agent behaved as it did. The second answers whether the work was completed. Keep them linked, but do not collapse them into one success flag. The current LHTB version drift is a reminder that even benchmark results need their own version and acceptance metadata before they can support a decision.

For the next agent workflow you ship, write the external done condition first. Then choose the smallest evidence record that can verify it, and make the gate independent of the agent's final sentence.