Programmatic memory keeps the evidence an agent may need later
PRO-LONG shows a concrete long-horizon agent pattern: keep a lossless interaction log, search it with code, and expose only verified evidence to the next action.
The product implication
For a long-running agent, the most important memory decision may happen before retrieval: do you throw information away when you write it?
A new Duke University preprint, PRO-LONG, takes the deliberately unfashionable route. It appends every observation, action, outcome, and short plan to a structured log, then lets a coding-capable agent search that log with tools such as
grep and Python. On the 25-game public set of ARC-AGI-3, the authors report an average 18.0 percentage-point improvement over the same base coding agents, with 4.2 to 5.8 times fewer billed tokens than specialized harnesses in matched comparisons. Those are preprint benchmark results, not a production SLA, but the design challenge is relevant to any team building an agent that must recover facts several steps after they appeared. 1The PM decision is not "summaries or vector databases are obsolete." It is whether your workflow has a class of facts whose future importance cannot be known at write time. If it does, keep a lossless evidence layer and make the agent retrieve from it in bounded, inspectable ways. Expose only a compact, verified state to the next action.
What changed
Memory moves from write-time judgment to read-time search
Most memory systems make an early choice about what to retain: summarize the transcript, extract a fact, create an embedding, or save a note. That reduces the next prompt, but it can also discard a detail whose importance becomes clear only after the agent explores further.
PRO-LONG changes the tradeoff. Its write operation is simple append-only logging. Its read operation is programmatic search over the log. The authors describe this as a complete record that remains accessible through code, with retrieval still tractable for logs exceeding 100,000 lines. In one reported ARC-AGI-3 example, a single run exceeded 320,000 lines, while the agent used the history to reason about a game involving rewinding and multiple "ghost" paths. 1
That distinction matters because the agent is not being asked to reread its entire past. It is being given a searchable archive and the ability to write a small query or parser when the current problem reveals what matters. The archive preserves recall; the code supplies selectivity.
The result is driven by analysis, not note-taking
The paper's ablation is unusually useful for product design. With GPT-5.5 on the benchmark, the reported score rose from 23.1 with read-only access to 27.2 after adding grep and regular-expression search, 38.3 after adding Python, and 41.2 after adding write/edit tools. The largest jump came from programmatic analysis, not from giving the agent a richer place to write notes. 1
The same pattern appears in the workspace test. Clearing the workspace between model calls reduced PRO-LONG's score from 41.2 to 40.7, while the no-log baseline fell from 24.0 to 19.9. That is evidence for a specific mechanism: the complete interaction log, rather than generic persistence, carried most of the gain in this setting. 1
The strongest reported results also need careful framing. On GPT-5.5, PRO-LONG reached 41.2% pass@1 and 60.1% best@5, compared with a prior best of 45.1% in the paper's comparison. With Opus 4.6, it reached 42.4% pass@1. With Fable 5, the paper reports 97.4% best@2 at a total cost of $1,750, while noting that this best@2 figure is a lower bound because not every game was rerun. The benchmark has high run-to-run variance, so these scores describe a scaffold under a particular action budget and model setup, not the reliability of a general agent product. 1
A second layer is still needed for execution
A raw log is good at preserving evidence. It is not, by itself, a safe execution policy.
Google's ADK 2.0 makes the complementary case. Its July 1 engineering post separates deterministic workflow routing from the language-model steps that require interpretation. Tool calls, conditional branches, retries, and human approvals can stay in code, while an LLM handles an ambiguous input or drafts a response. In an illustrative mock refund workflow, Google reports a reduction from 5,152 to 2,265 tokens and from 7.2 to 5.7 seconds when deterministic steps replace model-controlled orchestration. The figures are explicitly illustrative, using a mock API and Gemini 3.5 Flash, so they are an architecture example rather than a general cost claim. 2
Put the two ideas together and the architecture is clearer:
| Layer | Job | What can go wrong |
|---|---|---|
| Evidence log | Preserve observations, actions, results, and short plans | Retention cost, sensitive-data exposure, slow or noisy retrieval |
| Retrieval program | Find the small set of historical evidence relevant to the current decision | Bad queries, irrelevant matches, excess context |
| Verified state | Carry current requirements, accepted values, and completed checks | Stale state or false completion |
| Workflow runtime | Decide which tools and approvals are reachable next | Graph maintenance and overly rigid paths |
This is a synthesis from the two sources, not an experiment they ran together. PRO-LONG addresses information loss during long exploration. ADK 2.0 addresses uncontrolled execution and context bloat. A production system can use the first as a durable evidence plane and the second as the boundary around actions.
OpenAI's GPT-5.6 release points in the same implementation direction from a model-runtime angle. Its Programmatic Tool Calling lets a model write and run lightweight programs that coordinate tools and process intermediate results, rather than passing every intermediate payload through another model turn. That makes code-managed filtering a vendor-supported product pattern, although the release's own benchmark and customer claims should not be treated as independent validation of PRO-LONG. 3
Why PMs should care
The useful question is not whether an agent has "memory." That word can mean personalization, retrieval-augmented generation, transcript summaries, saved preferences, or durable task state.
Ask instead: which facts must remain recoverable even if their relevance is discovered late?
A customer-support agent may need the exact failed troubleshooting steps, not only a summary that says the issue was unresolved. A data-cleaning agent may need the original schema, excluded rows, and failed transformations. A research agent may need the source passage that caused it to reject an earlier hypothesis. In each case, a summary can be a useful working view, but it should not be the only record from which recovery is possible.
There is also a cost distinction. PRO-LONG's paper shows that the agent's ability to analyze the log matters more than merely keeping a persistent workspace. That suggests measuring memory as a retrieval-and-recovery system, not as a storage feature. Track whether the agent can locate the right evidence, how much context it needs to act on it, and whether a contradiction causes repair or silent continuation.
A final caution: the benchmark's environment is a sandbox, and the paper does not establish that append-only logs are safe for user data. Keeping everything can make deletion, retention, access control, and redaction harder. A log that improves recall but exposes credentials or private documents is not a product win.
How to implement now
- Choose a reversible workflow with real long-horizon failures. Start with spreadsheet cleanup, QA triage, internal IT troubleshooting, or research assembly. Avoid a first pilot that can issue refunds, change permissions, or send external messages without approval.
- Define an append-only event schema. Record the timestamp, observation, action, tool result, short plan, and run identifier. Store raw evidence separately from the compact state projection. Redact secrets before persistence, set retention rules, and make deletion auditable.
- Give the agent bounded programmatic access. Start with read-only search over a single run. Add Python or another sandboxed analysis tool only after you can log the query, returned lines, and token cost. Do not allow arbitrary network access from the retrieval environment.
- Keep the next model call small and verified. Retrieve a focused evidence bundle, then pass it with current requirements, accepted values, pending checks, and invalidated attempts. Only a verifier or deterministic tool should mark a requirement complete. The actor can propose progress; it should not certify its own success.
- Compare against your current memory design. Use the same model, task distribution, tools, and rollout gates. Measure task completion, false completion, repeated failed actions, recovery after contradictory evidence, retrieval latency, context tokens, billed tokens, human takeover, and data-policy violations. Look at performance after 20, 40, and 80 actions, not only at the first few turns.
- Roll out in shadow mode first. Let the log-backed agent retrieve and recommend while the existing system remains authoritative. Promote it to write actions only when recovery and audit metrics beat the summary or vector-memory baseline on the same workload.
The paper's public repository makes a small technical pilot feasible: it uses a Docker sandbox, an OpenCode analyzer, and ordinary Read, Grep, and Python tools to produce batched action plans. The queue drains actions without another model call until it empties or the score changes. That is a useful reference implementation for the control loop, not a drop-in enterprise memory service. 4
A recent X post from Rohan Paul gave the paper meaningful attention, with 5,845 views, 58 likes, and 11 reposts when retrieved on July 24. It is a useful signal that the idea is legible to AI practitioners, not independent evidence for the benchmark. 5
No paper-specific YouTube explainer surfaced in the search. The closest accessible primer was a July 13, 2026 weekly AI-papers video from Xiaol.x with 104 views; its chapters cover adjacent ideas including durable state, memory, retrieval, and harnesses rather than PRO-LONG itself. Use it for orientation, not as evidence of adoption. 6
The near-term product bet is therefore modest: keep the full evidence, project a small verified state, and make retrieval an observable tool call. If that combination reduces false completion and recovery time on your own long-running workflow, then the memory system has earned a larger pilot. If it only increases storage and retrieval complexity, keep the simpler baseline.
References
- 1
- 2Why we built ADK 2.0
developers.googleblog.com
- 3
- 4PRO-LONG repository
github.com
- 5
- 6
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.