The Co-Evolution of LLMs and Harness for Deep Research

LLMs, AI agents, deep research, AI co-scientists, OpenClaw, autoresearch: these terms are becoming saturated. Underneath them, however, the basic idea has remained largely the same. An LLM plus a harness forms an agentic system, and most agentic systems still follow some variation of the ReAct framework^[1]: the model reasons about what to do, takes an action, observes the result, and repeats. Deep Research, the subject of this article, is one such agentic application, but one with a specific purpose and its own technical challenges. I want to discuss how it developed because it was among the first strong practical applications of reasoning models. It has also become a useful model for subsequent work on long-horizon agents, including AI co-scientists and other emerging systems.

Workflow → Loop Engineering

In 2024 and before, LLMs were not yet very capable as agents. To make them reliable on a specific task, people often designed a chain of LLM calls or a relatively complex workflow around them. WikiChat^[2], for example, improves factuality through a manually designed pipeline of retrieval, generation, and verification. The workflow tells the model when to search, what to check, and when to revise its response. This reflects an early approach to managing LLMs during inference: the harness forces the model to search more deeply and verify its claims. The result quality improves, but the pipeline and the number of LLM calls must be specified manually.

Today, LLMs are increasingly used as agents. People are generally lazy and prefer an autopilot mode whenever it works, which makes flexible agentic orchestration more appealing than manually specifying every step. We give the model a set of tools and let it decide how to use them. This shift has been partly enabled by research on chain-of-thought reasoning^[3] and, later, test-time scaling in reasoning models^[4]. I think “loop engineering” is a useful name for this shift. We are no longer engineering a fixed workflow, but the loop in which the model can plan, act, inspect the result, and continue.

The key design problem then shifts to context management: how do we make sure the LLM chooses the right tool, progressively expose the context it needs, and strategically parallelize work through subagents? How do we enable the agent to reflect on its actions and deposit useful experience into memory?

AI Agents → Deep Research Agents

The evolution of LLMs points toward increasingly flexibility with less harness control, and eventually toward models capable of building their own. This is a process of co-evolution: humans identify tasks that current models struggle with, then create harnesses and skills to address them. The model and harness generate trajectories through real-world use, which developers turn into training data and pipelines for the next generation of models. As models learn from these trajectories and process-supervision data, workflows that once had to be specified explicitly can be internalized in the model parameters. Many general tasks may therefore be handled by a capable LLM inside a relatively simple loop, with little manual orchestration.

Deep research highlights one such capability: the sustained exploration of diverse knowledge sources to find, extract, and synthesize information. This is an important role for AI agents and one possible path toward more general intelligence. Deep research requires long-horizon exploration and reasoning across web pages, papers, databases, and user-provided files, ultimately producing a comprehensive report in response to a question. In principle, it uses the same ReAct loop as other agents. What sets it apart is how the system encourages the model to search longer, explore more directions, retain intermediate findings, and synthesize them at the end.

Element 1: Long-Horizon Research

Some LLM agents tend to simplify the task and use the minimum effort needed to produce a plausible answer. This behavior is reasonable for many everyday requests, but it is not ideal for serious research. If we ask an agent to conduct a systematic literature review, we do not want it to run two or three searches and immediately write a summary. A human-made review may require many rounds of query expansion, screening, citation tracing, and cross-checking.

The goal of long-horizon research is therefore not simply to generate more tokens, but to keep the agent working while useful research actions remain: broadening search coverage, following promising evidence, identifying missing perspectives, and revising the emerging answer. One lightweight approach is to build a harness that guides the model to structure its reasoning, search more extensively, and ultimately produce a stronger answer.

MindSearch^[5] illustrates this approach. Its WebPlanner decomposes a question into a dynamic search graph, while multiple WebSearcher agents investigate the resulting subquestions. This explicit structure encourages the system to explore several directions and integrate their findings instead of stopping at the first plausible search result.

When the capabilities of the base LLM become the bottleneck, further harness engineering may yield diminishing returns. The next step is to train the model itself to search and reason more effectively. Search-R1^[8], DeepRetrieval^[9], and s3^[10] explore this direction by designing reinforcement-learning rewards for different parts of the search process. More recent work extends this idea through long-horizon trajectory synthesis, as in OpenResearcher^[6].

OpenResearcher^[6] illustrates how long-horizon training data can be produced reproducibly and affordably. Creating a challenging research question is only the first step: training also requires a reliable ground-truth answer and a verifier that can evaluate the model’s response. The data pipeline must therefore sample suitable seed documents, generate high-quality questions and answers, and withhold answer-revealing content that could let the model exploit shortcuts. OpenResearcher runs this process offline over a 15-million-document corpus using three browser actions: search, open, and find. A teacher model generates more than 97,000 research trajectories—many containing over 100 tool calls—which are then used for supervised fine-tuning of a smaller model. The broader lesson is that improving long-horizon research depends as much on designing the task-generation and verification pipeline as on training the model itself.

Element 2: Coding Agents

At first glance, coding agents may seem unrelated to deep research, but they have substantially changed how research systems interact with external knowledge. Retrieval-augmented generation once centered on embedding documents and querying a vector index. Agentic search broadens this design space: when documents are available through a file system, operations such as grep, regular expressions, and SQL can be more direct and precise than semantic retrieval for many questions. Coding is also a general interface for virtual tasks and provides verifiable feedback during training. As a result, coding agents arrive with both stronger programmatic tools and a persistent workspace in which they can organize information beyond the context window.

Deep research can adopt the same architecture because it inevitably operates over large collections of documents and data. Instead of loading every retrieved item into the context window or exposing each operation as an isolated tool call, an agent can use code to search, filter, join, and summarize collections while storing intermediate results in files. Recent research^[13] evaluates this idea on long-context reasoning, retrieval-augmented generation, and open-domain question answering over corpora containing up to three trillion tokens. Across the tested benchmarks, off-the-shelf coding agents outperform prior published methods by 17.3% on average. The authors attribute this advantage to two capabilities that coding agents already possess: proficiency with executable tools and familiarity with navigating file systems. For deep research, this suggests that long-context processing can be externalized into explicit operations over persistent files rather than handled entirely through attention or semantic retrieval.

Paperclip is a system building deep research agents on top of coding agents. It gives coding agents command-line access to more than eight million papers through composable operations such as search, grep, map, sql, and figure inspection. Crucially, it preserves intermediate result sets, allowing an agent to narrow, revisit, and analyze the same collection across multiple steps. This turns literature research from a sequence of independent API calls into a stateful program that existing coding agents can explore directly.

Element 3: Heterogeneous Knowledge Sources

Scientific deep research extends far beyond searching the web or a collection of documents. Answering a single question may require evidence from publications, clinical trials, drug databases, protein resources, knowledge graphs, and specialized analytical tools. This heterogeneity introduces a new set of decisions for the agent: which sources should it consult, how should it connect evidence expressed in different schemas, and how should it allocate a limited search budget across them? Web and text search provide the foundation, but recent work increasingly focuses on agents that can navigate and integrate a broader ecosystem of domain-specific resources.

DeepEvidence^[12] asks what kind of harness is needed when deep research spans many heterogeneous biomedical knowledge sources. Its design centers on two decisions: how agents should search across those sources and which tools and execution environment should support them. The system uses a coding-agent-based, multi-agent architecture in which breadth-first agents survey multiple knowledge graphs, while depth-first agents follow promising entities and relationships. Its experiments show that programmatic tool use improves scientific evidence exploration and that combining broad coverage with targeted depth produces stronger results than either search strategy alone.

ToolUniverse^[14] illustrates the growing scale of the scientific tool ecosystem: it now provides a common interface to more than 1,000 models, datasets, APIs, and analysis packages. At this scale, access alone is not enough. Even a coding agent must determine which tools are relevant, how to sequence them, and whether their inputs and outputs are compatible. SciToolAgent^[15] addresses this orchestration problem with a knowledge graph that represents scientific tools and their relationships, enabling graph-based retrieval for tool selection and execution. Together, these systems suggest that tool discovery and orchestration are becoming core components of scientific deep research.

Element 4: Self-Evolving

Because a search agent consists of both a model and a harness, self-evolution can operate on either side. EvolveSearch^[16] improves the model by alternating supervised fine-tuning and reinforcement learning. Each iteration uses the current agent to generate new search trajectories, which become training data for the next iteration, allowing search behavior to improve without human-annotated reasoning traces. Harness-1^[11] makes the division between model and harness more explicit. The harness handles recoverable bookkeeping through a candidate pool, a curated evidence set, an evidence graph, a verification cache, and compressed observations. The policy is then trained to make the semantic decisions: what to search, which documents to keep or discard, what claims to verify, and when to stop. By removing routine state management from the policy, reinforcement learning can focus on search strategy rather than teaching the model to repeatedly reconstruct its own memory.

The harness itself can also become an optimization target. SkillOpt^[17] treats a natural-language skill file as the trainable state of a frozen model. An optimizer proposes bounded edits based on scored rollouts and accepts them only when they improve held-out validation performance. Agentic Harness Engineering^[18] expands this idea beyond instructions to tools, middleware, memory, and other runtime components. It converts execution traces into observable evidence, proposes harness changes, and tests whether each change produces the predicted improvement. For a broader overview of harness design, optimization, and self-improvement, see Lilian Weng’s recent essay, Harness Engineering for Self-Improvement.

What’s Next?

Deep research began as an extension of information retrieval, but it is increasingly becoming a general capability for acquiring, organizing, and reasoning over external knowledge. Behind this progress is a co-evolutionary loop mediated by people: users test agents and expose their limitations, engineers build harnesses to address them, and model developers use the resulting trajectories to internalize successful behaviors. This loop will continue well beyond deep research. Before agent research reaches its endpoint, however, several interesting questions remain.

First, which parts of the research process can—and should—be internalized by future models? Search strategies, tool selection, and context management may gradually move from explicit harness logic into model parameters. Yet some capabilities may resist internalization, while others should remain in the harness to preserve transparency, controllability, and monitoring. The goal is therefore not necessarily to eliminate the harness, but to find the right boundary between learned behavior and explicit system design.

Second, how should we evaluate open-ended research? As agents move beyond questions with easily verified answers, evaluation must consider evidence coverage, source quality, attribution, and whether the conclusions are genuinely supported. Many valuable research tasks still lack reliable outcome-based rewards, so progress will also depend on process-level evaluation of how agents search, select evidence, and revise their reasoning.

Finally, deep research may expand beyond finding information in the literature. Real scientific work also requires agents to acquire and interpret heterogeneous data, including datasets, tables, figures, and experimental outputs. Scientific agents will need to connect these sources, formulate hypotheses, run analyses or experiments, interpret the results, and revise their plans. At that point, deep research is no longer merely a retrieval application; it becomes a foundational capability of scientific AI.

References

1. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations. View paper
2. Semnani, S., Yao, V., Zhang, H., & Lam, M. (2023). WikiChat: Stopping the hallucination of large language model chatbots by few-shot grounding on Wikipedia. Findings of the Association for Computational Linguistics: EMNLP 2023, 2387–2413. View paper
3. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35. View paper
4. Snell, C., Lee, J., Xu, K., & Kumar, A. (2025). Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. International Conference on Learning Representations. View paper
5. Chen, Z., Liu, K., Wang, Q., Liu, J., Zhang, W., Chen, K., & Zhao, F. (2025). MindSearch: Mimicking human minds elicits deep AI searcher. International Conference on Learning Representations. View paper
6. Li, Z., Jiang, D., Ma, X., Zhang, H., Nie, P., Zhang, Y., Zou, K., Xie, J., Zhang, Y., & Chen, W. (2026). OpenResearcher: A fully open pipeline for long-horizon deep research trajectory synthesis. arXiv preprint arXiv:2603.20278. View paper
7. Wang, Z., Wang, H., Danek, B., Li, Y., Mack, C., Arbuckle, L., Biswal, D., Poon, H., Wang, Y., Rajpurkar, P., Xiao, C., & Sun, J. (2025). A perspective for adapting generalist AI to specialized medical AI applications and their challenges. npj Digital Medicine, 8, Article 429. View paper
8. Jin, B., Zeng, H., Yue, Z., Yoon, J., Arik, S., Wang, D., Zamani, H., & Han, J. (2025). Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516. View paper
9. Jiang, P., Lin, J., Cao, L., Tian, R., Kang, S., Wang, Z., Sun, J., & Han, J. (2025). DeepRetrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. arXiv preprint arXiv:2503.00223. View paper
10. Jiang, P., Xu, X., Lin, J., Xiao, J., Wang, Z., Sun, J., & Han, J. (2025). s3: You don’t need that much data to train a search agent via RL. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. View paper
11. Jiang, P., Shi, Z., Hong, K., Xu, X., Sun, J., Sun, J., Bashir, H., & Han, J. (2026). Harness-1: Reinforcement learning for search agents with state-externalizing harnesses. arXiv preprint arXiv:2606.02373. View paper
12. Wang, Z., Chen, Z., Yang, Z., Wang, X., Jin, Q., Peng, Y., Lu, Z., & Sun, J. (2026). Empowering biomedical evidence exploration and synthesis with deep knowledge graph research. Nature Machine Intelligence. View paper
13. Cao, W., Yin, X., Dhingra, B., & Zhou, S. (2026). Coding agents are effective long-context processors. arXiv preprint arXiv:2603.20432. View paper
14. Gao, S., Zhu, R., Sui, P., Kong, Z., Aldogom, S., Huang, Y., Noori, A., Shamji, R., Parvataneni, K., Tsiligkaridis, T., & Zitnik, M. (2025). Democratizing AI scientists using ToolUniverse. arXiv preprint arXiv:2509.23426. View paper
15. Ding, K., Yu, J., Huang, J., Yang, Y., Zhang, Q., & Chen, H. (2025). SciToolAgent: A knowledge-graph-driven scientific agent for multitool integration. Nature Computational Science, 5, 962–972. View paper
16. Zhang, D., Zhao, Y., Wu, J., Li, B., Yin, W., Zhang, L., Jiang, Y., Li, Y., Tu, K., Xie, P., & Huang, F. (2025). EvolveSearch: An iterative self-evolving search agent. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. View paper
17. Yang, Y., Gong, Z., Huang, W., Yang, Q., Zhou, Z., Huang, Z., Li, Y., Gao, X., Dai, Q., Liu, B., Qiu, K., Yang, Y., Chen, D., Yang, X., & Luo, C. (2026). SkillOpt: Executive strategy for self-evolving agent skills. arXiv preprint arXiv:2605.23904. View paper
18. Lin, J., Liu, S., Pan, C., Lin, L., Dou, S., Xi, Z., Huang, X., Yan, H., Han, Z., Gui, T., & Jiang, Y.-G. (2026). Agentic harness engineering: Observability-driven automatic evolution of coding-agent harnesses. arXiv preprint arXiv:2604.25850. View paper