Microsoft Agent Lightning v1.0: Train AI Agents Inside the Same Harness You Ship
Your AI agent already has a “body.” Training usually ignored it.
Modern AI agents are not just chat boxes. They live inside a harness — the software layer that manages tools, memory, sandboxes, and the step-by-step loop that turns a goal into actions.
That harness is where real work happens: opening files, running tests, searching the web, handing tasks to sub-agents. Yet for a long time, reinforcement learning (RL) training treated agents as if the model alone was the whole system. Teams often had to rebuild the agent loop inside the trainer — or train something that looked nothing like what they ship.
Microsoft Research and collaborators just published Agent Lightning v1.0, a lightweight open-source framework built around a simple idea: train the model using the same harness you deploy. They call this paradigm harnessed agentic RL.
The technical report — Agent Lightning v1.0: Towards Harnessed Agentic RL (arXiv:2608.17528) — shows the approach in practice, including a striking coding-agent result with only about 6,000 training examples.
What is Agent Lightning v1.0?
Agent Lightning is middleware between your existing agent and an RL training stack. Instead of rewriting OpenHands, mini-SWE-agent, or OpenClaw inside a trainer, you point the agent’s model calls at Agent Lightning’s proxy. The harness keeps doing its job. The trainer watches the requests and responses and improves the model from what actually happened in the wild loop.
The v1.0 redesign is intentionally small — roughly 3,500 lines of code — and ships with native Kubernetes support so agent rollouts can run as real jobs, not toy sandboxes bolted onto the trainer.
Real harnesses, zero rewrites
Agents keep their tools, context rules, and control flow. You switch the LLM endpoint to the proxy — not your agent architecture.
Lightweight by design
Simplicity is the design principle. A compact codebase is easier to reason about, audit, and extend.
Built for production clusters
Run agents locally for debugging or as Kubernetes Jobs when you need scale — without outsourcing the loop to a separate sandbox product.
How the pieces fit together
At a high level, three services connect the “agent world” to the “training world”:
| Piece | What it does |
|---|---|
| API Gateway | Proxies model calls and stores rollouts, events, and training traces |
| Rollout Controller | Launches and monitors agents (local process pool or Kubernetes) |
| Customized Trainer | Assembles samples and updates the policy (built on top of verl / vLLM) |
Agents with their harness sit on one side. Inference and training engines sit on the other. The gateway and controller keep both sides talking without forcing the agent loop into the GPU trainer.
Figure 1 from He et al., “Agent Lightning v1.0: Towards Harnessed Agentic RL” (arXiv:2608.17528).
Why “harnessed” training is a different game
In traditional agentic RL, the training engine often owns the interaction loop. The model sees one continuous token history growing turn by turn.
In harnessed agentic RL, the harness owns that loop. It builds each prompt, calls tools, manages context, and may spawn sub-agents. The trainer only sees a sequence of API-style request–response pairs — the same kind of boundary your product already uses.
That sounds like a small bookkeeping change. It is not. It changes how you assign credit, merge turns, and normalize losses when one task can explode into many training samples. The paper walks through those pitfalls carefully; Agent Lightning v1.0 is also a testbed for getting them right.
Figure 2 from He et al., “Agent Lightning v1.0: Towards Harnessed Agentic RL” (arXiv:2608.17528).
Why this matters for builders
Close the train–serve gap
You improve the model in the same scaffolding users hit in production — not a simplified stand-in.
Keep your favorite agent stack
Works with real coding and general harnesses instead of forcing a single “trainer-native” agent rewrite.
Scale rollouts independently
Agent execution and GPU training can live on different resources and grow on different schedules.
Reproduce coding-agent RL
The team released data cleaning, reward-hacking safeguards, and training scripts — not just a chart.
The headline result: coding agents get better with modest compute
The authors evaluate instruction-following, search, and coding agents. The coding story is especially useful for teams who have found RL tooling thin for software agents.
Using the open SWE-smith dataset and Qwen3.5-9B, reinforcement learning with Agent Lightning moved SWE-bench Verified from 41.8% to 56.4% — an absolute 14.6-point gain — with only about 6,000 training examples and what the authors describe as modest compute.
| Setting | Detail |
|---|---|
| Model | Qwen3.5-9B |
| Benchmark | SWE-bench Verified |
| Before RL | 41.8% |
| After RL | 56.4% |
| Absolute gain | +14.6 points |
| Training scale | ~6K examples, modest compute |
| Framework size | ~3,500 lines of code |
That is not “throw infinite GPUs at a closed stack.” It is a reproducible path for labs and product teams who want agents that learn from real harness behavior.
Who should pay attention
Agent product teams
If you already ship a coding or research agent, this is how you improve the model without discarding the harness you depend on.
RL and platform engineers
If you maintain training backends, the paper’s discussion of sample merging, advantages, and loss normalization is practical, not theoretical fluff.
Open-source builders
Code, docs, and training scripts are public under an MIT-friendly open posture — start from the repo and docs, not a slide deck.
Where to go next
- Technical report: arXiv:2608.17528
- GitHub: microsoft/agent-lightning
- Documentation: Agent Lightning v1.0 docs
- Earlier Microsoft Research overview of the approach: Agent Lightning blog
Agent Lightning v1.0 will not make every agent “smart overnight.” What it does is quieter and more important: it lets the model practice inside the same harness that runs in production — so training starts to look like real use.
