Quick answer
The Orchard-SWE recipe uses several methods for a core agent-training problem: a long trajectory may contain useful decisions even when the final task fails, while final pass/fail feedback is too sparse to explain which step mattered.
The published sequence is:
- credit-assignment supervised fine-tuning to retain productive portions of partial attempts;
- Balanced Adaptive Rollout for sparse outcome feedback;
- on-policy distillation and a process reward model for denser step or process guidance;
- a separate value model to rerank candidate solutions at inference time.
These methods solve different problems and should be measured separately. Do not describe the final reranked score as if it came from the base policy with one attempt.
Method decision table
| Method | Intended use | Main risk |
|---|---|---|
| Credit-assignment SFT | Recover useful signal from partial trajectories | Incorrectly crediting steps that only appear productive in hindsight |
| Balanced Adaptive Rollout | Allocate RL effort under sparse rewards | Changing the sampled task distribution or comparison budget |
| On-policy distillation | Add teacher feedback on current decisions | Importing teacher errors and preferences |
| Process reward model | Reward behaviors such as reproducing and testing a bug | A plausible process can still produce an incorrect patch |
| Value reranking | Select a candidate from multiple rollouts | Higher inference cost and an unfair comparison if candidate counts differ |
Measure each stage
Use one immutable holdout and report:
- task and trajectory counts, filtering, and leakage checks;
- model, harness, environment, and trainer versions;
- reward definitions and judge prompts;
- rollout and candidate budgets;
- base, post-SFT, post-RL, and reranked results;
- variance across seeds or repeated evaluations;
- manually reviewed reward-model false positives and false negatives.
Microsoft reports Orchard-SWE moving from a 61.4% baseline to 69.1% with Balanced Adaptive Rollout, 69.7% with dense-reward techniques, and 73.0% with value reranking on SWE-bench Verified. Those stages are useful research evidence, but reproduction requires the full setup rather than copying only the headline percentages.
Credit assignment is not authorization
A reward can say that a step helped solve a benchmark task. It cannot establish that the command was authorized, the data was allowed, the patch is secure, or the production change should ship. Keep environment permissions and human approval independent from the learning signal.
For the domain workflow, read the Orchard-SWE guide. For planned environment snapshots and branching described in the repository roadmap, start with the Orchard Env guide.
Frequently asked questions
What is credit-assignment SFT in Orchard?
It is a supervised fine-tuning approach intended to learn from productive portions of agent trajectories even when the complete attempt does not resolve the task.
What is Balanced Adaptive Rollout?
Microsoft describes Balanced Adaptive Rollout as a reinforcement-learning technique designed to use sparse success signals more effectively. Its behavior and benefit should be evaluated from the published method and reproduced under the target setup.
Is a process reward or value model ground truth?
No. A teacher, process judge, or value model is another learned estimator. It can improve training or selection while still rewarding plausible but incorrect behavior.
Official sources
Source check: August 4, 2026. Verify the paper, code revision, data, reward definitions, rollout allocation, candidate count, and benchmark protocol before reproducing the method.