Quick answer
Orchard-GUI is the browser-agent recipe in Microsoft Orchard. Microsoft describes a 4-billion-parameter vision-language model trained from 400 distilled demonstrations and 2,200 open-ended tasks, then evaluated on WebVoyager, Online-Mind2Web, and DeepShop.
The reported scores are 74.1%, 67.0%, and 64.0%, respectively, with a 68.4% average in the published comparison. Treat them as results for the documented research setup. Live sites change, judge quality varies, and a successful benchmark task does not imply safe action on a user’s account.
What the recipe is testing
Browser agents combine several error sources:
- visual grounding and page interpretation;
- state tracking across multiple turns;
- interaction with dynamic or delayed interfaces;
- recovery from navigation, timeout, and website failures;
- a final judge deciding whether the task succeeded.
The related OpenWebRL work described in the repository adds online multi-turn RL, multimodal context management, navigation retries, timeout handling, and structured failure attribution. That separation matters: a site outage should not automatically become negative model feedback.
Build a safe evaluation
- Pin the browser, viewport, model, harness, prompt, and Orchard revisions.
- Separate deterministic local fixtures from live-site evaluation.
- Record page state, actions, screenshots, tool errors, and judge evidence.
- Label infrastructure, website, policy, and model failures separately.
- Use test accounts and reversible tasks; block purchases, messages, deletions, and permission changes.
- Re-run a stable holdout after every policy or model update.
- Manually review apparent successes, especially when a judge relies on page text.
Do not treat visual access as action authorization. Authentication state, sensitive data scope, destination, spending, and irreversible operations need explicit controls outside the policy model.
When to use Orchard-GUI
Use the recipe when the research question is browser navigation under multimodal, multi-turn conditions. Use Orchard Env when the main issue is isolated execution infrastructure, or the Claw guide when email, calendars, and productivity tools define the task.
For training inside the same agent shell used later, read the Orchard harness guide.
Frequently asked questions
What is Orchard-GUI?
Orchard-GUI is Microsoft’s browser-navigation recipe built on Orchard Env. It trains a vision-language model with distilled demonstrations and open-ended web tasks.
How much training data does Microsoft report for Orchard-GUI?
The Microsoft Research post reports 400 distilled demonstrations and 2,200 open-ended training tasks. The repository’s released GUI dataset is larger because it also contains judge-verified rollout records; pin the exact subset used for each experiment.
Does an Orchard-GUI benchmark result authorize live browser actions?
No. A benchmark result does not grant account, data, purchasing, communication, or destructive-action authority. Production controls must be defined and tested separately.
Official sources
Source check: August 4, 2026. Verify the dataset split, live-site conditions, browser and harness versions, judge, retries, task permissions, and benchmark rules before comparing results.