Quick Answer
Google’s adaptive rubrics generate a tailored set of pass-or-fail tests for an individual prompt, then grade the response or agent trace against those tests with rationales. They can make open-ended evaluation more specific and explainable than one generic score.
They are still generated judgments. Review important rubrics, freeze them when comparing versions, and pair them with deterministic checks and human audits for consequential decisions.
How Adaptive Rubrics Work
The Google Gen AI evaluation service documents a two-step process:
- Rubric generation: analyze the prompt and create specific, verifiable tests a good response should meet.
- Rubric validation: assess the generated response against each test and return a pass or fail verdict with a rationale.
The result includes an aggregate pass rate and the individual rubric verdicts. A 75% pass rate means the response passed three quarters of that rubric set; it does not mean the answer is “75% true” or that the agent succeeds on 75% of production tasks.
Agent-Specific Metrics
Google documents adaptive-rubric metrics for different layers of agent behavior, including:
- final-response quality;
- hallucination or grounding relative to the agent configuration and tool use;
- tool-use quality;
- multi-turn task success;
- multi-turn tool-use quality;
- multi-turn trajectory quality.
Use the narrowest metric that matches the decision. A polished final answer can hide a dangerous tool call, while a correct tool sequence can still produce an unsupported conclusion.
Generated, Guided, or Static?
| Approach | Best for | Tradeoff |
|---|---|---|
| Adaptive rubrics | Diverse prompts where the criteria should follow each task | Generated criteria can vary or miss domain policy |
| Adaptive rubrics with guidelines | Stable business priorities expressed in natural language | Guidelines still need validation and versioning |
| Pre-generated, reused rubrics | Fair A/B or release comparisons | Less adaptive to new prompt types |
| Static custom rubric | Specialized policy or domain requirements | More authoring and calibration work |
| Deterministic metric | Exact values, schemas, permissions, or reference answers | Cannot capture every qualitative outcome |
Google recommends its general-quality adaptive rubric as a starting point for model evaluation. For agent decisions, add the agent-specific metrics that correspond to the tool and trajectory risks you actually have.
Review Rubrics Before Trusting Scores
For a high-impact sample, ask:
- Does each rubric follow from the user request or approved policy?
- Is it observable from the trace, response, and available configuration?
- Does it reward the correct outcome instead of a particular writing style?
- Could two rubrics double-count the same behavior?
- Is a critical safety or permission condition hidden in an average?
- Would a subject-matter reviewer agree with the verdict and rationale?
Reject criteria that require information the evaluator cannot see. Separate “not demonstrated” from “false” when the trace lacks evidence.
Make Comparisons Fair
If version A and version B receive newly generated rubric sets, score movement may reflect rubric variation. Google documents generating rubrics separately and reusing them during evaluation. Use that path for a controlled comparison:
- generate rubrics from the approved prompt set;
- review and version the rubric groups;
- run both agent versions on the same cases;
- validate both responses against the same rubric groups;
- inspect per-rubric regressions before comparing aggregate pass rates.
Record the metric name, version, evaluator model, guideline text, dataset revision, and evaluation date.
Calibrate With Human Review
Build a small gold set scored by qualified reviewers. Compare the automated verdicts with human decisions by failure type, not only overall agreement.
Track false passes and false failures separately. A false pass on authorization or harmful action can matter more than several false failures on tone. Define when a human must override or investigate an automated grade.
For the complete evaluation lifecycle, use the Google Agent Platform Evaluations GA guide. To decide where the rubrics run, see offline versus online agent evaluation.
Boundaries
Adaptive rubrics can improve specificity and diagnostic detail. They do not establish an objective ground truth for every task, eliminate evaluator bias, guarantee repeatability, or prove regulatory compliance.
Do not publish provider-reported methodology as an independently measured business outcome. Test the current service with representative prompts, languages, tools, and failure severity from your own deployment.
Official Sources
- Gen AI evaluation service overview
- Define evaluation metrics
- Managed rubric-based metric details
- Run an evaluation and reuse generated rubrics
Source check: August 1, 2026. Metric names, implementations, evaluator models, cost, and service behavior can change; verify them before setting release gates.