Quick answer
Amazon Bedrock AgentCore Gateway rate limiting lets platform teams set customer-defined traffic ceilings for Gateway targets without adding throttling logic to each agent or tool.
Use RPS or RPM to bound request arrivals, TPM to bound input-plus-output token throughput on inference targets, and CPS to protect capacity held by long-lived connections. Group buckets by stable target or identity dimensions, include a wildcard fallback, and verify the result against service-managed quotas.
AWS documents rate-limit evaluation as fail open and earlier than AgentCore Policy. Treat it as traffic management and quality of service, not as your only authentication, authorization, abuse, or cost control.
Choose the metric by failure mode
| Failure mode | Primary metric | Target scope |
|---|---|---|
| Retry loop or request burst | RPS / RPM | MCP, inference, and HTTP targets |
| Reasoning-heavy model traffic | TPM | Inference targets only |
| Slow stream or long-held session | CPS | MCP, inference, and HTTP targets |
One request counts once toward RPS or RPM regardless of duration. TPM includes input and output tokens: Gateway estimates and deducts incoming tokens before dispatch, then reconciles using provider-reported usage after the response. A long-lived request occupies connection capacity for its duration, so CPS protects a different resource than request count.
Build buckets from dimensions and entries
A configuration has:
- dimension keys, which define how Gateway groups requests into buckets;
- entries, which match dimension values and set request, token, or connection rates.
AWS documents target dimensions such as targetName, toolName, and qualifiedModelId; JWT claims through $.context.jwt.<claim>; and IAM dimensions through $.context.iam.principal and $.context.iam.sourceIdentity.
Specific entries take precedence over wildcard entries. A * fallback gives each distinct matched value an independent bucket at the configured rate. Without a catch-all, unmatched callers or targets can skip that customer-defined limit and fall through to service quotas.
Layer limits deliberately
Common layers include:
- a group or tier ceiling for shared allocation;
- a per-user ceiling to prevent one user from exhausting the group;
- a target ceiling to protect a backend;
- a combined user-target or role-model bucket for workload isolation;
- service-managed quotas as the account-level ceiling.
When several customer-defined limits match, each independently constrains the request. AWS says more-dimension configurations are evaluated first; within the same dimension count, tighter rates are evaluated first; evaluation stops on the first denial.
Preserve authorization and identity boundaries
Rate limits run before AgentCore Policy. A caller that Policy later rejects can already consume bucket capacity. AWS recommends explicit zero-rate entries for identities or groups that should not reach a target, while keeping AgentCore Policy as the actual authorization layer.
For agent-to-downstream traffic, determine whose identity is present. An on-behalf-of token exchange carries the original user’s identity, so user buckets can follow downstream calls. A machine-to-machine grant carries the agent’s identity instead; use a stable agent claim such as the provider’s authorized-party claim when appropriate.
Read the OBO versus M2M guide before assuming a caller’s per-user bucket also covers every tool and model invoked by an agent.
Safe rollout checklist
- Inventory target types, service quotas, expected bursts, tokens, and connection duration.
- Select stable, bounded dimensions; avoid nonce, request ID, or other unbounded claims.
- Add explicit entries for exceptions and a catch-all for everything else.
- Verify wildcard order and most-specific matching.
- Test authorized, denied, unmatched, exhausted, OBO, and M2M requests.
- Enable Gateway application logs and OpenTelemetry rate-limit attributes.
- Alert on throttling, unexpected fall-through, service-quota pressure, and identity drift.
- Keep authentication, Policy, WAF, downstream quotas, retries, and backoff independent.
Use the AgentCore rate-limit planner to estimate aggregate demand and document the bucket design before using the AWS CLI or API.
Frequently asked questions
What can AgentCore Gateway rate limits control?
Customer-defined limits can control request rate for all target types, token throughput for inference targets, and connections for all target types, grouped by supported target or identity dimensions.
Do customer-defined limits replace AgentCore service quotas?
No. Customer-defined limits are evaluated first and service-managed quotas remain the ceiling. Effective capacity is bounded by the lower applicable limit.
Are AgentCore Gateway rate limits a security boundary?
No. AWS documents fail-open evaluation. Use rate limits for traffic management and quality of service, with authentication, authorization, and AWS WAF for security enforcement.
Are rate limits evaluated before AgentCore Policy?
Yes. AWS says rate limits run before AgentCore Policy, so a request later denied by Policy can already have consumed its rate bucket.
Continue by decision
- RPS, RPM, TPM, and CPS explained
- OAuth and IAM dimensions
- OBO versus machine-to-machine limits
- AgentCore rate-limit planner
Official sources
- AWS: Configure rate limits for AI traffic on AgentCore Gateway
- AWS: New AgentCore temporal policy and rate-limiting capabilities
- Amazon Bedrock AgentCore Gateway rate limits
Source check: August 8, 2026. Verify current Regions, APIs, dimensions, quotas, logging, enforcement behavior, and pricing before production use.