Skip to content
⚠️ Before You Start
Risk: High
Duration: 2-5 days including canary
Last Validated: Not yet validated (draft)
Kind: Runbook
Risk Note: High (agents call the tool with old arguments and fail or act wrongly)

Ship a breaking tool schema change with versioning and canary

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-11-24). Author: James · dry-run: Sivaraj · sign-off: Yogeshwaran. Work items: AGT-02-T09, AGT-02-T10, AGT-02-T11, AGT-02-T12.

1. Summary

When a tool's input or output changes in a way old callers cannot handle, or when a downstream API version is retired.

Risk: High (agents call the tool with old arguments and fail or act wrongly). Typical duration: 2-5 days including canary.

2. When to use and when not to

Use it when: When a tool's input or output changes in a way old callers cannot handle, or when a downstream API version is retired.

Do not use it when: to be written by the author.

3. What you must know first

  • Breaking changes get a new major version; old versions stay until sunset
  • Clients may cache tool lists; MCP servers can send list_changed notifications
  • Model behaviour depends on descriptions as much as schemas
  • Canary by agent or tenant, not by random request, so threads stay consistent

4. Prerequisites and access

  • Registry entry and owner for the tool
  • List of agents and prompts that use it
  • Regression dataset covering the tool
  • Change approval

5. Steps

Outline of the steps; the author adds the commands and a check after each one.

  1. Publish the new major version alongside the old one
  2. Mark the old version deprecated with a sunset date
  3. Update one agent's tool binding and prompt to the new version in a branch
  4. Run the regression dataset and compare tool-call success
  5. Canary the new version to one agent or tenant
  6. Monitor validation failures and tool errors per version
  7. Move remaining agents in batches
  8. Remove the old version after sunset and confirm zero calls
  9. Rollback: point agents back to the old version, which stays live until sunset

Draft snippet: Registry entry with two versions, deprecation and canary

{
  "name": "orders.refund",
  "owner": "team-payments",
  "scopes": ["payments:refund"],
  "versions": [
    {
      "version": "1.4.0",
      "status": "deprecated",
      "sunset": "2026-12-31",
      "inputSchema": {"type": "object", "required": ["order_id", "amount"],
        "properties": {"order_id": {"type": "string"}, "amount": {"type": "number"}},
        "additionalProperties": false}
    },
    {
      "version": "2.0.0",
      "status": "active",
      "rollout": {"strategy": "canary", "percent": 10, "agents": ["support-agent"]},
      "inputSchema": {
        "type": "object",
        "required": ["order_id", "amount_minor", "currency"],
        "properties": {
          "order_id": {"type": "string"},
          "amount_minor": {"type": "integer", "minimum": 1},
          "currency": {"type": "string", "pattern": "^[A-Z]{3}$"}
        },
        "additionalProperties": false
      }
    }
  ]
}

Snippets to add:

  • snippets/registry/tools/orders.refund.json: Versioned registry entry with deprecation and canary (json)
  • snippets/ops/tool_version_usage.py: Version usage report (python)

6. Monitoring and alerts

Signal Alert when Action
Calls per tool version Old version calls after sunset date Find the caller and block removal
Validation failures on the new version Above 5 percent during canary Pause the rollout and fix descriptions

7. Rollback

To be written by the author: how to undo each risky step.

8. Known issues and fixes

Symptom Likely cause Fix
Model keeps sending old field names Prompt or examples still mention the old fields Update few-shot examples and tool descriptions
Some clients still see the old tool list Cached tools/list results Send list_changed where supported and restart long-lived clients
Calls to the old version continue after sunset Unknown consumer Log caller identity per call and contact the owner before removal

9. AWS and open-source equivalents

AgentCore Gateway targets and API Gateway stages can host versions side by side. In open source, version tools in the server name or tool name and route with an MCP-aware gateway.

10. Validation record

Pending. Planned dry-run scenario: A second engineer performs a breaking change on a sample tool in sandbox with two agents, keeps both working during the move, and shows zero calls to the old version before removing it.

11. Sources