ContractCanary

Pass, fail, and unknown

The CLI compares a saved contract with a candidate and returns one of three results. It does not invoke tools.

Example: a baseline requires id as a string. The candidate requires id as a number. ContractCanary exits 1 and the CI job fails. Adding an optional field exits 0. A missing schema exits 2, which is also unsuccessful.

Supported checks are explicit types, properties, required, boolean additionalProperties, items, scalar enums, and numeric or length bounds. Refs, unions, regex, formats, and composition are unsupported.

Adjacent tools cover a different job. Promptfoo evaluates prompts. MCPJam Inspector talks to a live MCP server. Braintrust stores evals. ContractCanary only fails the process when a saved schema breaks.

CLI guide · Pricing · Product