Test and simulate

K-Block gives you four ways to see what a policy would decide before you activate it. None of them changes how the Gateway decides live calls. They differ in what they evaluate (one call, a list of examples, or your recorded traffic) and in how much of the Gateway's decision they reproduce.

Test or simulate?

Use Test while you write the document. It answers "does this document decide the calls I care about the way I intend?". It works on calls you describe, runs against the unsaved editor text, and gives an answer in seconds. Saved examples are also your regression suite: every activation checks them.

Use Simulate before you activate a version. It answers "what would change for the calls my agents actually made?". It replays recorded traffic, so it finds the cases you did not think of, and it counts how many calls become less or more restricted. The activation sheet runs a 7-day simulation automatically.

Use a dry run when you need the full answer for one specific call: not only the policy's effect, but also the agent's delegation, the tool's binding, observe windows, approvers, and exactly what the agent would receive.

None of them proves what the Gateway will do on a future call. The active version, delegations, tools and members can change between your check and the call; the decision is always made by the Gateway at call time.

The Test tab

The Test tab of a policy has two parts: Example calls and Try a call. Both evaluate the current text of the Editor tab, even if you have not saved it, so you can test while you edit.

Saved examples

An example is a call described by its evaluation inputs (an action, an exposed tool name and an optional amount and currency) with the outcome you expect: an effect and the deciding rule, or the default. Examples never contain tool arguments.

1. Add examples

   Select Add an example and fill in ID, Label (optional), Action, Tool, Amount (optional) and Currency, then choose the Expected outcome (Allowed, Requires approval, Denied or No expectation) and the Expected rule. Leave the rule empty when you expect the default to decide. Select Keep this example to add it to the list.

2. Watch them run

   The examples run against the editor text about two seconds after each change, and when you select Run against the editor text. Each row shows ✓ passes, ✗ fails (with the effect and rule it got), an error code if the call cannot be evaluated, or no expectation. An example passes only when both the effect and the deciding rule match.

3. Save them

   The list is only stored when you select Save examples (Discard changes reverts it). Examples belong to the policy, not to one version: they apply to every version you activate later.

Owners, admins and developers can edit and save examples (and management keys with policies:write, through PUT /orgs/:orgId/policies/:id/examples). A policy holds at most 200 examples. Saving records a policy.examples_updated audit event. A policy created from a template starts with the template's examples.

Why they matter at activation: when an owner or admin activates a version, the server evaluates every saved example that has an expectation. If one fails, the activation is refused with 409 EXAMPLES_FAILING unless the admin ticks Activate anyway (recorded with the activation). See Activate, roll out and roll back.

Simulations are rate limited per organization (see Simulate against recorded history). If automatic runs hit the limit, the tab pauses them and tells you; select Run against the editor text when you are ready.

Try a call

Try a call evaluates one call against the editor text, and against the active version for comparison. Choose how to describe the call:

• Action, tool and amount: the evaluation inputs directly.
• A tool call with arguments: an exposed tool name and the call's Arguments (JSON). K-Block looks up the tool's action and reads the amount with the tool's amount mapping exactly as the Gateway does, so you also see INVALID_AMOUNT and similar refusals. The arguments are sent only in the request and are never stored or echoed.

Select Evaluate. The result shows the effect and deciding rule under the editor text, the full explanation, and how it compares with the active version. Nothing is stored or signed.

What the Test tab proves, and what it does not

It shows how the policy document decides the inputs you gave. It does not check the agent's delegation, the tool's binding or whether the tool is enabled, observe windows, or whether anyone can approve (NO_APPROVERS). For those, use a dry run.

Dry run

A dry run answers "what would K-Block do with this exact call, right now?". It runs the same decision logic the Gateway uses, on your organization's current configuration, writes nothing, and stops before anything is signed or executed.

Open it from an agent's page with Test a call, or from a recorded action with Re-run as dry-run with today's policy. On the Test a call page, choose the Agent, the Tool and the Arguments, pick the Environment of the agent's key (Live or Test), and select Run dry run. Under More options you can set:

• Idempotency key (optional): the key the agent would send; an earlier call with that key returns its stored outcome.
• Policy version: a hypothetical answer under another version of the agent's policy, decided under enforce (without observe relaxation).
• Decide as if no observe window were open.

The answer covers the agent, the tool and its binding, the argument checks, the amount, the policy decision and the enforcement level (including an open observe window), the delegation and its constraints, the approvers, and plan limits. It shows what the Gateway would answer, the explanation, exactly what the agent would receive (see What agents are told), and what a shadow version would decide.

A dry run does not check:

• gateway API key authentication and per-key rate limits (no key is used);
• the verification of the signed task itself, which K-Block makes when the Gateway decides a real call (nothing is signed, so the delegation checks this verification makes are reproduced instead);
• whether the tool's upstream address is allowed, whether its stored credential can be used, and whether the upstream is reachable;
• the live checks made immediately before execution, including the re-check against a newer policy version.

Every member can run dry runs, and so can management keys with the read scope (POST /orgs/:orgId/agents/:agentId/dry-run). Without an environment the API assumes live. Dry runs are limited to 60 per minute per credential (your session or the API key) and 600 per minute per organization; over the limit the request is refused with 429 RATE_LIMITED, and the Retry-After header says how long to wait. K-Block can sometimes accept more requests than these limits before it refuses one. A dry run writes nothing.

Simulate against recorded history

The Simulate tab evaluates a candidate version against the recorded calls of your agents and compares it with a baseline.

You can also Include the saved examples and Include the decision map (how the candidate decides every amount range of every enabled tool). Select Run the simulation.

The result counts the recorded calls and how many were evaluated, then how many become Less restricted, More restricted, decided by an Other rule, sent to Other approvers, or stay the Same. It also shows a matrix of baseline outcome against candidate outcome and lists the Calls decided differently, with links to the recorded actions.

What a simulation replays

K-Block records the evaluation inputs of every decision: the action, the exposed tool name, and the amount and currency. A simulation evaluates the candidate on those recorded inputs. It never uses tool arguments, which are not stored for this purpose.

It replays the policy effect only. It does not apply observe windows, delegations, approver resolution or tool bindings. Calls that were denied before the policy was evaluated, because their amount was invalid (INVALID_AMOUNT) or the tool's amount mapping was invalid (INVALID_TOOL_CONFIGURATION), have no policy input and are counted as Not evaluable.

Limits:

Every member can run simulations (POST /orgs/:orgId/policies/:id/simulate, read keys included). Running one writes nothing.

Save a simulation as evidence

When the candidate is a saved version, owners, admins and developers can select Save as evidence. K-Block stores the summary for 400 days and records a policy.simulation_saved audit event. If you then activate that version from the same page, the activation cites the saved simulation, and the server refuses the activation (400 SIMULATION_MISMATCH) if the simulation belongs to another version. A policy can save at most 100 simulations per 24 hours (409 SIMULATION_LIMIT). To save a simulation of the editor text, save the text as a version first.

Next steps