Observe and shadow

Observe and shadow both help you learn what a policy does to real traffic before you enforce it fully. They are very different in risk:

• An observe window is a time-boxed exception for one agent. While it is in effect, calls that the policy would deny or hold by its default run anyway and are recorded as relaxed. It changes live decisions.
• A shadow version is a non-active version that K-Block evaluates on recorded calls, next to the active version. It never changes a live decision: the Gateway does not even read it.

Observe or shadow?

Prefer a shadow version whenever it answers your question: it shows what a version would change without letting anything run that the active version refuses. Use an observe window only during onboarding, when the policy is still incomplete and blocking the agent would stop you from seeing what it really calls. Observe is not recommended for payments: the Finance operations template recommends shadow, then enforce.

Observe windows

What a window relaxes

Each window is started with one of two relax modes:

A window never relaxes:

• an explicit deny rule, including an amount rule that matched fail-closed;
• a require_approval rule marked guard: true (see Descriptions and guards);
• any check outside the policy's effect: gateway key authentication, the agent's status, the delegation (its presence, permissions and constraints such as the per-call amount limit), the tool binding pin, argument checks, replay protection and idempotency, plan limits;
• fail-closed refusals: INVALID_AMOUNT, INVALID_TOOL_CONFIGURATION, NO_POLICY, INVALID_POLICY, NO_APPROVERS, TOO_MANY_APPROVERS;
• an emergency stop: no window relaxes a policy whose active version is an emergency stop;
• approval requests created before the window started.

A relaxed call is allowed without an approval request. It is recorded with the enforcement level observe, the effect the policy would have applied, the rule and the window, and the signed task records the same policy fields. Each relaxed call sends an action.observe_override webhook, and the result returned to the agent says that the call was relaxed.

Live and test keys

A window covers the calls made with the Test gateway keys, the Live gateway keys, or both. Test keys are not a sandbox: they reach the same tools, upstreams and credentials as live keys. That is why every window, test-only ones included, needs the owner's live opt-in.

Owner opt-in

Observe is off by default. Only the owner can turn it on, on the organization's Settings page under Observe mode:

• Longest observe window (hours): 0 (off, the default) to 168.
• Allow observe windows (the live opt-in): required for every window.

A change that widens what can run also needs the box I understand, and relaxed calls will be reviewed before Save observe settings is enabled.

Admins see these settings read-only; through the API they receive 403 OWNER_REQUIRED. Saving a stricter setting ends windows at once: turning observe off or withdrawing the live opt-in ends every open window, and lowering the maximum ends every open window longer than the new maximum.

Start a window

1. Open the agent

   Open the agent's page. The Enforcement card shows Enforced, or the open window. Start observe… appears for owners and admins once the owner has enabled observe and given the live opt-in.

2. Choose the scope

   In the sheet, choose the Gateway keys the window covers, What the window relaxes and the Duration (whole hours, from 1 to the organization's maximum). Enter a Reason (recorded in the audit log) of 10 to 1000 characters: why observe, and who reviews the relaxed calls. The sheet lists the agent's delegation, which stays the hard ceiling, and the rules that stay enforced.

3. Confirm

   Type the agent's name and select Start observe for … h.

Starting a window requires a signed-in owner or admin; no API key can start one (POST /orgs/:orgId/agents/:agentId/observe). The agent must be active and use a policy with an active version that is not an emergency stop.

If K-Block cannot confirm that it has enabled observe windows on every Gateway, the window still starts, but the API response carries the warning LEGACY_GATEWAYS_ENFORCE (the dashboard does not show it). A Gateway without that support ignores the window and keeps enforcing the policy fully, so the window can relax fewer calls than you expect; it never relaxes more. If the warning keeps appearing, contact K-Block support and give them the warning code and the request ID of the response (its X-Request-Id header).

While a window is open

A banner on every page of the organization shows each open window and its countdown; it cannot be dismissed. The policy's Rollout tab shows the observe stage, and the policy summary counts the agents in observe mode.

A window is in effect for a call only while it is open, the owner's settings still allow it, the call's key environment is covered, and at least 5 seconds remain. A call the window relaxed that has not executed when the window ends is refused with OBSERVE_ENDED, and nothing executes; if a new version is activated before it runs, it is refused with POLICY_CHANGED. A relaxed call is never re-evaluated: the agent's retry gets a new decision.

Activating a new version of the policy does not end open windows: the new version is evaluated in observe mode too. The activation sheet lists these agents.

End a window

A window ends by itself at the end of its duration and cannot be extended. It also ends:

Starting and ending windows are recorded as agent.observe_started and agent.observe_ended audit events and sent as webhooks of the same names. A rollback does not end open windows.

Window report

Select Window report on the agent's page while the window is open, or Report next to a window under Earlier windows (API: GET /orgs/:orgId/observe-windows/:windowId/report). It counts the decisions made under the window, how many were relaxed (split by the effect the policy would have applied, by rule or default, and by tool), the sum of executed relaxed amounts per currency, and how many explicit denies and guards were enforced. Review every relaxed call before you finish the rollout.

Shadow versions

A shadow is a draft or superseded version that you want to compare with the active one. K-Block evaluates it when you read a report, on the recorded evaluation inputs of real calls (action, exposed tool name, amount and currency). It is never enforced and the Gateway never reads it, so it cannot change a live decision.

Start, replace or stop a shadow

On the policy's Shadow tab, pick a version under Shadow and select Start shadowing; while a shadow is set, pick one under Replace with and select Replace the shadow. You can also select Shadow on a version in the Versions tab, or Shadow v… on the Rollout tab. Stop the shadow removes it.

• A policy has at most one shadow, and it cannot be the active version (409 SHADOW_IS_ACTIVE).
• If someone changed the shadow at the same time, the change is refused with 409 SHADOW_CHANGED; reload and try again.
• Owners, admins and developers can change the shadow, and management keys with policies:write (PUT and DELETE /orgs/:orgId/policies/:id/shadow).
• Changes are recorded as policy.shadow_started and policy.shadow_stopped audit events.

Activating the shadow version clears the shadow, and so does a rollback that restores it. An emergency stop keeps it.

The shadow report

The Shadow report compares, for each recorded call since the shadow started, the policy effect recorded when the call was decided with what the shadow version decides. Choose the Window (Last 24 hours, Last 7 days or Last 31 days; the API accepts up to 31 days), which calls (agents that use this policy now or decisions made by this policy) and the Environment.

The report shows Recorded calls, Evaluated, Not evaluable, Same, Less restricted, More restricted and Other rule, what the shadow would decide, a recorded-to-shadow matrix and the Calls decided differently, linked to sample actions. The Rollout tab summarizes the last 24 hours.

The shadow's decision also appears on action and approval pages and in dry runs, marked as not enforced.

A shadow report compares policy effects only, like a simulation. To compare against a fixed window or another baseline, use the Simulate tab (Test and simulate).

Which of these affect live calls

Next steps