Policy outcomes and actions
What Passed and Failed mean for a policy and a gate, and what Block, Warn and Notify do
Policy outcomes and actions
Every time Safeguard evaluates a policy on a gate, it records an outcome. The gate's own outcome then follows from the outcomes of its policies and each policy's Action.
Policy outcomes are turned on per workspace by Safeguard. If your workspace does not show an Outcome column on the Gates and Policies tables yet, ask your Safeguard contact to turn it on. While it is off, policies and gates behave exactly as before.
When a policy is evaluated
A policy runs its starred search each time Safeguard evaluates it on a gate: when the policy is linked to the gate, and again after new assessment data arrives for the product.
A policy's outcome
| Outcome | Meaning |
|---|---|
| Failed | The policy's starred search found matching issues. |
| Passed | The search ran to the end and found no matching issues. |
| Error | The policy could not be evaluated, for example because its starred search is missing or the search failed or timed out. The reason is shown with it. |
| Not evaluated | The policy has not been evaluated on this gate yet. |
An Error is never counted as a pass.
Policies with a threshold
New policies have no threshold: any match fails the policy. A policy created earlier with a threshold above 0 keeps it. That policy fails only when the number of matches reaches the threshold, and passes below it.
What the Action does
Each policy has one Action. It decides what a failed policy does to the gate.
| Action | When the policy fails |
|---|---|
| Block | The gate fails. |
| Warn | The gate still passes, with a warning that names the policy. |
| Notify | The gate still passes, and Safeguard sends a notification to your workspace's configured channels: the in-app bell, your connected chat integrations, and email according to each person's preferences. |
Notify does not repeat the same failure on every evaluation. You are notified when a policy goes from passing (or never evaluated) to failing, and again when a failing policy finds new matching issues. A policy that keeps failing with the same matches is not notified again.
A gate's outcome
The gate's outcome is worked out from its linked policies, in this order:
- No policies linked: Not evaluated.
- Any Block policy failed: Failed.
- Otherwise, any policy could not be evaluated: Error, with the reason.
- Otherwise, any policy not evaluated yet: Not evaluated.
- Otherwise, any Warn policy failed: Warning. The gate passes, and the warning names the policy.
- Otherwise: Passed. Failed Notify policies are listed as notified.
A gate that has never been evaluated shows Not evaluated, never Passed.
Where you see outcomes
- Policies → Gates and Policies → Policies: the Outcome column shows Passed, Failed, Warning, Not evaluated or Error, with when it was last evaluated. A policy linked to several gates shows Failed if it failed on any of them.
- Expand a gate to see the outcome of each linked policy (its Action, outcome, effect on the gate, matches and when it was evaluated) and the gate's evaluation history, page by page.
Failing a CI build on a gate
safeguard ci scan --wait --fail-on-gate <gate name or id | all> waits for the scan to finish and for
the gate's policies to be evaluated on the new data, then:
- exits 0 when the gate is Passed or Warning (warnings and notified policies are printed);
- exits non-zero when the gate is Failed, and prints the Block policies that failed;
- exits non-zero when the gate is Error or Not evaluated, or its policies were not evaluated
before
--timeout, with a message that says why.
--fail-on <severity> keeps working as before and can be combined with --fail-on-gate. See
CI/CD Integration for setting up the secret key.
From an AI assistant (MCP)
Safeguard's MCP server has read-only tools for gate outcomes, policy outcomes and the evaluation history. They are available when policy outcomes are on for your workspace and your MCP access settings allow them.