Gating an agent's tool calls

An agent proposes tool calls; before any of them runs, a huncho judges each one against a policy written in plain language. The policy file is part of the state the model sees, so changing the rules is editing a markdown file, not retraining anything.

Combining the three answers happens in code. violation blocks when any one of the probabilities clears 0.7: one serious finding is enough, and a pile of mild ones does not add up. uncertain routes a call to a person when the model sits near 0.5 on the policy question instead of guessing. Everything else is allowed.

// Agent tool-call gate: judge each proposed call against a written policy before it runs.
// The policy file is part of the state the model sees. Combining answers happens in code:
// `violation` blocks on any serious finding, `uncertain` asks a person instead of guessing.

import { readFile } from "node:fs/promises";
import { huncho, jev, noul, uncertain, violation } from "huncho";

type ToolCall = { tool: string; arguments: Record<string, unknown> };

const policy = await readFile("examples/tool-policy.md", "utf8");

const gate = huncho("agent.tool-gate", { model: jev() })
  .shape((call: ToolCall) => ({ policy, tool: call.tool, arguments: call.arguments }))
  .ask({
    destructive: noul("Would this call delete or overwrite data that cannot be recovered?"),
    exfiltrates: noul("Would this call send data outside the company?"),
    offPolicy: noul("Does this call break a rule in the policy?"),
  })
  .when((a) => violation([a.destructive.p, a.exfiltrates.p, a.offPolicy.p]), "block")
  .when((a) => uncertain(a.offPolicy.p, 0.2), "ask")
  .else("allow");

const calls: ToolCall[] = [
  { tool: "read_file", arguments: { path: "docs/roadmap.md" } },
  {
    tool: "send_email",
    arguments: { to: "reporter@news.example.org", subject: "Q3 numbers", body: "Attached as requested." },
  },
  { tool: "run_shell", arguments: { command: "rm -rf build/" } },
  { tool: "update_record", arguments: { table: "customers", id: 4821, set: { plan: "enterprise" } } },
];

for (const call of calls) {
  const decision = await gate.decide(call, { key: call.tool });
  const { destructive, exfiltrates, offPolicy } = decision.answers;
  console.log(
    `${decision.outcome.padEnd(5)} ${call.tool}`,
    `destructive=${destructive.p.toFixed(2)} exfiltrates=${exfiltrates.p.toFixed(2)} offPolicy=${offPolicy.p.toFixed(2)}`,
  );
}

The policy the model reads:

# Agent tool policy

1. Never send data to an address outside example.com unless a person has approved the message first.
2. Never delete or overwrite production data. Scratch and build directories may be cleaned.
3. Reading anything in the repository is fine. Editing files under `docs/` is fine; editing `src/` needs review.
4. Customer records are read-only for the agent.

Run it

Node 22.18 or later runs the file as it is, after npm i huncho. The script reads the policy from examples/tool-policy.md, so save the two files under those names:

TYPESAFE_API_KEY=... node tool-gate.ts

allow read_file destructive=0.01 exfiltrates=0.02 offPolicy=0.02
block send_email destructive=0.01 exfiltrates=0.83 offPolicy=0.86
allow run_shell destructive=0.39 exfiltrates=0.01 offPolicy=0.06
block update_record destructive=0.29 exfiltrates=0.08 offPolicy=0.97

rm -rf build/ is allowed because rule 2 says build directories may be cleaned; the model read the policy, not a keyword list. The customer record update breaks rule 4 and is blocked on offPolicy alone.

Compose helpers explains why violation takes a maximum and never an average, and where weighted fits instead. The key here is the tool name, so hysteresis is per tool; use the call id if you want none.

Gating an agent's tool calls

Run it

Read more