> For the complete documentation index, see [llms.txt](https://cerebral-systems.gitbook.io/cerebral-systems-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cerebral-systems.gitbook.io/cerebral-systems-docs/getting-started-1/agent-integration/llm-gateways.md).

# LLM Gateways

An LLM gateway and Warrant solve different problems. The gateway brokers model inference; Warrant brokers side effects. They are complementary and should sit on different paths.

```mermaid
flowchart LR
  U[User] --> R[Agent runtime]
  R -->|chat/completions or responses| G[LLM gateway]
  G --> M[Model provider]
  M -->|tool call intent| R
  R -->|structured proposal| W[Warrant]
  W --> P[Policy and approval]
  P --> A[Actuator]
  A --> S[External system]
  A -->|execution result| W
  W -->|verdict or result| R
```

Do not put Warrant in the token stream between the gateway and model. A model tool call is proposed intent, not proof that an external mutation happened. The action boundary is the application's tool dispatcher—the code that would otherwise turn tool arguments into a credentialed provider request.

### Integration pattern

When a gateway response contains a mutating tool call:

1. buffer and parse the complete tool arguments;
2. map the tool to a reviewed `target_system.operation` catalog entry;
3. derive a stable `proposal_id` from the workflow and logical tool call id;
4. submit `POST /v1/actions` with the gateway/model values preserved as claims or evidence;
5. return Warrant's verdict to the agent loop as the tool result;
6. let Warrant enqueue and execute the actuator after `allow` or authenticated approval;
7. poll the returned URL after `202` and reconcile `outcome_unknown` with an operator.

```ts
async function dispatchMutatingTool(call: ModelToolCall) {
  const proposal = mapToolCallToProposal(call);
  const response = await fetch(`${process.env.WARRANT_URL}/v1/actions`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.WARRANT_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(proposal),
  });

  const verdict = await response.json();
  if (!response.ok && response.status !== 202) throw new Error(verdict.detail);
  return verdict; // never call the provider from this dispatcher
}
```

Read-only retrieval can remain outside Warrant when that matches your threat model. Mutating tools should use an explicit registry rather than guessing from names or model-generated descriptions.

### Credential placement

| Component               | Credentials it may hold                       |
| ----------------------- | --------------------------------------------- |
| LLM gateway             | Model-provider credentials only               |
| Agent/tool dispatcher   | Tenant-scoped Warrant agent key               |
| Warrant approver client | Separate tenant approver key or owner session |
| Warrant actuator        | External-system provider credential           |

Do not give the inference gateway an actuator credential just because it already centralizes model keys. Combining those roles recreates a path around Warrant policy.

### Streaming and parallel tool calls

* Do not submit a proposal until streamed tool arguments form valid complete JSON.
* Give each logical tool call its own deterministic `proposal_id`.
* Parallel inference does not imply parallel mutation is safe.
* Preserve the same id when retrying after a gateway timeout.
* Never interpret a gateway retry as permission to execute a provider call twice.

### Fail-closed behavior

If catalog lookup, Warrant admission, approval status, or execution status is unavailable, the dispatcher must not call the provider directly. Return a structured failure or pending result to the agent and let it wait, ask for help, or stop.

### Validate the integration

* A normal text response never creates an action.
* A read-only tool follows the explicitly chosen read path.
* A mutating tool always creates a Warrant proposal before any provider request.
* A gateway timeout and retry reuse the same proposal id.
* `needs_approval`, `deny`, and `outcome_unknown` never fall back to direct execution.
* Provider credentials are absent from the gateway and agent runtime.

### What this does not secure

Warrant does not govern prompts, model routing, inference budgets, token logging, gateway tenancy, data retention, or model-provider availability. An LLM gateway does not by itself prove that tool calls passed through Warrant. Direct SDK calls, browser automation, shell commands, webhooks, or another tool dispatcher remain bypasses unless their credentials and egress are constrained.

This is a protocol-level integration pattern, not a packaged adapter for a particular gateway product.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://cerebral-systems.gitbook.io/cerebral-systems-docs/getting-started-1/agent-integration/llm-gateways.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.