Skip to content

Middleware

Middleware is DeepAgents' primary extension point. Every middleware observes and can rewrite the full agent lifecycle - messages, system prompts, tool lists, and individual tool calls. The same mechanism powers DeepAgents' built-in structural pillars (planning, filesystem, subagents, summarization, human-in-the-loop) and the swappable capability toolsets (web, git, shell, macOS, ...).

AgentMiddleware protocol

public protocol AgentMiddleware: Sendable {
    var name: String { get }
    var tools: [any AgentTool] { get }

    func beforeAgent(_ state: inout AgentState) async
    func beforeModel(_ state: inout AgentState) async
    func afterModel(_ state: inout AgentState) async
    func afterAgent(_ state: inout AgentState) async

    func wrapModelCall(
        _ request: ModelRequest,
        _ handler: (ModelRequest) async throws -> ModelResponse
    ) async throws -> ModelResponse

    func wrapToolCall(
        _ request: ToolCallRequest,
        _ handler: (ToolCallRequest) async throws -> AgentMessage
    ) async throws -> AgentMessage
}

Hook execution order

run(...) called
│
├── beforeAgent  (all middleware, once per run, in registration order)
│
│   ┌── [Round N] ───────────────────────────────────────────────────┐
│   │  beforeModel  (all middleware, every round, registration order) │
│   │                                                                  │
│   │  wrapModelCall  ← NESTED, first-registered is OUTERMOST        │
│   │    └── session.nextTurn(...)                                     │
│   │  /wrapModelCall                                                  │
│   │                                                                  │
│   │  afterModel   (all middleware, every round, registration order) │
│   │                                                                  │
│   │  For each tool call:                                             │
│   │    wrapToolCall  ← NESTED, first-registered is OUTERMOST        │
│   │      └── tool.execute(...)                                       │
│   │    /wrapToolCall                                                 │
│   └── [End round] ────────────────────────────────────────────────┘
│
└── afterAgent   (all middleware, once per run, reverse order)

Nesting vs. sequencing

wrapModelCall and wrapToolCall are nested decorators - each middleware wraps the next, exactly like HTTP middleware stacks. The middleware registered first is the outermost layer: it can inspect and modify the request before any inner middleware sees it, and it sees the final response after all inner middleware have processed it. By contrast, beforeModel/afterModel are sequential callbacks that each mutate shared AgentState.

What each hook can do

Hook When What you can do
beforeAgent Once at run start Inject system prompt additions, initialise state, log run start
beforeModel Before each model call Rewrite messages, edit systemPrompt, add/remove tools
afterModel After each model call Inspect the just-produced assistant message, update state
afterAgent Once at run end Flush logs, release resources, report metrics
wrapModelCall Around the model call Intercept/retry the model call; rewrite request (messages, tools, prompt)
wrapToolCall Around each tool call Intercept/approve/deny/retry a tool call; rewrite arguments or result

AgentState holds the live conversation thread, system prompt, and tool list for the current round. Mutations in beforeModel are visible to the model call that follows.

Contributing tools

The tools property lets middleware own the tools it provides. Tools contributed this way are merged with the explicit tools: array at factory time, then filtered through disabledToolNames. Middleware should declare all the tools it manages via this property rather than injecting them through beforeModel - this ensures correct deduplication and policy enforcement.

If the middleware also appends prompt guidance describing those tools, gate it on contributesRenderedTools(to:) - with lazy tools on, a tool's schema can be withheld from the request while staying callable, and prose about a tool the model cannot see does active harm.

Structural pillars

Structural middleware implements the core agent architecture. createDeepAgent wires these in automatically; createAgent leaves them out unless you add them yourself.

TodoListMiddleware

Adds planning discipline to the agent. Appends writing guidance to the system prompt and contributes the write_todos tool, which lets the model maintain a structured task list that persists across rounds within a run.

FilesystemMiddleware

Provides file I/O through a pluggable FilesystemBackend. Two backends ship:

  • StateBackend - in-memory; good for tests and isolated runs
  • LocalFilesystemBackend - reads and writes real files on disk

Tools: ls, read_file, write_file, edit_file, mkdir. Pass a backend via createDeepAgent's backend: parameter; it defaults to StateBackend() when includeFilesystem: true.

With LocalFilesystemBackend, createDeepAgent also names the root in the system prompt ("Your working folder is …", plus a note that paths outside it are refused). Without it a model has nothing to go on and invents a path: across 47 on-device runs, 30 produced outside the allowed folder errors - 148 refused calls - including a model on macOS reaching for /home/user, and one reaching for a directory named after the project rather than the checkout it was running in. StateBackend names nothing, having no path a tool call could get wrong.

SubAgentMiddleware

Enables task delegation. Contributes the task tool; when the model calls task, the middleware routes execution to one of the registered SubAgent instances (or to a general-purpose sub-agent when includeGeneralPurpose: true). See Subagents.

SummarizationMiddleware

Hooks beforeModel and compacts the conversation when the context window reaches 80% of capacity. Summarised segments are replaced with a single synthetic message whose source field is set to identify it as compaction-synthesised. Configured via SummarizationConfig (pass nil to disable). See Summarization.

HumanInTheLoopMiddleware

Hooks wrapToolCall and calls your ToolApprovalHandler before every tool execution. The handler can approve, deny, or ask for user confirmation. Required for tools with side effects (file writes, shell commands, macOS automation). See Human in the loop.

AskUserMiddleware

Contributes the ask_user tool. When the model calls ask_user, execution suspends until your AskUserHandler returns a string. This lets the model request clarification mid-run without ending the run.

Capability catalog

Capability middleware provides toolsets that map cleanly to a single concern. All of these live in MiddlewareCatalog.all and can be enabled by ID. createDeepAgent with includeGeneralPurpose: true adds web, search, text, git, and shell automatically.

Middleware ID Type Tools contributed
web WebToolsMiddleware fetch, curl
search SearchToolsMiddleware grep, glob, tree
text TextToolsMiddleware head, tail, diff
git GitToolsMiddleware git_status, git_diff, git_log, git_show, git_blame
shell ShellToolsMiddleware shell (gated by ShellGuard)
macos MacToolsMiddleware mdfind, open, open_app, download, say, notify
filesystem FilesystemMiddleware ls, read_file, write_file, edit_file, mkdir
clipboard ClipboardMiddleware read_clipboard, write_clipboard
screenshot ScreenshotMiddleware take_screenshot, take_window_screenshots
apple_notes AppleNotesMiddleware list_notes, read_note, create_note, update_note
container ContainerShellMiddleware container_shell (sandbox mode)

macOS adapter required

screenshot, clipboard, apple_notes, macos, and container are provided by the DeepAgentsMacTools product and require macOS entitlements. Import DeepAgentsMacTools separately.

ToolSearchMiddleware - lazy tool loading

Every tool's JSON schema normally goes into the prompt on every round. ToolSearchMiddleware splits the tool set into two tiers so that most of it does not:

  • Core tools are rendered as usual.
  • Auxiliary tools are stripped from ModelRequest.tools in wrapModelCall, so they never reach the prompt - but they stay in ReactAgent.tools, and ReactAgent dispatches against that. They remain fully callable: invisible, but executable.

The agent finds them with the two meta-tools the middleware contributes:

Tool Purpose
search_tools Rank auxiliary tools against a description of what is needed; returns names and signatures as its tool result.
run_tool Call an auxiliary tool by name, for a model that will not emit a name absent from its own schema.

Enable it through createDeepAgent:

let agent = createDeepAgent(
    model: model,
    middleware: [GitToolsMiddleware(root: root), WebToolsMiddleware()],
    auxiliaryToolNames: policy.expand().auxiliaryToolNames,
    toolRetriever: ColBERTToolRetriever(),   // or omit for LexicalToolRetriever
    toolSearchLimit: 5
)

Why the schemas arrive as a tool result

MlxChatModel caches the KV of the prompt's stable prefix and fingerprints it on systemPrompt + toolNames, so growing the tool list mid-run would reset that cache and force a re-prefill - the obvious implementation of lazy tools is the one thing that cannot be done cheaply. So the rendered set never changes: it is a constant filter, and everything discovery produces arrives as conversation content, which appends past the cached tip. Editing a tier costs one re-prefill; discovering a tool costs none.

ReactAgent.renderedTools (via the ToolRenderFiltering protocol) is what the prompt-overhead estimate measures, so summarization is not triggered early by schemas the model never sees.

run_tool is rewritten into a direct call inside wrapModelCall, before ReactAgent normalizes, records, or dispatches it - which is what keeps the approval gate, the message log, and the transcript pointed at the real tool.

The cost of that choice: a catalogue in the data slot

Delivering schemas as a tool result has one failure mode worth knowing about, because it is created by the mechanism rather than inherited from the model.

A tool result is where every real result the model has ever seen arrived. A search_tools result is not data - it is a description of what data would look like, and descriptions are written in the future tense ("Returns the note's title and its plain-text body"). A small planner collapses that into the past tense. Observed on a 1.2B: handed read_note(title!: string, …) by a search, it called nothing and answered with a note title, a note id (A1B2C3D4) and a full note body - every character invented, all of it presented as the user's real data. Unlike a wrong-tool call, this produces no error and no failed call, so it reads as success.

Two countermeasures ship, and both are about making the failure loud rather than preventing it outright:

  • The result names its own genre: it opens "these are tool definitions you may now call, not results. Nothing has been read or run yet", and ends by saying that describing what a tool would return is not an answer.
  • ToolSearchMiddleware.wrapModelCall appends one follow-through turn whenever the conversation ends on a search result, naming the tools that were offered and repeating that nothing has run. It is transient - it goes into that one ModelRequest, never into state.messages, so it cannot reach the stored thread - and it sits at the end of the conversation because that is immediately before the generation point. It leaves an honest way out ("if none of them fits, say plainly that you could not do it"), so a search that genuinely found nothing never turns into a forced call.

Neither makes a weak planner competent. If you host this, prefer a mid-size planner or better, and treat a confident answer with no preceding tool call as suspect.

The structural alternative - promoting a discovered tool into the rendered set, where a chat template marks schemas with its own tokens and the model cannot mistake them for content - costs one re-prefill per session rather than none. It is recorded as the long-term option in STEPS/TOOL-SEARCH/DESIGN.md and deliberately not taken yet.

Withholding a schema withholds the prose

If you write middleware that contributes both tools and prompt guidance, gate the guidance. Stripping ModelRequest.tools does nothing to the prose a middleware appends in its own wrapModelCall, and a section that names a tool whose schema was withheld is worse than no section: the agent is told to call something it has no way to call, and reaches for an unrelated tool instead.

AgentMiddleware provides the check:

public func wrapModelCall(
    _ request: ModelRequest, _ handler: ...
) async throws -> ModelResponse {
    // Say nothing about tools that did not reach this request.
    guard contributesRenderedTools(to: request) else { return try await handler(request) }
    let composed = [request.systemPrompt, Self.systemPrompt]
        .compactMap { $0 }
        .joined(separator: "\n\n")
    return try await handler(request.override(systemPrompt: composed))
}

contributesRenderedTools(to:) is true when at least one of the middleware's own tools appears in request.tools. Middleware that contributes no tools keeps its guidance unconditionally, so the default is safe. All the built-in capability middleware use it.

Two things this rules out, both of which were live bugs:

  • Naming a tool in prose that the tier withheld. With apple_notes auxiliary, the prompt still read "you can read and write the user's Apple Notes - never claim you can't" while no such schema was rendered.
  • Naming an auxiliary tool in guidance that is otherwise tier-agnostic. Prose meant to steer the model away from something should describe the activity, not the tool - a bullet naming ls/read_file puts those names back in the prompt in exactly the configurations where the filesystem is itself auxiliary.

Retrievers

ToolRetriever ranks the auxiliary corpus. Two ship:

Type Product Notes
LexicalToolRetriever DeepAgents IDF-weighted term overlap. No model, no download - the default.
ColBERTToolRetriever DeepAgentsMLX LFM2.5-ColBERT-350M late interaction (MaxSim over per-token 128-d vectors), 8-bit or bf16 via ToolSearchModel.

createDeepAgent composition order

When you call createDeepAgent, middleware is assembled in this order before being handed to ReactAgent:

  1. ToolSearchMiddleware (when auxiliaryToolNames is non-empty)
  2. SummarizationMiddleware (when summarization != nil)
  3. TodoListMiddleware
  4. FilesystemMiddleware (when includeFilesystem: true)
  5. SubAgentMiddleware
  6. Your middleware array (in the order you provide)
  7. AskUserMiddleware (when askUserHandler != nil)
  8. HumanInTheLoopMiddleware (when approvalHandler != nil)

Both wrapModelCall and wrapToolCall nest with the first-registered middleware outermost, which is what fixes the two ends of this list:

  • ToolSearchMiddleware is registered first on purpose, so its wrapModelCall is the outermost one and the auxiliary schemas are already stripped before anything that might describe them runs. Every capability middleware decides whether to append its prompt guidance by inspecting request.tools (see contributesRenderedTools), and that check is only meaningful once the stripping has happened.
  • HumanInTheLoopMiddleware is registered last, so for wrapToolCall it is the innermost wrapper - the last layer before the tool actually executes. That is the stronger position for a gate: no other middleware can execute the call without passing through it.

SummarizationMiddleware hooks beforeModel rather than nesting, and is registered ahead of the other hooks so it compacts the history before they read it.

Disabling middleware and tools

Via disabledToolNames

Both factories accept disabledToolNames: Set<String>. Any tool whose name appears in this set is removed from the merged tool list at factory time. The model never sees the tool in its schema - it cannot call something that was never offered.

Via AgentToolPolicy

public struct AgentToolPolicy: Codable, Sendable {
    public var disabledMiddleware: Set<String>  // middleware IDs (catalog names)
    public var disabledTools: Set<String>       // individual tool names
    public var approvals: [String: ToolApprovalMode]
    public var sandbox: SandboxMode
    public var sandboxImage: String?
    // Lazy tools - see `ToolSearchMiddleware`. Inert while `toolSearch` is false.
    public var toolSearch: Bool
    public var auxiliaryMiddleware: Set<String>  // middleware IDs whose tools are auxiliary
    public var auxiliaryTools: Set<String>       // individual auxiliary tool names
    public var coreMCPServers: Set<String>       // servers promoted to core (MCP defaults to auxiliary)
    public var toolSearchModel: String?          // retriever repo id; nil = lexical
    public var toolSearchLimit: Int

    public func expand(
        catalog: [MiddlewareDescriptor] = MiddlewareCatalog.all,
        extraDefaults: [String: ToolApprovalMode] = [:],
        extraAuxiliary: Set<String> = []
    ) -> Expansion
}

AgentToolPolicy is a serialisable value (Codable) useful for per-user or per-session configuration - for example, persisting the user's approval preferences between sessions in Ripple. Call expand(...) to resolve the policy against the live catalog and get back a concrete set of tools to disable and approval modes to apply.

Disabling is at factory time

Tools removed via disabledMiddleware or disabledTools are never rendered into the model's prompt. This is architecturally different from the approval gate, which fires at dispatch time. Disabled tools cost zero tokens and cannot be called even accidentally; approval-gated tools appear in the prompt but require explicit authorisation before execution.