An AI agent is an application in which a model can decide what to do next, call tools, read their results, and continue toward a task. The model is one component. The surrounding application supplies instructions, connects data and tools, manages state, enforces permissions, and decides when the work is finished or must pause. A useful architecture diagram therefore includes both the model loop and the controls around it.
The example below uses a fictional law-firm intake packet. It illustrates a design, not a measured capability of any vendor product. The technical patterns apply outside law, but confidential matters, professional duties, and external actions make legal boundaries especially consequential.
Six parts of a working agent
Swipe sideways to see all columns →
| Part | What it does | Legal-work design question |
|---|---|---|
| Model | Interprets the task and generates a response or proposed next tool call. | What can the model decide, and what must a person decide? |
| Orchestrator | Runs the loop, passes tool results back, applies stop conditions, and assembles the final output. | Who owns a run, time limit, retry policy, and final approval? |
| Tools | Search, read, draft, or change something through a defined interface. | Which exact repository, matter, fields, and actions are allowed? |
| Retrieval | Finds relevant passages from approved sources at run time. | Which version was read, and can every material claim be checked against it? |
| State | Keeps run progress or persistent notes outside the model's current context. | What persists, for whom, for how long, and who can erase it? |
| Controls | Authenticate users, check scopes, validate tool arguments, require approval, and record actions. | Can a forbidden call be stopped before it changes anything? |
Anthropic's architecture guide describes the augmented model—retrieval, tools, and memory—as a building block, with workflows and more autonomous agents as different control patterns. This is a useful conceptual model, not a guarantee that adding each part improves a system. Complexity should earn its place through representative evaluations.
The model proposes a tool call; the application executes it
Suppose a lawyer asks for an internal issue list from an approved lease packet. The model may propose a search for “notice deadline,” receive two passages, ask to open the signed amendment, and draft a source-linked note. In ordinary function calling, the model produces a tool name and arguments; the application checks and executes the call, then returns the result to the model. The tool boundary can refuse a wrong-matter ID or an out-of-scope action. OpenAI's function-calling guide describes this model/application exchange. Exact implementation details vary by platform.
A tool is not simply a menu label. Its inputs, return shape, error behavior, authentication, and side effects determine what the agent can actually do. A narrow `search_approved_matter` tool is easier to govern than unrestricted file access. A `create_internal_draft` tool should not share a credential that can email or file documents. Anthropic's tool-design guidance recommends clear, distinct tools and bounded returns because overlapping tools and bulky results can impair decisions and consume limited context.
Retrieval provides current evidence; it does not retrain the model
Retrieval searches an external collection, selects passages, and puts them into the material the model can use for this run. It can make a response more grounded in the approved packet, but a retrieved passage may be stale, incomplete, from the wrong matter, or malicious. A citation is only useful if a reviewer can open the referenced source and confirm the claim. Retrieval is neither a substitute for legal research nor proof of factual accuracy.
The context is what the model receives for its current step: task, instructions, selected conversation history, tool descriptions, retrieved passages, and tool results. It is finite and can be crowded by irrelevant material. A saved transcript, case note, vector index, or database record is stored state outside that context. The application may bring part of it into a later step, subject to permissions. Training changes model parameters through a separate development process; a model reading today's lease does not thereby train itself on the lease. Some products may use submitted data under specific settings and terms, so check the actual service contract rather than inferring a universal policy.
Anthropic's context-engineering discussion explains retrieval at the time of need, finite context, compaction, and structured notes. Compaction can carry useful facts into a new context window, but it can also omit qualifications or preserve an earlier mistake. Stored notes should therefore include provenance, scope, and an expiration or correction path, especially when matters and users must remain separate.
A fixed workflow, a dynamic agent, or multiple agents?
Swipe sideways to see all columns →
| Pattern | Best fit | Tradeoff to test |
|---|---|---|
| Fixed workflow | Known sequence: intake, extract, human review, final record. | Predictable control flow; can be brittle when documents vary. |
| Single agent | A bounded task where the next search or comparison depends on what was found. | Flexible tool choice; more variable paths, cost, and failure points. |
| Multiple specialists | Distinct subtasks need different instructions, tools, permissions, or parallel work. | More handoffs, shared-state risks, review surfaces, and trace complexity. |
A multi-agent system may delegate research, document comparison, and formatting, with one component synthesizing the result. It does not create independent legal judgment or automatically improve accuracy. A handoff may transfer control, while a manager pattern keeps a primary agent responsible and calls specialists as bounded tools. OpenAI's orchestration guidance recommends adding specialists when their contracts actually differ. For a first legal pilot, one agent with a few narrow tools often makes attribution, permission checks, and debugging easier.
Controls must live outside the prose instructions
An instruction such as “do not email the client” is useful, but it cannot revoke an email credential. Apply identity and matter checks to every data read; restrict tool scopes at the service boundary; validate arguments and destinations before execution; require a human to approve consequential changes; and fail closed when an approval cannot be obtained. The person should see the exact proposed edit or message, not a broad request to approve the agent's goal. OpenAI's guardrail and human-review guide distinguishes model-level checks from tool-level checks and warns that input/output guardrails alone do not cover every tool call.
For a reviewable run, record task and permissions, source and application versions, searches and tool calls, passages used for material claims, denied or approved actions, and the final reviewer decision. Minimize copied confidential text and credentials in the trace. The permissions and oversight guide gives a practical ladder for reads, drafts, changes, and external actions.
A small architecture test before adding complexity
On a fictional packet, ask whether the system can (1) retrieve the correct signed version, (2) flag a conflicting date and missing exhibit, (3) cite exact passages without inventing an answer, (4) refuse a wrong-matter read, (5) stop an attempted external send, and (6) produce a trace a lawyer can actually review. Repeat cases and compare a fixed workflow with a single-agent variant on quality, reviewer effort, latency, and cost. Add another agent or a longer-lived memory only when those tests show a specific benefit that outweighs added risk. The legal workflow walkthrough shows the packet; the pilot scorecard shows how to record results.
The architecture makes certain behaviors possible, not reliable by itself. Actual performance depends on documents, permissions, prompts, models, tools, implementation, and ongoing monitoring. No diagram can establish that a given system is safe for a real matter; the next step is testing realistic failure cases, described in agent failure modes and response.
