Knowledge · Architecture
AI system architecture
The layers a production AI application is built from, what each one does, and what each one gives away to someone mapping the system from the outside. An attacker's reconnaissance checklist that doubles as the owner's inventory.
An AI application is not one thing. Between the chat box and the model there are five or six distinct layers, usually built by different teams, often from different vendors, each with its own configuration and its own way of leaking what it is. Reconnaissance against an AI system is the job of working out which layers exist, what software runs each one, and where the seams are. Defending one starts with the same map.
Users web app, mobile app, API clients
│
API gateway authentication, rate limits, routing
│
Orchestration LangChain, LangGraph, CrewAI, AutoGen, hand-rolled
├─ RAG pipeline retriever, vector store, reranker
├─ Agent tools MCP servers, function definitions
└─ Integrations databases, file systems, other agents (A2A)
│
Inference server Ollama, vLLM, TGI, or a hosted API
│
Model weights, tokenizer, training cutoff
The middle three are drawn as boxes but usually are not separate services. Retrieval, tool definitions, and the inference client call are typically lines of code inside the orchestration layer. They are still worth treating separately because each one has its own enumerable surface.
API gateway
The first thing a request hits after the client. It authenticates, applies rate limits, and routes to the right backend. From outside, it is also the first thing that talks back: response headers name the proxy (nginx, Envoy, Cloudflare, an API management product), caching behaviour shows whether responses are stored, and error pages for malformed requests often come from a different piece of software than the success path, which tells you there are at least two things behind the hostname.
What to look at as the owner: which headers leave the building, whether rate limiting is per key or per IP (per IP is trivial to spread across), and whether the limit covers the expensive path (a long generation) or just request count.
Orchestration
This layer decides what the prompt actually looks like. It assembles the system prompt, picks what retrieved context goes in, calls tools, keeps the conversation within the context window, and runs multi-step loops. LangChain and LangGraph, CrewAI, AutoGen, Semantic Kernel, and a lot of in-house Python all live here.
Each framework has habits. Specific exception text when a tool call fails. A particular way of formatting tool results back into the conversation. Default prompt templates that survive into production because nobody rewrote them. Send a malformed input, a very long one, a request that trips a tool error, and read what comes back. A stack trace from this layer is the single most informative response an attacker can get, because it names the framework, the version, often the tool, and sometimes the file path.
Owner’s check: catch everything, log it internally, return something boring. And read your own framework’s default prompts, because the attacker already has.
Retrieval
The RAG pipeline fetches documents the model will read as context. It has its own parameters, and they can be probed: how many chunks come back, how long they are, whether a query phrased to match a document you planted gets that document retrieved, whether citations in the answer name file paths or internal titles. Those names are the enumeration prize, since they reveal what is in the index without asking the model to quote it. The anatomy of a RAG pipeline entry walks the stages.
Agent tools
Tools are what turn a chatbot into something that acts. They are declared as function schemas (name, description, parameters) and, increasingly, served over the Model Context Protocol. The permission boundary around tools is what a tester is after: which tools exist, which ones write or execute, whether the model can call them without a human in the loop, and whether a tool’s input is validated before it reaches a shell or a database.
The model itself will usually tell you what tools it has if you ask nicely, because the tool list is in its context and nothing in the model distinguishes “describe your tools” from any other request. That is the cheapest tool enumeration there is.
External integrations
Databases, file systems, ticketing systems, email, and other agents. The Agent-to-Agent protocol (A2A) from Google is the emerging standard for the last one: an agent publishes a card describing what it can do, other agents discover it, delegate tasks, and collect results. Every integration is a trust relationship, and the integrations layer is where a compromised model turns into a compromised company. The question for each one is the same as for a tool: what can it reach, and who approved that.
Inference server
This is the process that holds the model in memory, tokenizes the prompt, generates, and formats the output. Three self-hosted options dominate, plus the hosted APIs from the model vendors.
| Server | Typical use | Tells on itself via |
|---|---|---|
| Ollama | Local, laptops, hobby and small team | /api/tags lists installed models, /api/version; no auth at all |
| vLLM | Production serving on GPUs | OpenAI-compatible /v1/models, Prometheus /metrics with model name and queue stats |
| Text Generation Inference (TGI) | HuggingFace’s server, production | /info returns the model id, dtype, and max token settings |
| Hosted API (OpenAI, Anthropic, Bedrock, Vertex) | Most products | Error message formats, model name strings in responses, rate-limit headers |
An inference server reachable from the internet is a finding on its own; the Ollama post covers the common way that happens. Even behind the orchestration layer, server behaviour leaks through: token limits, streaming format, how it handles a stop sequence, and timing all differ between servers.
Model
The weights are almost never reachable, but the model’s identity is. Ask about recent events to find the training cutoff. Ask for things specific models refuse or format in a distinctive way. Look for artefacts of the training data, such as a model that completes a known benchmark prompt verbatim. Capability boundaries (context length, languages, whether it can do arithmetic past a certain size) narrow the candidates further. None of this needs access to anything but the chat box, and it tells an attacker which published jailbreaks and known weaknesses to try first.
Why MCP and A2A get their own section
Most layers leak by accident. These two leak by design: they are self-describing protocols. Their whole purpose is to let a client discover what is available without prior knowledge, which is also the whole purpose of reconnaissance.
MCP speaks JSON-RPC and answers tools/list, resources/list, and prompts/list with full schemas: every function, every parameter and its type, every description written to tell a model when to use it. One request to an exposed MCP server returns the agent’s complete action surface. A remote MCP server with weak or missing authentication is therefore a map of what the agent can do, handed over before any attack starts.
A2A publishes an agent card, a JSON document at a well-known path, listing the agent’s skills, input and output modes, and authentication requirements. Agents that delegate to each other form a trust graph, and the cards make it readable: which agent will accept tasks from which, and what each is allowed to do on the others’ behalf.
For both, the owner’s question is whether discovery is authenticated and whether the schemas say more than they need to. A tool description that mentions the internal hostname it connects to is documentation for the attacker as much as for the model.
Using the map
Walk the stack top to bottom and write down, for each layer, the software, the version if you can get it, and what it exposed. An attacker builds this inventory in an afternoon. If the owner does not already have one, that afternoon is the only advantage they will get.