Building a BAA-Ready AI Agent Stack: Architecture, Vendors, and the PHI Boundary

The fastest way to kill a healthcare AI project is to demo it before you can name every vendor that will touch patient data. A compliance officer does not care that your agent routes a triage request in under a second. They care about one question: who has signed a Business Associate Agreement, and can you prove that protected health information never reaches a system that hasn't? Most agent stacks fail that question not at the language model, which is the part everyone worries about, but three hops downstream in a logging pipeline nobody drew on the whiteboard.

If you are the engineering lead who got handed the compliance ask, this is the map. Not a vendor list, a data flow. The difference matters, because a list tells you what is allowed and a flow tells you where you already broke the rule.

The BAA is a chain, not a checkbox

HIPAA splits the world into covered entities (providers, plans, clearinghouses) and business associates (anyone who handles PHI on their behalf). The moment a vendor creates, receives, stores, or transmits PHI for you, they become a business associate and need a signed BAA. The HHS guidance on business associates is blunt about the subcontractor rule: liability flows downstream. If your agent calls a service that calls another service, both need coverage. A gap anywhere in that chain is a gap in the whole thing.

The mental model that keeps teams honest is what I call tracing the token. Pick one piece of PHI, say a patient's name inside a support message, and follow it hop by hop through your architecture. Ingress gateway. Orchestration layer. Prompt assembly. The LLM call. The retrieval query. The response. The log line. The trace span. The cache write. At every hop, ask a single question: does this vendor have a BAA, and does this specific product fall under it?

You will find that the language model is almost never the problem. Every major provider signs BAAs now. The problem is the four or five ancillary systems that copy the payload for convenience: observability, debugging logs, analytics, a semantic cache someone added to cut cost. Those systems were never on the compliance review because they felt like plumbing, not data stores. They are data stores. They just hold your PHI in a place your auditor never looked.

A reference architecture built around the PHI boundary

Design the stack as concentric zones rather than a pipeline. The boundary is the line PHI is not allowed to cross without a BAA on the other side.

Zone 0, the clean room. Anything that never sees PHI: your marketing site, generic documentation retrieval, model-selection logic, feature flags. Keep this zone large. The less code that touches PHI, the smaller your audit surface.

Zone 1, the BAA-covered core. This is where PHI lives and moves. It contains your orchestration runtime, the LLM inference endpoint, the vector store holding embedded clinical text, and the primary application database. Every service here runs under a signed BAA or inside infrastructure you own. Encryption in transit and at rest is table stakes; NIST SP 800-66 Rev 2 is the practical reference for what the Security Rule actually expects around access control, audit controls, and encryption, and it is written for implementers rather than lawyers.

Zone 2, the danger zone. Everything PHI tends to leak into by accident: application logs, request tracing, eval and observability platforms, error monitoring, and caching layers. Treat this zone as guilty until proven covered. Either bring each tool inside the BAA boundary or strip PHI before it arrives.

A clean healthcare agent request looks like this. The ingress gateway authenticates and authorizes. The orchestration layer (in your VPC, behind private networking) assembles the prompt from a retrieval call to your vector store and a policy lookup. It sends inference to a covered endpoint. The response returns through the same layer, which writes an audit record to your covered database and returns the answer to the caller. Nothing crosses into Zone 2 unless that specific tool has been vetted. If you are building the audit side of this properly, the audit trail design for regulated industries is worth its own review, because "we log everything" and "we log compliantly" are different systems.

The vendor BAA matrix

Here is where each major building block stands. The important column is the last one, because "signs a BAA" is rarely the whole story. Coverage attaches to specific products and specific endpoints, not to the company name on the invoice.

Two practical notes. First, a signed BAA is necessary, not sufficient. You still have to configure the covered service correctly, which usually means private networking, disabled training on your data, and retention set to the minimum. Second, the fewer distinct vendors in Zone 1, the fewer BAAs you have to negotiate, renew, and prove. Consolidation is a compliance strategy, not just an architecture preference.

Where PHI actually leaks

This is the part most posts skip, and it is where real deployments fail their first audit. The leaks are almost always in Zone 2, and they are all variations of the same mistake: a system copied the payload for a good engineering reason, and nobody classified the copy.

Application and access logs. The default behavior of most web frameworks and API gateways is to log request and response bodies at debug level, or to log them on error. A single unhandled exception can dump a full prompt containing a patient record into a log aggregator that has no BAA. Redact at the logging layer, not after the fact, and log identifiers or hashes instead of raw content.

Eval traces and observability. Agent tracing tools exist to capture the full input, the retrieved context, the tool calls, and the output. That is precisely the PHI you are trying to contain. If you send production healthcare traffic to a default-tier tracing SaaS, you have shipped PHI to a vendor with no agreement. Either self-host the tracer inside Zone 1, upgrade to a tier that signs a BAA, or run a scrubbing step that removes identifiers before the span is exported.

Prompt and semantic caching. Caching inside a covered provider (Bedrock, the Anthropic API) is fine, because the cache lives in already-covered infrastructure. The trap is the caching layer teams add themselves to cut token cost: a semantic cache keyed on embeddings, or a proxy that stores request and response pairs. Those often run outside the boundary and persist PHI in a store your auditor never sees. Before enabling any cache, find out where the bytes physically land.

Embeddings are still PHI. A vector is not a de-identification technique. An embedding of a clinical note is a lossy but reversible enough representation that it must be treated as PHI, and embedding endpoints and vector stores belong inside Zone 1 for the same reason the chat model does. HHS has never blessed embeddings as de-identification, and neither should you.

Tool calls that fan out. The whole point of an agent is that it calls tools in a loop, and each tool that receives PHI is another link in the chain. An eligibility check against a payer API, a fax gateway, a scheduling system: every one is a business associate if it sees patient data. This is also where the security surface widens, since an agent with data access plus external reach is the classic exfiltration risk covered in how AI agents leak data.

De-identification and the self-host escape hatch

There is one clean way to move data out of the PHI boundary entirely: de-identify it. HIPAA recognizes two methods in the HHS de-identification guidance. Safe Harbor requires removing 18 specific identifiers (names, dates more specific than year, geographic detail below state level, and so on). Expert Determination requires a qualified statistician to certify a very small re-identification risk. Once data clears either bar, it is no longer PHI, and the vendor that receives it no longer needs a BAA.

Do not treat this as a free pass. Structured fields are straightforward to scrub. Free-text clinical notes are not, because a discharge summary can re-identify a patient through a rare diagnosis, an unusual date sequence, or a provider name buried mid-sentence. Machine de-identification of narrative text is useful but imperfect, and a single missed identifier means you shipped PHI to an uncovered vendor while believing you were safe. If de-identification is load-bearing for your compliance story, validate it against held-out records and keep the raw path inside the boundary as the default.

The self-hosting decision follows the same logic. Running Postgres with pgvector on infrastructure you control removes a vendor from the chain, which is genuinely simpler on paper. What it hands you in return is real: disk encryption, key management, network isolation, patch cadence, backup encryption, and access logging that you now own end to end. Self-hosting is the right call when your security team already operates hardened database infrastructure and wants fewer external agreements. Managed services win when you would rather inherit a mature control set and a signed BAA than build one. Neither is universally correct, and the honest version of this tradeoff belongs in the same conversation as taking an agent from prototype to production, where the compliance work usually turns out to be the long pole.

How OpenNash Can Help

Most teams do not fail this because they cannot read a compliance doc. They fail because the PHI boundary was never drawn as an artifact, so the leaks got added later by well-meaning engineers optimizing cost and observability. OpenNash builds the boundary first: we map the token path through your specific stack, produce the vendor BAA matrix for the components you actually use, and design the ingress, logging, and retrieval layers so PHI stays inside Zone 1 by construction rather than by reminder. That includes the parts teams skip, like scrubbing at the tracing layer, verifying cache persistence, and setting up audit records that hold up when an assessor asks for them.

We are not the right fit for everyone. If your data is already cleanly de-identifiable and your use case is low-risk, a managed platform with a BAA may be all you need, and we will tell you that. Where custom work earns its keep is a real PHI workflow with fan-out tool calls, strict auditability, and a requirement that you own the deployment after handoff. If that is the situation you are scoping, book a call to map this reference architecture to your workflow.

Start by tracing one token. If you cannot name every system it touches and prove each one is covered, you do not have a BAA-ready stack yet. You have a demo that hasn't met its auditor.