Every healthtech founder I speak to in 2026 has some version of this question, usually asked slightly sideways because they are worried the answer is no.
The answer is not no. It is also not the yes people are hoping for. Here is the framework I actually use, as the fractional CTO of a platform running AI over medical imaging.
Start with the distinction that most discussions skip
A vendor can be HIPAA-eligible. A use is compliant. These are not the same thing and conflating them is the most common mistake I see.
Getting a BAA signed feels like the finish line. It is closer to the starting gun. The BAA establishes that the vendor accepts obligations as a business associate. It says nothing about whether:
- you sent more data than the task required,
- your users understood their information would be processed this way,
- the model’s output — which now contains PHI — is being written somewhere uncovered,
- the feature should exist at all.
I have seen a team with a properly executed BAA pipe completions into a logging service with no BAA, and a separate team send an entire patient record when the task needed two fields. Both had the paperwork. Neither had a defensible position.
The four options, in the order I evaluate them
1. Do not send PHI
The one everyone skips, and often correct.
A surprising share of “we need AI on patient data” turns out to need AI on something adjacent to patient data. Summarising public clinical guidelines. Drafting appointment reminders from a template. Categorising inbound support messages. None of that requires a single identifier to leave your boundary.
Before designing a compliant pipeline, spend an hour establishing that you need one. It is the cheapest hour in the project.
2. De-identify, then send
If you can reduce the payload below the PHI threshold before it crosses your boundary, the third-party question largely disappears — you are no longer disclosing protected health information.
HIPAA gives you two routes: Safe Harbor, which enumerates identifiers to remove, and expert determination, which is a statistical opinion that re-identification risk is very small.
Where this works well: structured data, lab values, coded observations, anything where the identifying fields are separable from the clinically meaningful ones.
Where it breaks:
- Clinical free text. Notes are dense with re-identifying detail that is not in an identifier field — an unusual occupation, a specific date, a named facility. Automated de-identification of notes is a genuine research problem, not a library call, and the tools that exist have error rates you need to actually measure rather than assume.
- Imaging. The pixels can be identifying, and DICOM headers carry a great deal beyond the obvious fields. Stripping headers is necessary and not sufficient.
Do not treat de-identification as a checkbox. If you claim it, be prepared to defend the method.
3. BAA-covered API, minimum necessary
This is where most real systems land.
The mechanics: an executed BAA with the provider, the specific endpoints and models the BAA covers (it is rarely everything they sell), zero-retention configured where offered, and a hard discipline of sending only the fields the task requires.
The parts teams get wrong, in the order I find them:
The output is PHI too. A completion generated from PHI is PHI. If you log prompts and responses for debugging — and you will want to — that log is now a PHI store. It needs the same encryption, access control, audit trail, and retention policy as the record it was derived from, and whoever hosts it needs a BAA. Debug logging is the single most common way PHI ends up somewhere nobody inventoried.
The vendor stack is deeper than the vendor. Your error tracker sees exception payloads. Your APM sees request bodies. Your analytics sees event properties. Each is a separate business associate question and each is easy to forget, because nobody thinks of Sentry as a healthcare vendor.
Minimum necessary is a real obligation. Sending the whole record because it is easier is not a neutral engineering choice. Build the payload deliberately, field by field, and be able to explain each one.
Model choice interacts with scope. Routing to a cheaper model on a different provider is an ordinary optimisation right up until it silently moves PHI to a vendor with no BAA. Whatever routing layer you build, make the covered-vendor set an explicit constraint in code rather than a convention.
4. Self-host
Removes the disclosure question. Replaces it with owning an inference stack’s security posture, patching, access control, and audit trail.
For medical imaging this is often the right trade — the payloads are large, hard to de-identify, and the volume makes per-token pricing unattractive anyway. For occasional text summarisation it is usually over-engineering.
Be honest that “self-hosted” is not a synonym for “secure.” It means the exposure is yours.
The architecture that keeps scope from spreading
Whichever option you pick, the structural goal is the same: inference over PHI should not widen your compliance boundary.
Concretely, on the platform I work on:
- The inference path runs against controlled storage with explicit, logged access. Model calls are made from inside the boundary, not from a client.
- Prompt and completion logging is treated as a PHI store from day one — same encryption, same access controls, same retention clock, same audit trail as the clinical records.
- Product analytics and telemetry operate on de-identified data. Answering “how many users hit this feature” must never require reading a PHI store, or every dashboard becomes a compliance surface.
- The set of vendors that may receive PHI is an explicit allowlist enforced in code, not a shared understanding.
- Every model interaction over PHI is recorded in the audit trail as an access event, because it is one.
That last point deserves emphasis. If a regulator asks who accessed a record, “an LLM summarised it at 14:22 on request from clinician X” needs to be an answer you can produce. Most teams instrument human reads and forget machine ones.
What I would actually tell you on a call
Establish whether you need PHI in the prompt at all. If yes, de-identify if the data shape permits. If it does not, get the BAA — and then spend your real effort on the parts nobody signs: what you send, where the output goes, which vendors are in the path, and whether the audit trail knows about any of it.
The paperwork is the easy half. The architecture is where teams get caught.
Verify current vendor terms directly. Provider policies on BAAs, retention, and covered endpoints have changed repeatedly and will change again. Anything you read about specific contractual terms — here or anywhere — should be confirmed with the vendor in writing before data moves. I implement to requirements; your counsel and compliance officer set them.
I’m a fractional CTO working at the intersection of healthtech and applied AI — currently running production AI over medical imaging under HIPAA. If you’re designing this and want a second opinion before you build it, the 30-minute call is free. If you want it examined properly, that’s the Technology Health Check.