TL;DR
- HIPAA compliance is not a feature you turn on.
- If your AI app touches PHI, the whole data path matters.
- HTTPS helps with data in transit, but that is only one piece.
- Data at rest, logs, backups, vector databases, and LLM calls must also be protected.
- A BAA is essential when vendors touch protected health information.
- The wrong AI vendor can break compliance even if your app code is solid.
- Part 1 explains the compliance checklist. Part 2 breaks down HIPAA-ready AI tools and platforms.
1. The question I keep hearing now is simple.
Can we claim this AI system is HIPAA-compliant?
It sounds like a yes-or-no question.
It is not.
That is the trap.
HIPAA compliance is not one setting. It is not one API checkbox. It is not “we use HTTPS, so we’re good.”
It is a full system of controls around who sees protected health information, where that information goes, how it is stored, how it is transmitted, who your vendors are, and whether you can prove all of it later.
That last part matters.
Because in compliance, “we think it was secure” is not the same as “we can show exactly what happened.”
2. AI makes HIPAA more complicated because the data moves more.
Traditional healthcare software was already hard enough.
But AI adds new layers:
- LLM prompts
- Model responses
- Vector databases
- Embeddings
- Logs
- Temporary memory
- Third-party APIs
- Cloud storage
- Human review workflows
That means PHI can leak into places teams didn’t even think about.
A prompt log.
A debug trace.
A vector database.
A support tool.
A screenshot.
That is why I keep saying AI compliance has to be designed from the beginning.
Not cleaned up later.
3. The simple framework is this.
For any AI app handling protected health information, ask three questions:
- Who can see the data?
- Where does the data go?
- How is the data protected?
That sounds basic.
Good.
Basic is what keeps you out of trouble.
If you cannot answer those three questions clearly, you are not ready to claim HIPAA compliance.
In Part 1 of the blog series, I broke this down into a practical checklist covering encryption, access control, audit logs, vendor risk, staff training, and breach response.
Read Part 1 here:
HIPAA Compliance for AI Systems: A Practical Checklist for AI Builders
HIPAA Compliance for AI Systems: A Practical Checklist for AI Builders
4. HTTPS is necessary, but it is not enough.
This came up in a real discussion recently.
If all API calls use HTTPS, then yes, you are protecting data in transit.
That is good.
But HIPAA does not stop there.
You also need to think about data at rest.
That means data sitting in:
- Databases
- File storage
- Backups
- Vector databases
- Logs
- Cached model outputs
If someone gets access to the raw storage, can they read the data?
If yes, you have a problem.
Encryption at rest means the stored data is unreadable without the proper keys.
And no, an API key is not the same thing as encryption.
An API key controls access.
Encryption protects the data itself.
Different layer. Different job.
5. Vector databases deserve special attention.
A lot of people assume embeddings are safe because they do not look like normal text.
That assumption is dangerous.
Embeddings are not automatically anonymous.
Depending on the system, data structure, metadata, and retrieval behavior, embeddings can still carry sensitive information or be tied back to patients.
So if your AI app uses a vector database for healthcare data, treat that vector database as if it contains PHI.
That means:
- BAA (Business Associate Agreement) coverage
- Encryption at rest
- Access controls
- Audit logs
- Clear retention rules
This is where a lot of AI prototypes fall apart when they move toward production.
6. The vendor chain can make or break you.
Let’s say your own app is secure.
Great.
But your app calls an LLM API.
That LLM API calls another system.
Your vector database stores embeddings.
Your logging provider stores request traces.
Your cloud provider stores backups.
Now the question is not “is our app secure?”
The question is:
Is every vendor touching PHI covered properly?
That is where BAAs come in.
A BAA, or Business Associate Agreement, is the contract that says a vendor who touches protected health information agrees to safeguard that data under HIPAA requirements.
If a vendor touches PHI and will not sign a BAA, that vendor should not be in your HIPAA stack.
Simple as that.
7. The practical vendor question is the one everyone really wants answered.
Which AI tools can we actually use?
That is what Part 2 covers.
The blog looks at the practical stack questions:
- Which LLM providers may support BAAs
- Which cloud platforms are commonly used in HIPAA environments
- Which vector databases are safer choices
- Which tools to be careful with
- Why SOC 2 is not the same thing as HIPAA readiness
- Why “enterprise-ready” does not automatically mean BAA-ready
This is the part where builders need to slow down.
A tool can be powerful.
A model can be smart.
A demo can look amazing.
But if it cannot fit into the compliance chain, it may not belong in a healthcare AI system.
Read Part 2 here:
HIPAA-Ready AI Tools in 2026: Which Ones Actually Have BAAs
HIPAA-Ready AI Tools in 2026: Which Ones Actually Have BAAs
8. Here is the blunt version.
If you are building AI apps that touch medical data, you need to know your stack.
Not vaguely.
Specifically.
You should be able to say:
- This is where PHI enters the system.
- This is where it is stored.
- This is where it is sent.
- This is who can access it.
- This is how it is encrypted.
- This is how access is logged.
- These are the vendors with signed BAAs.
- This is what happens if something goes wrong.
That is the level of clarity clients need.
And frankly, that is the level of clarity builders should demand from themselves.
9. Why this matters for business leaders.
Most companies do not want AI theory.
They want business results.
Faster intake.
Better documentation.
Smarter search.
Improved patient support.
Reduced admin burden.
But in healthcare and healthcare-adjacent businesses, the AI system has to earn trust first.
That trust does not come from saying “we use a secure model.”
It comes from architecture.
It comes from process.
It comes from documentation.
It comes from being able to sit across from a client and explain the data path without hand-waving.
That is where real AI consulting is going.
Not prompt tricks.
Not shiny demos.
Operational AI that can survive legal, security, compliance, and real-world use.
10. My take.
HIPAA-ready AI is going to become a serious differentiator.
A lot of people can build demos.
Far fewer can build systems that are useful, secure, documented, and defensible.
That gap is where the opportunity is.
Because clients do not just need AI apps.
They need AI apps they can actually deploy.
Final thought.
HIPAA compliance for AI is not about fear.
It is about maturity.
If you build the stack correctly, you can use AI in sensitive industries.
But you have to respect the data.
You have to know the vendors.
You have to control the flow.
And you have to document the system.
That is the difference between AI experimentation and AI implementation.
This week’s related blogs:
If this topic got you thinking, hit reply and tell me what you want me to cover next.
Should I break down:
- A real HIPAA-ready AI architecture
- OpenAI vs Azure OpenAI vs AWS Bedrock for healthcare AI
- How to handle PHI in prompts
- Vector databases and embeddings for regulated data
- How to sell compliant AI systems to healthcare-adjacent clients
Thanks for reading Signal Over Noise,
where we separate real business signal from AI noise.
where we separate real business signal from AI noise.
See you next Tuesday,
Avi Kumar
Founder: Kuware.com
Subscribe Link: https://kuware.com/newsletter/