Subscribe to Our AI Marketing Newsletter

Issue #48: Where Does Your Data Actually Go When You Use AI?

TL;DR

When an AI vendor says “we don’t train on your data,” that answers only one privacy question.

It does not automatically mean your prompts are not retained.

It does not mean your data stays inside your company.

And it does not mean nobody else can access it.

The privacy picture changes depending on whether your team uses a consumer AI account, a business workspace, an API, a routing service, a cloud-hosted open model, or a model running entirely inside infrastructure you control.

The bigger issue for most companies is not picking the “most private AI.”

It is understanding the complete path the data takes.

1. “Does It Train on My Data?” Is the Wrong First Question

This is usually where the AI privacy conversation starts.

A company asks:

“Does ChatGPT train on what we type?”

Or Claude.

Or Grok.

Or whatever AI tool someone on the team started using last month.

That is a reasonable question.

But it is not enough.

There are at least four different things you need to know:

Does the provider use your data to train its models?

How long does it retain your prompts and outputs?

Who can potentially access that information?

Where is the data processed and stored?

A “no training” policy answers the first question.

It does not answer the other ones.

That distinction matters a lot once employees start putting customer data, contracts, financial information, internal documents, or intellectual property into AI systems.

2. Consumer AI and Business AI Are Not the Same Product

This is one of the easiest privacy mistakes for a company to make.

Someone already has a personal ChatGPT account.

Someone else prefers Claude.

Another employee is using Grok.

So the company thinks:

Why pay for business accounts? Everyone already has AI.

Because the privacy terms can be different.

Business and enterprise products generally give organizations stronger controls around model training and data handling than consumer accounts.

The same applies when you move from a consumer chat interface to an API.

But stronger privacy terms do not mean zero exposure.

An API provider may still retain requests temporarily for abuse monitoring or security.

That is why the next layer matters.

3. Zero Data Retention Is Better. It Still Does Not Mean “Nothing Left the Building.”

Some AI providers offer Zero Data Retention, usually called ZDR.

That sounds like the end of the privacy conversation.

It is not.

ZDR generally means the model provider processes the request and does not persist the prompt and response afterward.

Useful?

Absolutely.

But the prompt still went to the provider.

And other systems may still have seen it.

Your application may have logged it.

Your observability software may have captured it.

Your routing layer may have processed it.

A connected agent may have passed part of it to another tool.

This is why we prefer a much simpler question:

Show us the complete path the data takes.

AI Privacy Is an Architecture Problem, Not a Checkbox

If you are not sure what happens to company data when employees use ChatGPT, Claude, Grok, APIs, agents, RAG systems, or open models, talk it through with Avi.

We can map where the data is going, what is being retained, which systems have access, and where the real exposure exists.

Set up a meeting here

No need to arrive with an AI security architecture. Start with the tools your team is already using.

4. Self-Hosted AI Changes the Equation

Suppose you install something like OpenWebUI on infrastructure your company controls.

Now your employees have one internal interface instead of everyone using separate AI accounts.

That is already a meaningful improvement.

Your conversation history can stay inside your environment.

You can control access.

You can centralize which models employees are allowed to use.

But here is the important part:

If OpenWebUI sends the prompt to OpenAI, Anthropic, xAI, Mistral, or another external model provider, the prompt still leaves your environment.

You self-hosted the interface.

You did not self-host the model.

Those are very different things.

5. Running the Model Yourself Is Different

Now change the architecture again.

Instead of calling an external model API, run an open-weight model on infrastructure you control.

That could be a Llama model, a Mistral model, a DeepSeek model, or another open-weight model.

Now the prompt does not have to leave your environment at all.

This is where people often get confused about models such as DeepSeek.

Using DeepSeek’s hosted service is one thing.

Downloading the model weights and running them on your own server is another.

The model may have the same name.

The privacy architecture is completely different.

That distinction can matter enormously for companies dealing with sensitive documents, healthcare data, legal work, proprietary processes, financial information, or confidential customer records.

6. The Router Matters Too

Another layer companies increasingly use is an AI router.

OpenRouter is one example.

A router lets your application access many different models through one API.

Very convenient.

But now you have another question to answer:

Who actually processed the request?

The same model can sometimes be available through several different infrastructure providers.

Those providers can have different data retention policies.

Different geographic locations.

Different security terms.

Different Zero Data Retention options.

So choosing the model is not enough.

You may also need to control the provider that serves the model.

This is a good example of why the phrase “we use Model X” tells you surprisingly little about privacy.

7. The Biggest Leak May Be Sitting in Your Office

You can design an excellent enterprise AI architecture.

Then an employee copies a confidential customer spreadsheet into a personal ChatGPT account because the approved system felt inconvenient.

That is shadow AI.

And for many companies, it is probably a much more immediate risk than debating which frontier model has the best privacy policy.

This is why a corporate AI policy cannot simply say:

“Do not put confidential information into AI.”

That sounds good in the policy document.

Then real work starts.

Someone has a deadline.

They need to summarize a contract.

They need to analyze a spreadsheet.

They need an answer in five minutes.

And the AI tool they already know is sitting in another browser tab.

Governance only works if you give employees an approved alternative that is easy enough to use.

Otherwise people route around the policy.

8. Agents and RAG Create a Completely Different Privacy Problem

Now suppose the company builds an internal AI assistant.

It connects to Google Drive.

The CRM.

Slack.

Email.

Maybe the accounting system.

The model provider can have perfect privacy terms and you can still have a serious security problem.

Why?

Because the AI now has access to your internal systems.

An employee asks:

“What did we decide about the acquisition?”

The AI finds a confidential document in Drive and answers the question.

The problem was not model training.

The problem was permissions.

As companies move from AI chatbots to AI agents, questions around identity, document permissions, tool access, audit logs, and human approval become much more important.

The AI is no longer just reading what someone pasted into a prompt.

It can go looking.

Read the Full Article

The newsletter only scratches the surface.

The full article goes deeper into:

Consumer AI accounts versus business workspaces

API privacy and temporary retention

Zero Data Retention and what it really covers

OpenWebUI and self-hosted AI architecture

DeepSeek hosted versus DeepSeek running locally

OpenRouter and provider routing

Data residency

RAG and agent permissions

Shadow AI

When a business should run models inside its own infrastructure

Why hybrid AI architectures may make more sense than trying to make everything fully private

Want to understand where your company’s AI data is actually going? Let’s talk.

Set up a meeting with Avi

Final Thought

The AI privacy question should not be:

“Does this vendor train on our data?”

The better question is:

“Show us everywhere our data goes.”

From the employee’s browser.

To the AI application.

Through the API.

Through a router.

Into a model provider.

Into logs.

Into connected systems.

Into storage.

That is the architecture you need to understand.

Because AI privacy is not a feature you buy.

It is a system you design.

And as AI moves deeper into company operations, that distinction is going to matter a lot more.

Read the full AI Privacy for Business article here

Thanks for reading Signal Over Noise,
where we separate real business signal from AI noise.

See you next Tuesday,

Avi Kumar

Founder: Kuware.com

Subscribe Link: https://kuware.com/newsletter/

Signal Over Noise: AI Insights for Business Leaders