AI Data Privacy for Business: Risks, Laws and a Practical Plan

One in five organizations in IBM’s 2025 breach research reported a breach linked to shadow AI, which means employees using AI tools the company never approved. Companies with a lot of shadow AI paid about $670,000 more per breach on average. In many of these cases the cause is plain. Someone pasted a customer list, a contract or a block of source code into a chat window, and the data left the building.

That is the core of AI data privacy for business. Your team already uses AI, with or without a policy. What matters is where prompts and files go, who can read them, how long they are kept and which laws apply once personal data is involved.

This guide covers the main AI data privacy risks, the data privacy laws that matter (GDPR, the EU AI Act, CCPA and a few more), where your data travels when you use ChatGPT, an API or a self-hosted model, and a plan you can start this week. You also get a table of privacy settings in the main AI tools, a list of things that should never go into an AI tool, an incident plan, an outline for an employee policy and a checklist.

Key takeaways

• Most AI data privacy incidents start with staff using personal or unapproved tools, not with outside attackers.
• The plan decides a lot. Personal plans may train on your chats, while business and API plans usually do not by default.
• “No training” is not “no retention”. Check storage time, access, location and subprocessors too.
• GDPR, the EU AI Act and California rules can apply at the same time, so map your data before you pick tools.
• An approved tool, a one-page policy and a DPA with each vendor remove most of the risk.
• If someone pastes personal data into an unapproved tool, treat it as a possible breach and act within hours.

What AI data privacy means for a business

AI data privacy covers how personal and confidential data is collected, used, stored and shared when an AI system touches it. That includes data used to train or fine-tune a model, text typed into a prompt, files uploaded to a tool and the answers the tool produces. The old data privacy principles still apply: a clear purpose, only the data you need (data minimization), a legal basis, honesty about what you do with it, and respect for people’s right to see and delete their data.

What changes in AI data privacy is how data moves. A normal database keeps a record in one place. An AI workflow can copy the same record into a prompt, a log file, a vector database, a vendor server and a monitoring tool within seconds. Data collected for customer support can end up training a marketing model. The model can also infer things people never told you. If some of the terms in this guide are new, the AI terms glossary explains them in plain language.

AI data privacy vs AI security

Data privacy asks whether you are allowed to use the data this way and whether people know about it. Security asks whether someone who should not have the data can get to it. You need both, and one does not cover the other. A locked-down server full of customer data you had no right to use for AI training is secure but not compliant. A tool with a solid legal basis and weak access controls is compliant on paper and leaks in practice.

What counts as business data in an AI tool

Most people think only of the prompt. The list is longer:

• Prompts and chat history
• Uploaded files such as contracts, spreadsheets and PDFs
• CRM, billing and support records
• Source code and internal documents
• Metadata such as account details, timestamps and usage logs
• The outputs the tool generates, which can repeat what you put in

How AI systems collect data

AI tools get data in more ways than a form field. Models are trained on huge collections of text, images and code, often scraped from the public web. Apps collect what users type and upload. Connected tools pull in email, calendars, files and chat history. Some products add cameras, sensors and voice, which brings in biometric data such as faces and voiceprints. Behind the scenes, systems also write logs, store usage data and keep text as embeddings in vector databases. Each of these is a separate place where personal data can sit, and each needs its own answer to the question of why you hold it.

The main AI privacy risks businesses face

Most AI privacy problems come from a handful of sources. Some start with your own people, some with vendors and some with the way the model works.

Employees pasting data into AI tools

This is the biggest and most boring AI privacy risk, and the usual form of shadow AI. Someone needs a quick summary, the approved system is slow, and a personal chatbot account is already open in another tab. A 2024 Cyberhaven report found that almost 75% of ChatGPT accounts used at work were personal ones, with fewer controls than a business plan. The fix is not a stern memo. It is an approved tool that is almost as easy to use, plus clear rules, which the guide on AI in the workplace covers from the employee side.

Vendors that train on or keep your prompts

Every one of the third-party AI vendors you use is another place your data can sit. Consumer plans of many tools can use your chats to improve their models unless you switch that off. Business and API plans usually do not train on your data by default, but they may still store it for a while. Your data privacy position depends on which of these you are on, and if you never read the vendor terms you do not know. Terms also change, so a check done last year is not a check.

Model memorization and output leakage

Language models can memorize rare or repeated pieces of training data and repeat them later, including names, emails and phone numbers. Research has shown that large models can be made to repeat training data, including personal details. The same risk exists inside your company. A model fine-tuned on support tickets can repeat a customer’s details to the next person who asks the right question. This applies to any generative AI for business project that trains or fine-tunes on internal records, and it is one of the main routes for data leakage.

Inference and profiling of sensitive traits

Models are good at spotting patterns, so they can guess things nobody typed in. Purchase history can hint at pregnancy or money trouble, and location data at religion or health visits. Under GDPR and other data privacy laws, inferred data about a person is still personal data, and some of it counts as special category data with stricter rules. If your system scores, ranks or segments people, treat those guesses as data you hold.

Training data used without a valid basis

If a model was trained on personal data that was scraped or reused without a lawful basis, the problem can follow it into your product. That matters whether you build the model or buy it. European regulators have said that unlawful processing during development can create legal risk for the company that later deploys the model. Ask vendors where their training data came from, and for your own projects, record the legal basis before you start training or fine-tuning.

Black box decisions you cannot explain

Many AI systems cannot show why they produced a result. That becomes a privacy issue as soon as the result affects a person, for example a rejected loan, a filtered job application or a flagged transaction. People have a right to meaningful information about automated decisions, and you cannot give it if your own team does not understand the tool. Ask vendors for documentation on how the system works, keep a person in the loop for decisions with real consequences, and log what the system decided.

Data reused for a new purpose

Data collected for one reason tends to drift into another. Support transcripts become training data, a CRM export feeds a sales model and call recordings go into an analytics tool. The GDPR purpose limitation rule says you need a compatible reason or a new legal basis for that. Check every new AI use against what you told people when you collected the data.

Prompt injection and data exfiltration

An attacker can hide instructions in a web page, email or document that an AI assistant reads. The assistant follows them and sends private information somewhere it should not, or forwards a file. This is called prompt injection, and the result is data exfiltration, meaning data theft through the assistant itself. Any assistant that reads outside content and also has access to internal data needs firm limits on what it can send out.

Over-permissioned RAG and AI agents

Connect an assistant to Google Drive, SharePoint, your CRM and Slack, and the vendor privacy terms stop being the main issue. The issue becomes permissions. If the assistant can read the acquisition folder, so can anyone who asks it a clever question. The same goes for AI agents for business that take actions. They need user-level permissions, audit logs and an approval step for anything that sends, deletes or pays.

Where your business data actually goes

A vendor saying “we do not train on your data” answers one question out of four. Ask all four before you approve any tool, because each one is a separate data privacy question.

Consumer accounts vs business accounts

For AI privacy, the plan type matters more than the brand. On personal plans, many providers may use your conversations to improve their models unless you opt out. On business, enterprise and API plans, the big vendors say they do not train on your data by default, though you should confirm that in the current terms. Handing employees an approved business workspace is a very different policy from telling them to use whatever account they already have.

API retention and zero data retention

APIs are usually cleaner than chat apps, but they are not invisible. Several providers keep API requests for about 30 days for abuse monitoring unless zero data retention is switched on for your account. Even then, zero retention covers the provider side only. Your own application logs, a monitoring service or a routing layer can still hold a copy. “No training” and “no retention” are two different promises.

Self-hosted, open-weight and cloud models

Self-hosting a chat interface does not make the model private. If that interface sends your prompt to a remote API, the prompt still leaves your network, and your hosting company joins the list of parties that handle it. Running an open-weight model on hardware you control keeps the prompt inside, but you take on patching, monitoring, backups and GPU costs.

Watch the difference between using a vendor’s hosted service and running the same model yourself. DeepSeek’s hosted service, for example, stores data in China under its privacy policy, while downloaded weights running in your own cloud never contact DeepSeek. For European customers, check data residency as well, because some providers host in the EU by default and others do not.

When to self-host and when a hybrid setup fits

Self-hosting makes sense when data is highly regulated, contracts forbid sending it to third parties, or the volume is big enough to justify the hardware and a team to run it. For most companies a hybrid setup works better. Sensitive tasks run on a model you control or on a business plan with strict terms, while low-risk tasks such as drafting public copy use a standard tool. Model routers and gateways add another party between you and the model, so treat them as vendors and check their logging and retention as well.

Privacy settings in ChatGPT, Gemini, Copilot and Claude

AI data privacy depends heavily on which plan someone is using. The table below summarizes what each vendor says in its own documentation as of October 2026. Menus and terms change, so confirm the details before you roll anything out.

Business plans cost more per seat, and the guide on how much AI tools cost shows typical price ranges. The extra money buys admin controls, a DPA and no training by default. It buys nothing if half the team keeps using free personal accounts, so the rollout matters as much as the purchase.

Laws and rules behind AI data privacy

Which data privacy rules apply depends on where your customers live, what data you handle and what the AI does with it. Most companies end up under more than one set of AI privacy rules.

GDPR

The GDPR is the main data privacy law behind most AI projects, and it applies to personal data of people in the EU, wherever your company sits. For AI, the parts that bite are a lawful basis for each use of data (consent is only one of them), purpose limitation, data minimization and transparency. Article 22 gives people the right not to be subject to decisions made only by automated processing when those decisions have legal or similarly significant effects, which covers things like hiring and credit scoring. The right to erasure is awkward when personal data has already shaped a trained model. Fines reach 20 million euros or 4% of global annual turnover, whichever is higher.

Lawful bases for AI projects

GDPR gives you six lawful bases, and consent and legitimate interests come up most often with AI. Consent has to be specific, informed and easy to withdraw, which is hard to prove when data was collected for something else. Legitimate interests need a written balancing test that weighs your benefit against the impact on people. The European Data Protection Board gave an opinion on AI models in December 2024 and said legitimate interests can work but must be assessed case by case. Pick the basis before you start, and write it down.

EU AI Act

The EU AI Act entered into force in August 2024 and applies in stages. It sorts systems by risk, bans a few practices such as untargeted scraping of faces for recognition databases, and puts strict duties on high-risk systems: risk management, data governance, documentation and human oversight. It works next to the GDPR, not instead of it. A company outside the EU can be covered if its system is placed on the EU market or its output is used there.

CCPA, CPRA and California ADMT rules

Under California data privacy law, residents can ask what personal data a company holds, ask for it to be deleted or corrected, and opt out of its sale or sharing. Updated CCPA regulations that took effect on January 1, 2026 add rules for automated decision-making technology (ADMT), with notice, access and opt-out rights for certain uses and later compliance dates for some of them. Check the current dates before you build a timeline.

HIPAA and other sector rules

HIPAA covers health information in the US, GLBA covers financial institutions, FERPA covers student records, and Brazil’s LGPD works much like the GDPR. If your AI touches one of these data types, the sector rule adds to the general data privacy law. It does not replace it.

Cross-border data transfers

Sending personal data of EU residents to a provider outside the EU counts as a transfer under GDPR, and you need a valid mechanism for it. Common options are an adequacy decision, such as the EU-US Data Privacy Framework for certified US companies, or standard contractual clauses. Many AI vendors process data in the US or run support from several countries, so ask where prompts are processed and which subprocessors can see them. Some vendors offer EU data residency, which simplifies the question.

People’s rights over their data

People can ask what data you hold, ask you to correct it, ask you to delete it and object to certain uses. Under GDPR you generally have one month to respond. With AI, the hard part is finding the data. One person’s details can sit in chat logs, support tickets, a vector database and a fine-tuning dataset. Deleting a record from a trained model is often not possible, so keep personal data out of training sets where you can and make sure you can remove it from everything else.

NIST AI RMF and ISO/IEC 42001

Neither is law. The NIST AI Risk Management Framework gives you four functions (govern, map, measure and manage) to organize how you deal with AI risk. ISO/IEC 42001 is a standard for an AI management system and can be certified. Both help when a customer or auditor asks how you run AI governance and handle AI data privacy, and both work as a skeleton for an internal program.

What to do when personal data goes into an AI tool

An AI incident does not need an attacker. If an employee pastes customer data into an unapproved tool, the data went to a third party without a legal basis, and depending on the data and the vendor terms that can count as a personal data breach. Under GDPR you must tell the supervisory authority within 72 hours of becoming aware of a breach that is likely to put people at risk, and tell the affected people if the risk is high. A short plan saves time:

1. Stop the exposure: remove access, close the account or switch off the connector
2. Record what was shared, with whom, when and in which tool
3. Ask the vendor to delete the data and confirm it in writing
4. Assess the risk to the people involved and decide whether you must notify anyone
5. Fix the cause, whether it is a missing approved tool, a training gap or loose permissions

Cases that show how it goes wrong

Samsung and ChatGPT

In 2023, Samsung engineers reportedly used ChatGPT to debug confidential code, which meant that code sat on a system Samsung no longer controlled. The company banned generative AI tools internally soon after. Nobody broke in. People used a convenient tool without knowing how it handled input. That is why an approved alternative matters more than a ban.

Zoom, LinkedIn and Clearview AI

In 2023 Zoom changed its terms in a way that appeared to let it use customer calls for AI training. Users objected, and the company revised the terms to say it would not train on customer content without consent. In 2024, LinkedIn users noticed they had been opted in automatically, with no fresh consent, to having their data used to train generative models. Clearview AI scraped billions of face images from the web without permission and drew fines and bans from European regulators.

The lesson on AI data privacy is the same each time: public or already collected does not mean free to use for AI. Customer chat tools raise the same question, because they store personal details by design, and the AI customer service guide looks at that from the support side.

AI data privacy best practices for business

None of these steps needs a big budget. They need an owner and a few weeks. Together they add up to privacy by design: protection built in before launch, not patched on after a complaint.

Build an AI inventory and map your data flows

List every AI tool in use, approved or not. For each one, write down what data goes in, where it is stored, who can access it and who the vendor passes it to. This also exposes shadow AI. A simple spreadsheet works, and it doubles as the record of processing that GDPR asks for. If you are still deciding what belongs in your stack, the piece on an AI tool stack for business shows how to keep it small and controlled.

Minimize data before it reaches an AI tool

Send only what the task needs. This is the data minimization principle from GDPR applied to prompts. Strip names, account numbers and addresses before pasting text into a prompt. Use IDs instead of names where you can. If you need 3 fields, do not send 10. Less data in means less data privacy risk and less to explain to a regulator.

Anonymize or pseudonymize where you can

Remove or replace identifiers before data reaches a tool. Pseudonymization swaps names and IDs for codes and keeps the key somewhere else, but the result is still personal data under GDPR. Real anonymization means nobody can be identified, even with extra information, and that is hard to reach with text, location or behavior data. Do not call a dataset anonymous until someone has tried to re-identify it.

Use business accounts and approved tools

Pay for business or enterprise plans with admin controls, single sign-on and a written promise not to train on your data. Block or warn on personal accounts through your browser and network tools. A tool people like using will beat a policy people ignore, and it does more for AI privacy than any warning email.

Encrypt data and limit access

Check that data is encrypted in transit and at rest. Give each person access to only the data and tools their job needs, turn on audit logs, and use data loss prevention tools that flag sensitive text before it leaves. Treat every AI tool as a possible route for data leakage.

Set retention and deletion rules

Decide how long chats, logs, uploaded files and embeddings are kept, and set auto-delete in the admin console. Data you no longer hold cannot leak or be demanded by anyone. Add AI tools to your normal retention schedule, and confirm that deleting a chat also removes it from backups and connected stores.

Vet vendors and sign a data processing agreement

When a vendor processes personal data for you, it is a processor under GDPR and you need a data processing agreement (DPA) in place. Vendor data privacy terms often sit in separate documents, so ask for the DPA, the security overview and the subprocessor list. Before signing, get clear answers to these questions:

• Does the vendor train on your data, and can you turn that off?
• Where is data processed and stored?
• How long are prompts and outputs kept, and is zero data retention available?
• Which subprocessors are involved?
• How do deletion and access requests work?
• Does it hold SOC 2 or ISO 27001 certification?

The guide on how to choose AI tools for business helps with the non-privacy side of the comparison, such as fit, price and support.

Run a DPIA before you deploy

A data protection impact assessment (DPIA) is required under GDPR Article 35 when processing is likely to be high risk, which usually includes profiling, automated decisions and large-scale use of sensitive data. Do it before launch, while the system can still change, which is what privacy by design means in practice. Think of it as the standard data privacy check for high-risk systems. Put legal, engineering and the business owner in the room, and update it when the model, data source or purpose changes.

Test and monitor before and after launch

Test a system before it goes live. Try prompts that attempt to pull out private data, check whether it repeats details from training or retrieval sources, and see what happens when the input contains personal data you did not expect. After launch, review outputs and logs on a schedule, and repeat the tests whenever you change the model, the data source or the permissions.

Use privacy-enhancing technologies where they fit

• Differential privacy adds statistical noise so trends show up without exposing individuals.
• Federated learning trains a model across devices or sites without moving raw data.
• Synthetic data replaces real records with artificial ones for testing and training.
• Homomorphic encryption lets a system compute on encrypted data.

These data privacy techniques cost time and money, and most small and mid-sized companies get more from minimization and access control first.

Update your privacy policy and ask for consent

Your privacy policy should spell out your AI data privacy practices in plain words. Name each AI feature, say what data it uses, say whether data is used for training and list the third-party AI providers involved. A line like “we may use AI to improve our services” will not hold up. Ask for separate consent when you want to use customer data for something new, such as training a model, and let people withdraw it.

Name a privacy owner or a DPO

Someone has to own this. GDPR requires a data protection officer for public bodies and for organizations that monitor people or process sensitive data at scale. Everyone else still needs a named privacy lead who approves new tools, keeps the inventory current and handles incidents. Without a name, nobody checks.

How AI can help protect your data

The same technology can help. AI-based tools can scan systems to find and label sensitive data, flag unusual access patterns, catch personal data in outgoing prompts and speed up replies to access and deletion requests. They add to the steps above and do not replace them, and they raise their own privacy questions, because they also read your data.

AI data privacy in healthcare, finance, retail and HR

Some sectors face stricter rules because the data is more sensitive.

• Healthcare: patient data falls under HIPAA in the US. Before staff use any AI tool with health information, get a business associate agreement (BAA) with the vendor and confirm the tool is covered by it.
• Finance: banks and lenders handle financial data under GLBA, PCI DSS for card data and national regulators. Automated credit and fraud decisions raise GDPR Article 22 questions.
• Retail and marketing: shopper profiling, personalized offers and chat assistants rely on behavior data. Check consent, opt-out rights and what the tool does with chat transcripts.
• HR and hiring: screening tools rank people, so bias and automated decision rules apply. New York City, for one, requires a bias audit and notice for automated employment decision tools.

What should never go into an AI tool

If you wonder what not to put in ChatGPT or any other chatbot, start with this list. Keep these out unless the tool is approved for that data type and covered by an agreement:

• Passwords, API keys and access tokens
• Customer personal data such as names with contact details, ID numbers and payment data
• Health, biometric and other special category data
• Contracts and documents under NDA
• Unreleased financial results and board material
• Source code and trade secrets
• Employee records, salary data and HR cases

If you are unsure about something, ask the person who owns data privacy before you paste. Waiting an hour costs less than a breach notice.

AI usage policy for employees: what to include

A one-page AI usage policy beats a forty-page one nobody reads, and it is the fastest way to put AI governance and AI data privacy rules into practice. It should cover:

1. Which tools are approved, and who to ask about a new one
2. Which data classes (public, internal, confidential, regulated) may go into which tool
3. What is banned outright, using the list above
4. Rules for personal accounts and personal devices, which is where shadow AI usually starts
5. Whether a person must review AI output before it goes to customers
6. Limits for AI agents, including which actions need approval
7. How to report a mistake without blame, and what happens next

Train people on it with examples from their own job, and review it every six months, because tools and terms change fast. For the wider rollout, the walkthrough on how to use AI in business goes through it step by step.

AI data privacy for small business

A small company has fewer systems but the same data privacy duties. Start with three moves. Pay for business plans so training is off by default. Turn off model improvement in the settings of any consumer tool you still use. Sign a DPA with every vendor that touches customer data. Then write the one-page policy above. You do not need a privacy team, but you do need one named person who owns the topic, even if it is only part of their job. The AI for small business guide has examples of tools that fit a small team and budget.

Common AI data privacy mistakes

These come up again and again when companies review how they use AI:

• Banning AI tools without offering an approved alternative, which pushes use out of sight
• Assuming a business plan label means no risk, without reading retention, subprocessors and location
• Forgetting meeting recorders, browser extensions and plugins that capture calls and pages
• Connecting AI tools to shared drives without fixing permissions first
• Calling data anonymous when it is only pseudonymized
• Never rechecking vendor terms after the first approval
• Writing a policy and skipping training and enforcement

AI data privacy checklist

Use this AI data privacy checklist as a quarterly review. Each line should get a clear yes or no.

1. Every AI tool in use is listed in an inventory
2. Each tool has an owner and an approved data type
3. Business plans are in place, with training off by default
4. A DPA is signed with each vendor that processes personal data
5. Data flows are mapped, including logs and connected systems
6. A DPIA is done for high-risk uses
7. Access follows least privilege, with audit logs on
8. The privacy policy names AI features and providers
9. Employees have a written policy and short training
10. An incident plan covers AI mistakes, and the whole setup is reviewed twice a year

Conclusion

Good AI data privacy for business is less about one tool setting and more about knowing the full path your data takes: from the person typing, through the vendor, the logs and the connected systems, to deletion. Companies that map that path, give employees an approved tool and write down simple AI governance rules avoid most of the incidents described here. Start with the inventory and the one-page policy. Everything else builds on those two.

Frequently asked questions

What is AI data privacy?

AI data privacy is the protection of personal and confidential information as AI systems collect, process, store and generate it. It covers training data, prompts, uploaded files and outputs, and it follows the same principles as general privacy law: purpose, minimization, transparency and user rights.

Does ChatGPT train on my business data?

It depends on the plan. Personal accounts may use chats to improve models unless you opt out. OpenAI states that ChatGPT Business, Enterprise and API data is not used for training by default. Terms change, so check the current policy and confirm the setting in your admin console.

Does GDPR apply to AI tools?

Yes. GDPR applies whenever an AI system processes personal data of people in the EU, whether you built the system or use a vendor’s. You need a lawful basis, a clear purpose, only the data you need and a DPA with any vendor that processes data for you.

Does the EU AI Act replace GDPR?

No. The AI Act regulates AI systems by risk level, while the GDPR keeps governing personal data. If a system processes personal data in a relevant context, you may have to meet both sets of rules at the same time.

What should employees never enter into AI tools?

Passwords, API keys, customer personal data, health and other special category data, NDA material, unreleased financials, source code and HR records. The exception is a tool your company has approved for that data and covered with an agreement.

Is zero data retention the same as private?

No. Zero data retention means the provider does not keep your prompt or response after processing. Your application logs, monitoring tools, routing layers and connected systems may still store copies, and the prompt still reached the provider. Treat it as one control among several.

How can small businesses protect data when using AI tools?

Use business plans, turn off model training in settings, share only the data a task needs, sign a DPA with vendors, write a one-page usage policy and name one person who owns privacy. These steps cost little and cover most of the common risks.

When is a DPIA needed for an AI system?

Under GDPR, a DPIA is needed before processing that is likely to create a high risk for individuals. Profiling, automated decisions with significant effects and large-scale use of sensitive data usually qualify. Run it before launch and update it when the system changes.

Can employees use personal ChatGPT accounts for work?

It is best to say no. Personal accounts may use chats for training unless the user opts out, and the company gets no admin control, audit log or DPA. Give staff an approved business account instead, and state the rule in your AI usage policy.

What should we do if someone pasted customer data into an AI tool?

Act quickly. Note what was shared and where, delete the content, ask the vendor to confirm deletion and assess whether it is a personal data breach. Under GDPR you may have 72 hours to notify the authority if the risk to people is likely.

Do we need a data protection officer?

Under GDPR, public bodies and organizations that monitor people or process sensitive data at scale must appoint one. Others do not have to, but should still name a privacy lead who owns AI approvals and incidents.