Decide where each piece of work runs The useful question isn't whether AI is secure. It's where each piece of work runs, and what it can reach once it's there. Our view at IDG is short: where a system runs is a decision, not a default, and it is decided workload by workload. There are three places worth considering for work you cannot afford to leak: - A machine in your office. The system runs on hardware on your own network, and the office walls are the boundary. - Your own cloud account. The system runs in an account your organization owns, under your identity rules, your network rules and your keys. - A private setup run for you. An environment set apart for your organization by someone else, for work you would rather not host yourself. A public tool is not on the list. By that we mean free chat apps and free developer tiers whose terms let the vendor use or review what you type. For sensitive work we rule them out. The same organization can and usually should use more than one of these. Searching client files, drafting from public material and producing month-end reports have different risks, different quality needs and different people to look after them. The rest of this guide helps you place each one. It is not legal advice; where a professional rule applies, your counsel decides. What makes a tool public, in a vendor's own words The clearest definition of a public tool comes from a vendor. Google's Gemini API Additional Terms of Service (last updated 2026-04-28, checked 2026-10-25) split its developer API into Unpaid and Paid Services. For the unpaid tier, Google says it uses what you submit and the responses "to provide, improve, and develop" its products, that "human reviewers may read, annotate, and process your API input and output", and, in plain terms: "Do not submit sensitive, confidential, or personal information to the Unpaid Services." The paid tier is different. The same terms say Google logs prompts and responses "for a limited period of time, solely for detecting and preventing violations" of its use policy. Google's separate usage policies page (last updated 2026-06-09) says it retains prompts, contextual information and output for "fifty-five (55) days" to detect and prevent policy violations, and does not split that figure by tier. Two lessons follow. First, the same model can sit behind very different terms, so ask which contract applies as well as which model. Second, even paid terms usually keep something for abuse checks. The next sections show how much, and who can read it. If your staff already paste work into free tools, start there. What to do when staff are already using public AI covers that first. A machine in your office Here the model runs on hardware you own, on your own network. Prompts, files and answers do not go to an outside AI vendor. For work whose data carries the strictest rules, that removes a whole category of questions about third parties. It is practical now because capable models are published with open weights under permissive licenses. Two examples, as their publishers describe them (checked 2026-10-25): - OpenAI's `gpt-oss-20b` model card says it runs "within 16GB of memory", and `gpt-oss-120b` on "a single 80GB GPU". The card lists an Apache 2.0 license. - Google's Gemma 4 documentation lists memory to load the weights: 17.5 GB for the 31B model at 4-bit, 69.9 GB at 16-bit. Google notes that these figures cover loading the weights and leave out the extra memory for the context window or supporting software. Desktop-sized hardware has grown into this. NVIDIA's DGX Spark page lists a 128 GB unit for models of "up to 200B parameters". Those are vendor statements, and we have not tested them. The trade-off is model quality. Epoch AI found that since January 2026 "the most capable open-weight models have lagged frontier closed models by an average of four months" on its capabilities index. Our view is that those leading open models are usually too large for an office machine. For models small enough for one consumer graphics card, Epoch's 2025 study found they caught the frontier within 6 to 12 months, and cautioned that the real-world lag "is likely longer" because the best-scoring open models are more likely to be overfit to benchmarks. Treat both as estimates on benchmarks, not on your documents. The other cost is upkeep. Someone has to patch the machine, back it up, watch it and update models. OWASP's guidance on supply-chain risk for AI applications warns that "LLM models on device increase the supply attack surface" and that "Currently there are no strong provenance assurances in published models." Local is not automatically safe. It moves the security work to you. In our view the office machine suits steady work on your most sensitive material where a capable small model is enough: search across your own files, extraction, summaries of documents you supply. Your own cloud account Here the model runs as a service inside a cloud account your organization owns. The three large clouds publish similar basic terms for this. What they say, as checked on 2026-10-25: - Microsoft's page on data privacy for Foundry Models sold by Azure says prompts and completions "are NOT available to OpenAI" or other model providers, and are not used to train foundation models "without your permission or instruction". - AWS says Amazon Bedrock model providers "don't have any access" to the accounts that run their models, and so no access "to customer prompts and completions", and that "your content is not used to improve the base models". - Google, which has renamed Vertex AI as Gemini Enterprise Agent Platform, says it "won't use your data to train or fine-tune any AI/ML models without your prior permission or instruction." The differences are in the details your security lead will ask about. Abuse monitoring. Microsoft says that when its systems detect possible abuse, "a sample of customer's prompts and completions may be selected for review", by AI models by default and by human reviewers "as necessary". Customers who meet extra eligibility criteria can apply for modified abuse monitoring, after which "the data storage and human review process described above is not performed", though "automated review may still be conducted". AWS describes a "zero operator access (ZOA) data security model" for Bedrock, then lists exceptions by model. For most OpenAI models it keeps classifier-flagged traffic for up to 30 days, and eligible customers can ask their AWS account team for full zero data retention. For two named Anthropic models it keeps all traffic for up to 30 days, and flagged traffic may get human review by AWS. Google says it "may log prompts" for abuse monitoring, and that to reach zero data retention "customers must take specific actions" across several features. Where the work is processed. Each cloud has a global option that widens it. Azure's "Global" deployments "may be processed in any geography where the relevant model sold by Azure is deployed". Bedrock's global cross-Region inference routes to "Any supported AWS commercial Region worldwide", and AWS lists it at about 10% cheaper than staying in a geography. Google says requests to its `global` endpoint "may be processed in any Google Cloud location around the world." Each also has a regional or geographic setting that keeps processing inside a boundary you choose. Network. Each offers a private path. Azure lets you disable public access so that "Private endpoint connections will be the exclusive way to access this resource." AWS describes using PrivateLink so "your data isn't available over the internet". Google says a VPC Service Controls perimeter blocks "all public internet access" to the service. The pattern that matters: the strictest settings take an approval or a deliberate configuration step. They are not the default. AWS, for example, has a data retention mode, `none`, under which "No request or response data is written to durable storage by AWS or shared with the model provider", and Bedrock will block calls to a model that requires retention. Someone has to set it. The cloud suits work that needs a frontier model or room to grow. You still own the identity rules, keys, network and these settings. A private setup run for you Some work is too sensitive for a public tool, but nobody in the firm wants to look after a machine or a cloud account. A private setup run by someone else covers that gap: an environment set apart for your organization, inside the operator's own. The catch is that you are now trusting the operator, and the same questions apply to them as to any cloud vendor. Who can read the data, how long is it kept, where is it processed, what happens when you leave. IDG offers this option, so read this paragraph as applying to us too. Ask us the questions below, and expect the answers in writing. For tax preparers, part of this is written into federal rules. The FTC's guide to the Safeguards Rule lists "tax preparation firms" among the businesses it covers, and says your contracts with service providers "must spell out your security expectations, build in ways to monitor your service provider's work, and provide for periodic reassessments of their suitability for the job." Worked example: a law firm and ABA Formal Opinion 512 The ABA's ethics guidance (Formal Opinion 512) says a lawyer needs the client's informed consent before client information goes into a self-learning AI tool. The opinion, dated July 29, 2024, explains why: such tools "raise the risk that information relating to one client's representation may be disclosed improperly, even if the tool is used exclusively by lawyers at the same firm." It adds that "merely adding general, boiler-plate provisions to engagement letters" to authorize AI use "is not sufficient." The opinion says the risk analysis "will be fact-driven and depend on the client, the matter, the task, and the GAI tool used to perform it." That is the workload-by-workload question in an ethics frame. Where the tool runs affects the facts the opinion asks about. It cites earlier cloud-computing and outsourcing opinions that suggest lawyers should ensure a tool "is configured to preserve the confidentiality and security of information, that the obligation is enforceable, and that the lawyer will be notified in the event of a breach or service of process regarding production of client information", and "determine whether the [GAI tool] retains information submitted by the lawyer before and after the discontinuation of services". Two cautions. The opinion does not say that any place of deployment removes the need for consent, and neither do we. And the opinion is based on the ABA Model Rules, and it anticipates that state and local bar ethics committees will likely offer updated guidance, so check your own state's rules and opinions too. Bring this to your ethics counsel with the facts of each workload. Worked example: a tax practice and IRC 7216 Preparers generally need the taxpayer's consent before using or disclosing return information (IRC §7216). The statute makes it a misdemeanor for a preparer to knowingly or recklessly disclose return information, or use it "for any purpose other than to prepare, or assist in preparing" a return. The regulations define disclosure as making the information known "to any person in any manner whatever." Read plainly, sending return information to an outside AI vendor looks like a disclosure, and our view is to treat it as one until your counsel says otherwise. The regulations at 26 CFR 301.7216-2 allow some disclosures without consent. One is to another preparer "located in the United States" to prepare a return or provide auxiliary services, "so long as the services provided are not substantive determinations or advice affecting the tax liability". In our reading, then, where a vendor processes the work, inside or outside the United States, can change the analysis. In our view, that is one more reason the regional settings in the cloud section matter. In June 2026 the IRS Office of Professional Responsibility issued Introductory Guidelines for Responsible AI Use in Federal Tax Practice (Issue Number 2026-19). It warns of disclosure risk "especially when data is uploaded to unsecured or public systems" and says practitioners "must strictly handle all client data using only secure, enterprise-approved AI." As we read it, it mentions §7216 for its penalties and does not say whether sending return information to an AI vendor needs consent. A model on the preparer's own machine does not send return information to an outside AI vendor. Whether that settles the consent question for a given use is a legal judgment for the practice's counsel, not for us. Four questions that place each piece of work We place work by answering four questions about it, in this order: - What is the strictest rule its data carries? Client confidences, return information, health records and contract terms each come with their own duties. The stricter the rule, the fewer outside parties you want in the picture. - How good does the model need to be? Search and summary of your own files is often within reach of a model that fits in an office. Long, open-ended drafting and reasoning usually needs a frontier model in the cloud. - Is the volume steady or spiky? Steady work suits hardware you own. Work that comes in bursts, or will grow, suits the cloud. - Who will look after it? If nobody will patch a machine or manage cloud settings, that decides a lot by itself. Then pick the least exposed place that still meets the quality bar. Here is how one organization might split its work across the three places: - Search across client files goes on a machine in the office, because the files are the most sensitive thing the organization holds. - Drafting from public sources goes in its own cloud account, because nothing in it is confidential and it needs room to grow. - Month-end figures and reports go to a private setup run for it, because nobody wants to look after a machine, and the work still stays out of public tools. That is an example, not a template. If you are still deciding which work deserves AI at all, start with whether anything is worth building. IDG's Where it runs page shows the same three places and the rules that apply inside each. The rules that apply wherever it runs The place sets the outer wall. What happens inside it matters as much. We hold every system to three rules, wherever it runs: - Nobody reaches anything through AI they couldn't already open. A person who cannot open a folder today cannot reach it through AI either. This speaks to the within-firm risk the ABA opinion describes, though it does not replace consent. - A person approves anything consequential. Drafts move freely; anything that goes out, gets signed or changes a record waits for a person. - Every step is written down: who asked, what was used, what came back. The Safeguards Rule points the same way for covered firms: the FTC's guide calls for a Qualified Individual to supervise the information security program, and for notice to the FTC "no later than 30 days after discovery" of a breach involving at least 500 consumers' unencrypted information. In our view, an office machine that holds client data belongs inside that program. What to get in writing before anything runs Bring your security lead to the first conversation with any provider, IDG included. These are the questions we would want answered in writing for each workload: - Is our data used to train or improve any model, and where does the contract say so? - What is kept for abuse monitoring, for how long, and can a person read it? Have we applied for the strictest setting, and how do we confirm it is on? - Which processing location is configured: regional, geographic or global? Who can change it? - Is public network access turned off, and how do our offices connect? - Who holds the encryption keys? - What happens to our data, logs and system when we stop working together? - Who patches, backs up and monitors this, and who is told when something goes wrong? Vendor terms change; four of the Google pages cited here were updated three days before we checked them. The ones quoted here were checked on 2026-10-25; recheck them before you rely on them, and again each quarter. IDG's Infrastructure service builds the environment in whichever place you choose.