On Premise AI vs Cloud: Real Costs for Private Files

Every few weeks someone in Metro Vancouver asks us the same question. They have client files, medical charts, or payroll data, and they want to know whether using AI means handing all of it to a company in California. It's a fair question. The honest answer is that choosing between on premise AI and a cloud API is less about security theatre and more about three things you can actually price: what the vendor contract says, what the hardware costs, and who is going to keep it running at 2am.

Here's what the published sources say, checked in September 2026.

What the two options actually are

On premise AI means running an open weight model on hardware you control, whether that's a GPU server in your Burnaby office or a machine you rent and administer yourself. Nothing leaves your network unless you send it.

Cloud AI means calling a vendor's API. Your prompt goes out, an answer comes back, and the vendor's terms govern what happens to that text in between.

Most owners assume the second option means their files get used to train somebody's model. For the major enterprise platforms, that isn't what the documentation says.

Read the cloud contract before you buy a server

Microsoft's data, privacy and security page for models sold by Azure in Microsoft Foundry states that your prompts, completions, embeddings and training data "are NOT available to other customers," "are NOT used by providers of Models sold by Azure to improve their models or services," and "are NOT used to train any generative AI foundation models without your permission or instruction." The same page describes the models as stateless: "no prompts or completions are stored in the model."

Keep reading, though, because there's a detail the marketing summaries skip. Abuse monitoring can store a sample of your text. Microsoft's page says that when abuse indicators are detected, "a sample of customer's prompts and completions may be selected for review," done by automated means "with additional reviews by human reviewers as necessary." Those reviewers are authorized Microsoft employees working through Secure Access Workstations with just in time approval. Customers can apply for modified abuse monitoring, which turns off that storage and human review, and you can confirm it's off by checking that the ContentLogging capability reads "false" in the Azure portal.

Location matters too. Standard deployments process prompts inside the geography you pick, but the page is explicit that Global and DataZone deployment types may process your prompts "in any geography where the relevant model sold by Azure is deployed." If cross border processing is your actual concern, the deployment type you choose matters more than the on premise question.

The published cloud numbers

Anthropic's pricing documentation, as published in September 2026, lists Claude Haiku 4.5 at $1 per million input tokens and $5 per million output tokens, Claude Sonnet 5 at $2 and $10, and Claude Opus 5 at $5 and $25. The Batch API takes 50% off both directions, and a prompt cache hit costs 0.1x the base input price.

That same page gives a worked example worth memorizing: processing 10,000 support ticket conversations averaging about 3,700 tokens each on Haiku 4.5 comes to roughly $37.00.

The published on premise numbers

On the hardware side, NVIDIA's specifications for the L40S list 48GB of GDDR6 memory and a max power consumption of 350W. It's a common inference card, and it's the GPU behind AWS G6e instances, where a single card comes with 48GB of GPU memory.

For power, BC Hydro's General Service business rates publish a Small General Service energy charge of 14.06 cents per kWh, plus a basic charge of 40.89 cents per day.

Now some illustrative arithmetic, not a quote from anyone. One L40S drawing its full 350W for a year is 0.350 kW times 8,760 hours, or 3,066 kWh. At BC Hydro's published 14.06 cents, that's about $431 a year in energy charges for the card. Real cards don't sit at full draw all day, so treat it as a ceiling.

Here's the uncomfortable comparison. That same $431, spent at the published Haiku 4.5 rate, would cover roughly 116,000 of those support ticket conversations. Electricity is not what makes on premise expensive. The card, the server around it, the spare in case it dies, the patching, and the person who owns all of it are what make it expensive. Nobody publishes a price for that last one.

When on premise genuinely wins

• A contract or a regulator flatly forbids disclosing the data to a third party processor, and no data processing agreement fixes it
• Volume is high and steady enough to keep the GPU busy, so you're paying for capacity you actually use
• The site is air gapped, or connectivity is unreliable, or the workload has to run offline
• The corpus itself is the business asset and you won't put it on anyone else's disk at any price
• You already run infrastructure and have someone who patches it, not someone who would inherit it

If two or more of those are true for your New Westminster or Vancouver operation, price the on premise build seriously. If none are, you're probably buying a server to solve a paperwork problem.

Where this doesn't apply

Some honest counterweights.

Most confidential data leaks through staff pasting into a free consumer chatbot on a personal account, not through a sanctioned API. An on premise server does nothing about that. A written acceptable use policy and one approved tool do more for your exposure than any hardware purchase.

Open weight models you can self host still trail the frontier hosted models on hard reasoning and long document work. A wrong answer generated inside your own building is still a wrong answer, and you now own the review process for it.

You also become the security team. A neglected GPU box running an unpatched inference server is a worse outcome than a vendor with an audited compliance program and a published data handling document.

And accountability does not move with the hardware. Canada's federal, provincial and territorial privacy authorities published principles for generative AI in December 2023 that state accountability "for decisions rests with the organization, not with any automated system," and direct organizations to only use tools that respect privacy laws, to use anonymized or de identified information in prompts where possible, and to apply safeguards "commensurate to the sensitivity of the information." None of that changes based on where the GPU sits.

Regulators do act on this. The joint investigation of OpenAI reported in May 2026 was conducted under PIPEDA alongside Quebec's private sector law, Alberta's PIPA, and British Columbia's PIPA, and found problems with consent, transparency about training data, accuracy of personal information in outputs, and retention policy. BC PIPA applies to you too.

Finally, scale changes the calculus. Statistics Canada reported that 19.2% of Canadian businesses used AI to produce goods or deliver services in the 12 months before the second quarter of 2026, up from 6.1% two years earlier. In that same release, cybersecurity or privacy concerns were a barrier for 30.0% of businesses with 100 or more employees but only 11.6% of those with 1 to 4 employees. Larger firms worry more, and they usually have the compliance obligations and the volume to justify the build.

The middle path most shops end up on

Classify your data first. Sort it into what can never leave, what can go to a contracted processor under a data processing agreement, and what nobody would care about. Most businesses find the never leave bucket is smaller than they feared, and that a cloud API with the right deployment type, retention settings, and a signed agreement covers the middle bucket. Then, if the never leave bucket is big enough and busy enough, price a local model for that slice alone.

If you want help sorting your files into those three buckets and pricing both paths honestly, book a free call with Autana Solutions. We'll read your actual contracts and workloads before recommending anything, and if the answer is that you don't need a server, we'll tell you that.

Sources

• Anthropic. "Pricing," Claude Platform Docs, 2026. https://platform.claude.com/docs/en/about-claude/pricing
• Microsoft. "Data, privacy, and security for Foundry Models sold by Azure in Microsoft Foundry," Microsoft Learn, 2026. https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy
• Office of the Privacy Commissioner of Canada and federal, provincial and territorial counterparts. "Principles for responsible, trustworthy and privacy-protective generative AI technologies," 2023. https://www.priv.gc.ca/en/privacy-topics/technology/artificial-intelligence/gd_principles_ai/
• Office of the Privacy Commissioner of Canada. "PIPEDA Findings #2026-002: Overview of the Joint Investigation of OpenAI OpCo, LLC," 2026. https://www.priv.gc.ca/en/opc-actions-and-decisions/investigations/investigations-into-businesses/2026/pipeda-2026-002-overview/
• Statistics Canada. "Analysis on artificial intelligence use by businesses in Canada, second quarter of 2026," 2026. https://www150.statcan.gc.ca/n1/pub/11-621-m/11-621-m2026010-eng.htm
• NVIDIA. "NVIDIA L40S" product specifications, 2026. https://www.nvidia.com/en-us/data-center/l40s/
• Amazon Web Services. "Amazon EC2 G6e Instances," 2026. https://aws.amazon.com/ec2/instance-types/g6e/
• BC Hydro. "General Service Business Rates," 2026. https://app.bchydro.com/accounts-billing/rates-energy-use/electricity-rates/business-rates.html