The fastest route to keeping customer data inside your account is deploying a managed inference service, such as AWS Bedrock or Microsoft Foundry, inside your own cloud account with zero data retention enabled. For full isolation, you run an open-weight model on dedicated GPUs you rent or own. RunPod lists H100 PCIe Pods at $2.89/hr as of October 2026, per RunPod's GPU Cloud Pricing page, which is the floor for serious inference capacity.
| Provider | Published price | Read on |
|---|---|---|
| RunPod | H100 NVL pod (94 GB): $3.19 per hour H100 PCIe pod (80 GB): $2.89 per hour | 15 Oct 2026 |
| CoreWeave | HGX H100 node (8 GPUs), on demand, North America: $49.24 per hour L40S node (8 GPUs), on demand, North America: $18.00 per hour | 15 Oct 2026 |
| Lambda | H100 cluster, 16 GPUs, 2 weeks to 1 year: $6.16 per GPU per hour H100 cluster, 64 GPUs, 2 weeks to 1 year: $5.85 per GPU per hour | 15 Oct 2026 |
What do the published GPU prices actually tell you?
RunPod's Pod pricing gives you a single-tenant GPU at $2.89/hr for an H100 PCIe or $3.19/hr for an H100 NVL, as published on their GPU Cloud Pricing page in October 2026. These are per-card rates for dedicated Pods, not shared inference endpoints, so your model weights and inference traffic stay on that hardware.
CoreWeave's on-demand HGX H100 cluster is listed at $49.24 per hour for North America as of October 2026, per CoreWeave's published pricing page. That rate covers a full 8-GPU HGX node, not a single card, which is why it sits higher in the table. Lambda publishes reserved rates for multi-GPU allocations: $6.16 per GPU per hour for a 16-GPU H100 reservation and $5.85 per GPU per hour for 64 GPUs, both requiring a direct conversation with their team. All of these list prices change; verify each provider's page before you finalize a budget.
The CoreWeave on-demand rate illustrates a real pain point: a brand that needs a full node for occasional batch jobs pays for idle capacity every hour. Committing to a reserved block at Lambda or RunPod, or switching to a serverless endpoint for variable traffic, closes that gap without requiring long hardware procurement cycles.
Which deployment model keeps customer data inside your account?
Understanding private AI for business starts with a simple principle: data isolation is a function of where inference runs, not which model you choose. Three patterns exist for ecommerce brands at the $5M-$200M scale.
The first pattern is a managed foundation model deployed inside your own cloud account with zero data retention. AWS Bedrock sets the baseline clearly.
Amazon Bedrock uses a zero operator access (ZOA) data security model, subject to the model-specific abuse and CSAM detection detailed below.
Bedrock also lets you configure retention at the account or project level per Region, but that setting does not travel automatically. AWS states that the setting governs every request in the Region where you set it and does not propagate to other Regions, so you must configure each Region your brand operates in separately. Zero data retention also has an exception for Anthropic's most capable models: since June 2026, Anthropic keeps prompts and outputs sent to its covered models (Mythos-class) for 30 days on every platform where they are offered, Bedrock included.
The second pattern is a self-hosted open-weight model, such as Llama or Mistral, running on GPUs you rent from RunPod, CoreWeave or Lambda. Inference never leaves your rented instance. RunPod supports enterprise clusters with guaranteed availability and SLA-backed uptime at scales up to thousands of GPUs, per their October 2026 pricing page. This approach requires your team to manage model serving, updates and access control, which is the tradeoff for complete network isolation.
The third pattern is on-premise hardware: physical GPUs in your data center or co-location facility. This is rarely the right starting point for a DTC brand, because the procurement cycle is long and capital costs are high. It becomes worth considering when you have strict data residency requirements outside the US or when inference volume justifies owned hardware over rented capacity.
For brands that want the Claude model family with additional controls, Anthropic makes customer-managed encryption keys available to eligible organizations. Microsoft Foundry applies AES-256 encryption at rest by default, with an option for customer-managed keys, per Microsoft's May 2026 data privacy page. Both options keep encryption control on your side of the account boundary. Our work on AI data security on AWS covers how these controls map to ecommerce compliance needs.
What do brands get wrong when they set this up?
The most common mistake is treating a single Region configuration as global. Bedrock's per-Region retention scope means a brand that sets ZDR in us-east-1 but routes some traffic through eu-west-1 for latency reasons leaves that second Region outside the policy until it is explicitly configured.
A second mistake is standing up a private model and stopping there. A private LLM with no connectors to your catalog, customer data or campaign tooling sits idle and produces no business value. Isolation and utility are not opposites, but they require separate engineering work. Brands that have completed the isolation step often need help wiring the model to the data it is meant to act on.
Prompt and file travel to a vendor's shared GPUs is the risk that motivates private deployment in the first place, and it is worth auditing every tool in the marketing stack that touches customer records, not only the primary LLM. Multiple disconnected AI subscriptions each handle data in their own way, and none of those habits are visible from a central point without deliberate instrumentation. For a detailed comparison of the tradeoff, see Self-hosted LLM vs ChatGPT Enterprise for a marketing team: when self-hosting is worth it.
Frequently asked questions
Does using AWS Bedrock mean our customer prompts are used to train Anthropic's models?
No. Anthropic's commercial terms state that it may not train models on customer content from its services. Retention is a separate question: prompts to Anthropic's covered models are kept for 30 days on every platform, Bedrock included, so set retention in each AWS Region your workloads use and check which models you call.
What encryption standard applies to data stored in Microsoft Foundry?
Microsoft states that stored data is always encrypted at rest with AES-256 by default, with an option to supply a customer-managed key, per their May 2026 data privacy page. Some preview features may not yet support customer-managed keys.
Can a $10M ecommerce brand realistically self-host an LLM on rented GPUs?
Yes, if inference volume is predictable enough to justify a dedicated Pod or cluster. At RunPod's published H100 PCIe rate of $2.89 per hour, a single-GPU Pod is a manageable line item; the real cost is the engineering time to serve, monitor and update the model.