A self-hosted large language model runs entirely on infrastructure your team controls, whether cloud GPUs you rent or servers you own, so your prompts and customer data never leave your environment. ChatGPT Enterprise routes traffic through OpenAI's systems under contractual privacy terms that, by default, exclude your data from model training, according to OpenAI's January 2026 enterprise privacy page.
The right choice turns on four factors: where customer data goes, what vendor contracts say about training and retention, what compute actually costs, and what your team must operate on its own.
What exactly is a self-hosted LLM, and why do marketing teams care?
A self-hosted LLM is an open-weight model, such as Meta's Llama family or Mistral's released weights, loaded onto GPU compute that you rent or own and exposed to your tools through an API you control. No inference request leaves your environment. For a marketing team that stores customer purchase history, email engagement data or first-party behavioral signals alongside a model, that boundary matters for data governance and, in some categories, for legal compliance.
The deeper question is what your AI vendor's contract actually permits. Keeping prompts private is not the same as understanding what a vendor can see. For a fuller picture of the governance options available to ecommerce brands, the piece on private AI for business explains the spectrum from fully hosted to fully self-hosted. The distinction shapes every cost and staffing decision that follows.
Where does customer data go under each model?
With ChatGPT Enterprise, your prompts and outputs stay within OpenAI's infrastructure. OpenAI's January 2026 enterprise privacy page states that business data is not used for training by default, and that zero data retention is available on request for eligible API endpoints. Fine-tuned models are isolated: they are never served to other customers and never used to train other models, and training files are kept until you delete them.
When ChatGPT models are accessed through Microsoft's Azure AI Foundry, Microsoft adds a further contractual layer. Microsoft's May 2026 privacy documentation states that prompts and outputs are not available to OpenAI or other model providers sold through Azure. Data at rest is encrypted with AES-256 by default, with an option for customer-managed keys. Microsoft also states that data is not used to train foundation models without your explicit permission or instruction.
With a self-hosted model, no inference data leaves the GPU environment you rent. The trade-off is that you become the data controller in full: access logs, encryption, retention schedules and incident response are your team's responsibility, not a vendor's.
What do the vendor contracts say about training and retention?
Anthropic's September 2026 data retention documentation covers what it calls Covered Models, a designation that extends to its most capable releases. Anthropic explains the scope of that designation this way:
This applies to Mythos-class models and future models with similar capabilities that we designate as covered models.
For organizations running those models, Anthropic offers customer-managed encryption keys and access transparency audit logs as optional additions. On the question of how reviewer access is controlled, Anthropic's documentation is specific:
Every instance of access is recorded in a tamper-proof log that reviewers cannot suppress or modify.
That level of auditability matters if your brand operates in regulated categories such as health, finance or children's products, where demonstrating data access controls is a compliance requirement, not just a preference. With a self-hosted model, you produce those audit logs yourself, which requires tooling and staffing your team may not currently have.
What does compute actually cost, and what are you comparing?
ChatGPT Enterprise is sold per seat. The Brand Media Works register holds no Enterprise seat price, so a direct comparison with GPU-hour pricing needs a quote from OpenAI.
For self-hosting, you pay for GPU compute by the hour. List prices shift frequently, so the table below reflects prices read directly from provider pricing pages on 15 October 2026. Check each provider's page before you budget.
| Provider | Published price | Read on |
|---|---|---|
| Together AI | HGX H100, dedicated inference: $5.49 per GPU per hour HGX H100, GPU clusters, hourly: $3.99 per GPU per hour | 15 Oct 2026 |
| RunPod | H100 NVL pod (94 GB): $3.19 per hour H100 PCIe pod (80 GB): $2.89 per hour H100 SXM pod (80 GB): $3.99 per hour | 15 Oct 2026 |
| CoreWeave | HGX H100 node (8 GPUs), on demand, North America: $49.24 per hour L40S node (8 GPUs), on demand, North America: $18.00 per hour A100 node (8 GPUs), on demand, North America: $21.60 per hour | 15 Oct 2026 |
| Lambda | H100 cluster, 16 GPUs, 2 weeks to 1 year: $6.16 per GPU per hour H100 cluster, 64 GPUs, 2 weeks to 1 year: $5.85 per GPU per hour H100 cluster, 256 GPUs, 2 weeks to 1 year: $5.54 per GPU per hour | 15 Oct 2026 |
Read the units before comparing. CoreWeave's $49.24 per hour is for a whole HGX H100 server of eight GPUs, which works out at about $6.16 per GPU per hour (our arithmetic). RunPod's single H100 pods ranged from $2.89 to $3.99 per GPU per hour on the same date. RunPod describes its enterprise reserved clusters as scaling to 10,000 or more GPUs with SLA-backed uptime and discounted rates.
Lambda lists commitment-term pricing for clusters of 16 to 256 H100 GPUs, ranging from $5.54 to $6.16 per GPU per hour, all requiring a sales conversation. The compute cost for self-hosting is real and recurring, and it sits on top of the engineering hours needed to deploy, secure and maintain the model.
What does your team have to run itself, and where do brands struggle?
Self-hosting transfers operational responsibility entirely to your team. That includes model deployment, version management, inference scaling, access control, encryption at rest and in transit, audit logging and incident response. Scaling AI use beyond a handful of successful pilots often stalls because the technical and compliance infrastructure was never built to support it at volume.
Finding people who combine hands-on ML infrastructure skills with enough marketing domain knowledge to configure a model usefully is genuinely hard. Retaining them is harder. Aligning internal stakeholders on what the system should and should not do, and managing the expectation that a self-hosted model behaves identically to a frontier API, adds further friction. These are the practical constraints that determine whether a self-hosting project ships or sits unfinished for months.
BCG's June 2026 paper, citing its AI Radar survey, found that 72% of CEOs still see themselves as the main decision maker on AI across the enterprise, which means that even well-resourced marketing teams often cannot move on infrastructure decisions without executive sign-off on something the executive has not directly evaluated. That governance gap slows deployment regardless of which model architecture you choose.
When does self-hosting clear the bar, and when does it not?
Self-hosting is worth the operational cost when at least one of these conditions is true: your legal or compliance team has confirmed that prompts containing customer PII cannot leave a specific jurisdiction; your security policy prohibits any inference data touching a third-party API regardless of contractual controls; or you need to fine-tune on proprietary catalog or behavioral data and the fine-tuning data retention terms of hosted providers are not acceptable to your counsel.
For most $5M-$200M ecommerce brands, ChatGPT Enterprise or Azure-hosted models satisfy the data governance requirements that actually exist today, at lower operational overhead and with contractually defined training-use defaults. The private LLM on open-weight models route makes sense when the compliance case is clear, the engineering capacity exists or can be contracted, and the brand's use case genuinely requires fine-tuning on data that cannot leave its own environment.
If you are still mapping the cost of bringing in outside expertise, the piece on AI consultant cost in 2026: what consultancies charge covers how different types of firms structure and price that work. Brand Media Works, an AI consulting and marketing data engineering firm for ecommerce brands, structures engagements around the decision your team is actually facing, not a preferred architecture.
Frequently asked questions
Does ChatGPT Enterprise use my marketing team's data to train OpenAI's models?
No. OpenAI's January 2026 enterprise privacy page states that business data is not used to train models by default, and that zero data retention is available on request for eligible API endpoints.
Can I fine-tune a hosted model on my own customer data without that data being shared?
OpenAI's January 2026 documentation states that fine-tuned models are for your use alone and are never shared with other customers or used to train other models, and that training files are retained until you delete them. Microsoft's Azure AI Foundry documentation states that prompts and outputs are not available to OpenAI or other model providers sold through Azure.
What is the minimum GPU setup a DTC brand needs to self-host a capable open-weight model?
It depends on the model size and your inference volume, and no single published figure covers every case. RunPod lists single H100 pods from $2.89 per GPU per hour, and Lambda lists reserved H100 clusters at $5.54 to $6.16 per GPU per hour.