For most ecommerce brands, Amazon Bedrock is the better first home for a private LLM: the model runs inside AWS, model providers never see the prompts, zero retention is a setting, and an open model costs well under a dollar per thousand customer replies. A self-hosted model on rented GPUs wins when the model runs most of the day, or when the brand needs a model or setup that Bedrock does not offer. The comparison below uses AWS's own terms and the published prices of AWS, RunPod and CoreWeave.
What happens to customer data on Bedrock and on a self-hosted LLM?
Both keep prompts away from the company that built the model. The difference is who runs the servers and who sets retention. On-premise or private-cloud LLM covers the deployment options in more detail.
| Question | Amazon Bedrock | Self-hosted LLM on rented GPUs |
|---|---|---|
| Who runs the model | AWS, called from the brand's AWS account | The brand or its partner, on servers it rents |
| Can the model maker see prompts? | No. AWS says model providers have no access to Bedrock logs, prompts or completions | No. The open-weight model's maker is not involved |
| Retention | Set per AWS Region. Mode "none" means zero retention. Anthropic's Fable models require up to 30 days of AWS review | Whatever the brand configures |
| Network and keys | Private connection from the brand's VPC through AWS PrivateLink. Keys in AWS KMS | The brand's own network and keys |
| Fine-tuning | Tuned on a private copy of the base model | Tuned on the brand's own servers |
AWS explains the separation in its Bedrock data protection documentation. Each model provider's models run in an AWS-owned deployment account that the provider cannot reach:
Because the model providers don't have access to those accounts, they don't have access to Amazon Bedrock logs or to customer prompts and completions.
Retention is the setting to check. Bedrock's "none" mode means no request or response data is written to durable storage by AWS, and AWS notes that only that mode guarantees zero retention. Some models require more:
For models requiring `aws_review` (currently Claude Fable 5 and Claude Fable 5.1): user prompts and completions are retained within the AWS boundary for up to 30 days and may be reviewed by AWS to meet the human review requirement the model provider imposes as a condition of access.
For support replies on order history or segmentation on customer profiles, that points to an open model on Bedrock with retention set to "none". Two details matter for residency: the setting applies only in the Region where it is made, and if cross-region inference is on, retained data is stored in the Region that processed the request.
How do Bedrock and GPU costs compare?
Bedrock bills per token, or per active minute for a model the brand imports. A GPU server bills every hour it runs, whether or not it is answering requests. The table shows the published prices that matter for this choice.
| Option | Published price | How it is billed |
|---|---|---|
| Bedrock, Qwen3 32B | $0.20 per million input tokens, $0.78 per million output tokens | Per token, nothing when idle |
| Bedrock, DeepSeek v3.2 | $0.62 per million input tokens, $1.85 per million output tokens | Per token, nothing when idle |
| Bedrock, fine-tuning Qwen3 32B | $80 per training hour, $1.95 a month to store the model | Per training hour |
| Bedrock, imported custom model | $0.05718 per Custom Model Unit per minute. A Llama 3.1 70B model needs 8 units | Billed in 5-minute windows. Scales to zero after 5 minutes without requests |
| RunPod, H100 SXM 80 GB | $3.99 an hour | Every hour the server runs |
| RunPod, A100 80 GB | $1.79 an hour | Every hour the server runs |
| CoreWeave, HGX H100 server | $49.24 an hour for 8 GPUs, about $6.16 per GPU | Every hour the server runs |
Two worked examples, both our arithmetic on the published prices:
- Support replies on a Bedrock model. 10,000 replies a day at about 1,500 input tokens and 300 output tokens each is 15 million input and 3 million output tokens. On Qwen3 32B that is about $5.34 a day, or about $162 a month. No GPU server comes close at that volume.
- A fine-tuned 70B model. Imported into Bedrock, 8 Custom Model Units cost about $27.45 an hour while the model is active. At 16-bit precision a 70B model's weights take about 140 GB, so self-hosting needs at least two 80 GB GPUs: two RunPod H100s cost $7.98 an hour, or about $5,825 a month running all the time. Bedrock costs the same after about 212 active hours a month, roughly 7 hours a day. Below that, Bedrock is cheaper. Above it, the GPUs are.
Neither example includes the engineering time to run the servers, which falls almost entirely on the self-hosted side. What a private LLM costs to run covers storage, monitoring and the other costs beyond the model.
What does each option take to run?
On Bedrock, AWS runs the model servers. The brand sets retention per Region, which at launch has no console screen and is set through the API, connects its VPC through PrivateLink, controls keys in AWS KMS and can send request logs to its own S3 bucket or CloudWatch. The work is configuration and access control, which an AWS-literate team can own.
Self-hosting means running an inference server such as vLLM, which offers an OpenAI-compatible API and supports more than 200 model architectures. Someone has to provision GPUs, load and update models, scale for peak traffic, patch the servers and watch latency and errors. That is a standing engineering job, not a setup task.
Which should an ecommerce brand choose?
Start on Bedrock with an open model, retention set to "none" and a private connection from the brand's VPC. It suits support replies, segmentation and product data work at typical DTC volumes, and it bills nothing when idle. Use Custom Model Import once the brand fine-tunes a model on its own data, as long as the model is busy for less than about 7 hours a day. Move to self-hosted GPUs when the model runs most of the day, or when the brand needs a model or configuration Bedrock does not offer. For workflows that carry customer records, avoid models that require AWS review if 30 days of retention is not acceptable.
Brand Media Works, an AI consulting and marketing data engineering firm for ecommerce brands, builds private LLMs on open-weight models on Bedrock or on GPU instances in the brand's own AWS account, with the retention, access and logging controls covered by AI data security on AWS. Private AI for business covers when a marketing team needs a private model at all, and private AI vs public AI compares it with ChatGPT-style tools.
Frequently asked questions
Does Amazon Bedrock share prompts with the model provider?
No. AWS says model providers have no access to the accounts where Bedrock runs their models, so they cannot see Bedrock logs, prompts or completions. Where a provider requires human review, as Anthropic does for its Fable models, AWS performs it within AWS.
Does Bedrock keep my prompts?
It depends on the retention mode and the model. With the mode set to "none", AWS writes no request or response data to durable storage. Models that require AWS review, currently Anthropic's Fable 5 and 5.1, keep prompts and completions for up to 30 days.
Can I fine-tune a model on Bedrock with my own customer data?
Yes. AWS says a tuned model is based on a private copy of the base model. Fine-tuning Qwen3 32B is listed at $80 per training hour, plus $1.95 a month to store the result.