Choose a private LLM partner by the answers it gives to eight questions: where the model runs, what you own, which controls are live before launch, how quality is measured, what data it needs, what it costs to run, how the work is phased and who keeps it running. Brand Media Works builds private LLMs for ecommerce brands, so this checklist applies to us too. The answers below draw on what private LLM partners publish about their own work, collected in the Brand Media Works competitor register.
What should an ecommerce brand ask a private LLM partner?
The table sets out each question, why it matters when the model will see customer data, and what a good answer looks like. Private AI for business covers when a brand needs a private model in the first place.
| Question | Why it matters | What a good answer looks like |
|---|---|---|
| Where will the model run, and does any request leave our account? | Customer records go wherever the model runs | A named deployment (the brand's own cloud account, a private network or an air-gapped setup) and a list of every outside service a request can reach. Some products sold as private AI run on the provider's own servers by default and send questions to public models, with the brand's own servers offered only as an add-on |
| Who owns the code, prompts, test sets and trained weights? | Ownership decides whether you can switch partners | A contract line saying the brand owns the code, the prompts, the evaluation sets and any model weights trained on its data |
| Which controls are live before staff use it? | Access mistakes expose customer data from day one | Single sign-on, document-level permissions and audit logging in place before the system goes to the wider team |
| How will you measure quality, and on what test set? | Accuracy claims cannot be compared without one | A test set built from the brand's own questions, with a pass mark agreed before the build. Claims that domain-specific models reach 90% accuracy against 60% for general models mean little without one |
| How much of our data do you need, and in what form? | Data preparation is often the longest step | An estimate tied to the task. Published guidance for fine-tuning ranges from 1,000 examples for simple tasks to 100,000 for complex ones |
| What will it cost to run each month after launch? | A fixed server bill and a per-token bill behave differently | A monthly running-cost model, compared with the brand's current AI API spend, produced during discovery |
| How is the work phased, and what do we commit to? | Phases limit what you pay before you see results | A short discovery, typically about two weeks, then builds quoted one phase at a time. A first working private workspace can be ready in about four weeks |
| Who keeps it running after handover? | Models, connectors and permissions need ongoing care | Ask for a named owner, a runbook and the monitoring the brand will see |
The first question matters most. A product sold as private can still send each question to a public model provider, or hold customer data on the partner's servers rather than the brand's. Neither is wrong for every use, but a brand putting order histories or customer profiles through the model should know which one it is buying. On-premise or private-cloud LLM explains the deployment options behind the answer.
Why ask for controls and test sets before launch?
Because governance is the step most often skipped. EY's September 2026 survey of senior AI decision-makers found that controls are routinely bypassed under time pressure:
About half (47%) say their organization has previously bypassed its AI governance process for urgent deployments.
The same survey found that 41% say senior leaders do not have visibility into all AI tools operating in their organization. A partner who puts single sign-on, permissions and audit logs in place before the first user logs in removes that risk from the project rather than leaving it to a later phase.
Test sets matter for the same reason. A published accuracy figure says nothing about how a model will answer your customers' questions about your products. Ask the partner to build an evaluation set from your own support tickets and product questions, agree a pass mark before the build, and keep the set when the project ends.
Which answers should make a brand walk away?
- No clear location for the model, or a refusal to list the outside services a request can reach.
- The partner keeps the trained weights or the prompts, so leaving means starting again.
- Accuracy numbers with no test set, or a test set the brand cannot run itself.
- No running-cost estimate before the build is signed.
- One large contract for everything instead of a scoped first phase.
What should an ecommerce brand add for its own customer data?
Ask which systems the partner will connect, such as Shopify orders, Klaviyo profiles and support tickets; where copies of that data will sit; whether personal data is removed before it reaches the model; and how a customer's deletion request reaches every copy. The risks of using public AI tools with customer data sets out what can go wrong, and AI data security on AWS covers the redaction, access control and logging a brand should expect. For the build itself, see private LLMs on open-weight models.
Frequently asked questions
How long does a private LLM project take?
It depends on scope. A focused first system usually reaches production in eight to twelve weeks. Fine-tuning an existing model takes 2 to 4 weeks of training against 3 to 6 months to train one from scratch, and a full fine-tuning engagement, including dataset creation and evaluation, typically runs 3 to 4 months.
Should the partner host the model, or should we?
If the model will see customer records, hosting it in the brand's own cloud account keeps those records under the brand's control and contracts. Partner hosting can suit lower-risk work, as long as the brand knows where the data sits and which outside models are called.
Should we own the model weights?
Yes, if the partner trains or fine-tunes a model on your data. Owning the weights, prompts and evaluation sets means the brand can move the model to another partner or run it itself.