A UK SME can run an open-weight model, Gemma, Mistral, Llama, Qwen and others, on a server or workstation it controls, so data never leaves the building and there is no per-token bill. It is the right answer in a narrow set of cases: a firm whose policy or regulator forbids data leaving its premises, a high-volume repetitive task where the small-model quality is enough and the token bill would otherwise be large, or a product where inference cost is the margin. For most SMEs it is the wrong answer, because a rented frontier model on a business plan or the API is better at the hard tasks, costs pence per document at SME volumes, needs no hardware and nobody to run it, and is covered by a data processing agreement that answers the questionnaire. The licences differ: Mistral’s main models and Gemma 4 are Apache 2.0; Llama uses Meta’s community licence with its own conditions; read the actual terms before building a product on any of them.
- Buys you
- Data on premises; no per-token bill
- Costs you
- Hardware, a person, and quality below the frontier
- Right for
- Policy constraints, very high volume, product margin
What an open-weight model is
A model whose trained weights are published for download, so anyone can run it on their own hardware rather than through the developer’s API. Meta’s Llama, Google’s Gemma, Mistral’s models, Alibaba’s Qwen, DeepSeek and Microsoft’s Phi are the main families. “Open source” is used loosely; several are released under custom licences rather than open-source ones, which matters for commercial use. Tools such as Ollama make running a small model on a workstation a ten-minute job, which is why the question now reaches small firms.
What it buys, and what it costs
Side by side, the trade is control against convenience and quality. The row that decides it for most firms is set-up.
| Run your own | Rent (business plan or API) | |
|---|---|---|
| Where data goes | Stays on your hardware | To the provider, under a DPA, with the region as offered |
| Per-unit cost | None; electricity and hardware amortised | Pence per document on a small model; more on frontier |
| Quality | Small and mid models; below the frontier on hard tasks | Frontier available when needed |
| Set-up | Hardware, a person who can run it, monitoring, updates | An account and a policy |
| Questionnaire answer | “On premises”, which some clients love and some want attestations for | The provider’s attestations and DPA |
| Vendor change | You control when the model changes | The provider retires models on its schedule |
RelatedWhat AI features cost to run Which Claude model for which job
The licences, in one paragraph each
Mistral publishes much of its lineup under Apache 2.0, which permits commercial use without conditions beyond attribution. Google’s Gemma 4, released in 2026, moved to Apache 2.0; earlier Gemma releases carried Google’s own terms with a prohibited-use policy. Meta’s Llama uses its community licence, which permits commercial use with conditions, including an obligation to seek a separate licence above a very large user threshold and restrictions on using the outputs to improve competing models. DeepSeek and Phi are MIT-licensed. The practical rule: read the licence of the exact version you download, keep a copy with the runbook, and have it checked before you sell a product built on it.
The hardware, honestly
A small model of a few billion parameters runs on a modern workstation with a capable graphics card or a recent Apple machine, fast enough for a single user or a low-volume background task. A mid-sized model that competes on quality for real business tasks needs a server-class graphics card or several, which is a capital cost in the thousands to tens of thousands of pounds, plus power, cooling, backups and a person who can keep it running and update it. That person is the cost firms forget. Compare it against the token bill you would actually pay: at SME document volumes on a small rented model, that bill is often pounds a month, and the hardware never pays back.
RelatedAI document processing for a UK SME What an AI agent costs to build and run
When it is the right answer
Three cases. A policy or regulatory constraint that data must not leave the premises, where the hyperscaler routes to a rented model in a UK region do not satisfy the rule. A repetitive, high-volume task, classification, extraction, tagging, where a small model’s quality passes the test and the rented bill would be significant; here a local model for the volume and a rented frontier model for the exceptions is a sound design. And a product where inference cost is the margin, which is a software business decision rather than an SME automation one. Outside those, rent.
RelatedAI data residency for a UK business A DPIA for AI, with a template
A middle path
Open-weight models can be rented too. Several cloud providers host Mistral, Llama and Gemma models in UK and EU regions on a per-token basis under their standard agreements, which gives the licence freedom and the regional control without the hardware or the person. For a firm that wants a specific open model in a specific region, that is usually the better route, and it can be swapped for a self-hosted deployment later if volume justifies it.
RelatedAI vendor due diligence checklist Custom AI solutions
Common questions
Can a small business run its own AI model?
Yes. Open-weight models from Mistral, Google, Meta and others run on hardware you control, and tools such as Ollama make a small one a ten-minute set-up. Whether it is worth it is a different question, and for most SMEs the answer is no.
Is a local AI model more secure?
Data stays on your hardware, which answers one risk. It creates others: you run the security, backups, updates and access controls that a provider would, and prompt injection applies to any model. Some clients prefer on-premises; others want the provider’s attestations.
Which open-source AI models can be used commercially?
Mistral’s Apache 2.0 models and Gemma 4 without conditions beyond attribution; DeepSeek and Phi under MIT; Llama under Meta’s community licence with its conditions. Read the licence of the exact version and keep it with the runbook.
What hardware do I need to run an AI model locally?
A small model: a modern workstation with a capable graphics card or a recent Apple machine. A mid-sized model for real business quality: server-class graphics hardware costing thousands to tens of thousands of pounds, plus a person to run it.
Is running my own model cheaper than paying per token?
Rarely at SME volumes. A small rented model costs pence per document and the bill is often pounds a month; the hardware and the person never pay back. It can be cheaper at very high volume or when inference cost is a product’s margin.
Can I use an open model without buying hardware?
Yes. Cloud providers host Mistral, Llama and Gemma in UK and EU regions per token under standard agreements, which gives the licence and regional control without the hardware.