Anyone bringing a language model (LLM) into a company soon reaches a fork in the road: use the cloud services of the big providers, such as OpenAI, Anthropic’s Claude or Google’s Gemini, or download an open-weight model and run it on your own servers.
There is no answer that suits everyone, and we are wary of anyone who offers one. There are, however, clear criteria for deciding case by case, and the most robust solution often combines both routes. In this article we go through them: confidentiality, quality, cost, response times, maintenance and dependence on a single vendor.
The two routes
Cloud models. You use them through an API, an interface your applications call over the internet, and you pay for what you use. Hardware, updates and capacity are the provider’s job. You can buy them directly from the model makers and, for many models, through the major cloud platforms such as AWS, Azure and Google Cloud as well.
Open-weight models. The weights, the parameters that make up the trained model, can be downloaded and run wherever you like. Examples include Meta’s Llama and the many models published on Hugging Face; in August 2025 OpenAI also released open-weight models, gpt-oss, under the Apache 2.0 licence. “Open weights” does not always mean open source, though: Llama’s licence, for instance, sets its own conditions and should be read before any commercial use. To try them on a laptop or a test server, a tool like Ollama is enough; in production you use engines built to serve many users at once, such as vLLM.
Confidentiality and compliance
This is often the deciding factor. With a model in-house, data never leaves your infrastructure: it is the natural choice for health or legal data and trade secrets, or when a contract or internal policy forbids sending them to third parties.
That does not rule out the cloud on principle. With a data processing agreement (DPA, under Article 28 of the GDPR), the provider’s commitment not to use your data for training and processing in European regions, many use cases can be handled within the rules. Two examples: Anthropic states that it does not train its models on data from its commercial products, the API included, and since February 2025 OpenAI has offered European data residency to eligible API customers. The terms still need reading and checking case by case, together with whoever handles privacy in your company.
Quality, response times and maintenance
- Quality on your task: the largest models tend to do best on complex work such as long chains of reasoning, code or lengthy documents; for narrow tasks (classifying, extracting fields, summarising) a mid-sized open model may be enough. Public leaderboards change every few months: the test that counts is the one run on your data and your questions.
- Response times (latency): in the cloud they depend on the network and the service’s load, in-house on your hardware. A small model on a nearby server can be very fast; a large one on undersized hardware will not be.
- Maintenance: in the cloud, updates and capacity are the provider’s concern, but models are retired with notice and their replacement has to be tested again. In-house, it is all on you: updates, security, monitoring, managing GPUs and load peaks.
Costs and vendor lock-in
The cost structures are opposites. The cloud is pay-per-use, based on the tokens (words or parts of words) processed: no upfront investment and spending in line with usage, which suits getting started and low or variable volumes. In-house, the cost is mostly fixed: GPU servers, bought or rented, energy and the people who run them. It pays off when volumes are high and steady, or when confidentiality demands it anyway. We don’t quote figures: they depend on the model, the volumes and the hardware, and they change often. A serious comparison is based on numbers measured in a pilot.
Then there is vendor lock-in: an application built around one service’s specific features becomes expensive to move. Open models reduce that risk, but do not remove it.
Managed cloud services remain the simplest choice when they do exactly what is needed. For Artika, the theatre streaming platform we designed and built on AWS in the Ireland region, we used managed AI services: Amazon Transcribe detects the spoken language and writes the subtitles, and Amazon Rekognition checks videos before publication, including with a tailor-made Custom Labels model. They are not language models, but the reasoning behind the choice is the same: no AI infrastructure to run and data kept in a European region.
The hybrid approach
In practice, the route we suggest evaluating first uses both options, each where it works best:
- sensitive data stays on a model running in-house;
- everything else goes to the cloud model that best fits on quality or cost;
- an abstraction layer, an internal gateway or a library such as LangChain, gives applications a single interface and decides where to route each request;
- routing rules are explicit (by data type, department, cost) and every call is logged.
This layer also makes it easier to switch model or provider. Many tools, Ollama included, offer an interface compatible with OpenAI’s: often you only need to change the address and the model name in the configuration, although compatibility is never complete and always needs testing.
If you are deciding where to run your models, we can help you compare the options on your own data and volumes, and build the layer that leaves you free to change. Get in touch to talk it through.
