LLMs in business: where to start

Language models often reached the office before they reached the IT roadmap: some people use them to draft an email, others to summarise a contract. In Italy, according to Istat, the national statistics institute (“Enterprises and ICT” survey, December 2025), the share of companies with at least 10 employees using at least one artificial intelligence technology rose from 8.2% in 2024 to 16.4% in 2025, and the obstacle cited most often is a lack of skills.

For decision makers, the question is no longer “does it work?” but “where do I start, without wasting budget or putting data at risk?”. This article sets out what needs to be clear before you begin.

What an LLM is, in plain words

An LLM (large language model) is a program trained on huge amounts of text to predict, one small piece at a time, the most plausible continuation of whatever it receives. Out of that mechanism come some very useful abilities: answering questions, summarising, translating, extracting information, writing code. The best-known names are OpenAI’s models, Anthropic’s Claude, Google’s Gemini and, among the models you can run yourself, Meta’s Llama.

Two things to be clear about. An LLM does not “know” things the way a database does: it generates plausible text, usually correct but not always. And it knows nothing about your company’s documents unless you supply them with the request.

Where it really helps a business

The use cases that work best share three traits: lots of text, reasonably clear rules and a person who checks the result.

• Support and internal assistants: draft replies for the help desk, an assistant that answers colleagues’ questions on procedures and policies.
• Drafting and summarising: minutes, proposals, summaries of contracts or long email threads, always as a draft for someone to review.
• Data extraction: reading invoices, orders, CVs or forms and turning their fields into structured data, ready for your management system.
• Search over company knowledge: plain-language questions over manuals, wikis and archives, with the answer and its sources. This is the territory of RAG (retrieval-augmented generation), which we cover in a separate article.
• Help with software development: code completion, tests, documentation, always with a developer’s review and automated tests.

Start small: one process, one measure, one pilot

The most common mistake is to start from a vague goal such as “bringing AI into the company”. The opposite route works better:

1. Pick one process: frequent, repetitive, text-heavy, with a cost you already know (time per case, response times, errors).
2. Measure the starting point: without a “before” there is no “after”. Two or three indicators are enough.
3. Run a pilot: a few people, real but controlled data, a fixed duration. Record the cases where the model gets it wrong, too.
4. Decide, then scale: if the indicators improve, integrate with your existing systems (access control, logs, monitoring) and extend to other departments; if not, you have spent little to find out.

How the costs work

• Pay-per-use cloud services: you pay by the token, the small chunks of text (words or parts of words) the model reads and writes, at prices that vary by model and are often different for input and output. No upfront investment, but spending grows with usage.
• Models on your own servers: you pay for the hardware (usually servers with GPUs, the processors that speed up these calculations), the energy and the people who run it. Costs are more predictable, but you need an initial investment and operational skills.

We don’t quote figures because price lists change often: the right estimate is the one based on the volumes measured in the pilot, with a spending cap from day one. Comparing the two routes is the subject of our article on running models in-house or in the cloud.

Privacy and GDPR: where the data goes

Every piece of text sent to a cloud model leaves your perimeter, and it needs the same governance as any supplier that processes data on your behalf.

• Which data: classify what goes into the model. Personal data, health data or trade secrets need more care, from pseudonymisation to a ban on sending them at all.
• The contract: you need a data processing agreement (DPA, under Article 28 of the GDPR) and you should read the terms on model training. Anthropic, for example, states that it does not train its models on data from its commercial products, the API included; Google says the same for the paid Gemini API, but not for the free tier.
• Retention: check how long prompts and responses stay on the provider’s servers and whether zero-retention options exist.
• Location: choose European regions where you can. Since February 2025 OpenAI has offered European data residency to eligible API customers; Claude on Amazon Bedrock and Gemini on Google Cloud Vertex AI can be used with processing inside the EU. For the most sensitive data, a model installed on your own premises remains the safest route.

As for the EU AI Act, its transparency obligations (such as telling people when they are dealing with an AI) apply from 2 August 2026, while the Digital Omnibus, in force since 27 July 2026, pushes the rules on high-risk systems back to 2027 and 2028. The picture is still moving, so check it at the start of every project.

The risks: hallucinations and human review

The main risk is called hallucination: the model produces false information in the same confident tone as true information. It can be reduced, not eliminated. The safeguards we recommend:

• make the model work from your documents, not from its memory, and always ask for sources;
• keep a person in the loop for anything that reaches customers or has real consequences (payments, contracts, decisions about people);
• build a set of real questions with expected answers and rerun it whenever the model or the instructions change;
• treat external text (emails, web pages, attachments) with care, as it may hide instructions designed to manipulate the model (prompt injection).

If you are working out where to start, we can help you choose the first process, set up and measure the pilot, and integrate it with your systems. You can find what we do on our services page; to talk it through, get in touch.