What is a small language model?
An SLM (small language model) is a language model with far fewer parameters than a large language model, which makes it cheaper, faster and able to run on your own infrastructure. It is used for narrow, well-defined tasks such as classification, extraction, routing and short summaries, not for open-ended conversation on any topic. On the right task it can come close to a large model at a much lower cost.
Definition
An SLM (small language model) is a language model with far fewer parameters than a large language model, which makes it cheaper, faster and able to run on your own infrastructure.
How does an SLM work?
A small language model uses the same architecture as a large language model, with far fewer parameters. Microsoft Phi, Google Gemma and the smaller Mistral models are well-known examples. Fewer parameters mean less memory and less compute. The model can therefore run on a single server, a laptop or directly on a device.
The trade-off is general ability. A small model has less broad knowledge and is weaker at multi-step reasoning. On a narrow task, fine-tuning it for that task can close much of the gap.
When should you choose an SLM?
When the task is narrow and repetitive, volume is high and response time matters. Sorting incoming email into categories, extracting set fields from a document and routing a request to the right team are typical jobs. A common setup lets the small model handle most requests and passes the hard ones to a large model. This lowers the cost per token while keeping quality.
What to watch for
The first reason teams in regulated sectors look at SLMs is data. A small model can run on your own servers or in a data centre in the country you choose, so data does not leave your environment. This simplifies international transfer questions under GDPR and, for data processed in Türkiye, KVKK, Türkiye’s data protection law. Make the compliance assessment together with your legal counsel.
Self-hosting also moves responsibility to you. Servers, updates, monitoring and security become your team’s work. Compare that operating effort with the per-request cost of a hosted model before you decide.
Further reading
How to choose an LLM for your businessCriteria, access routes, data retention, open models and a test method for choosing a language model for business use.
KVKK and generative AI: a checklistFrom staff chat tools to API integrations: a step-by-step KVKK checklist for generative AI, with the differences from GDPR.
What drives the cost of a RAG project?One-off and running cost drivers of a RAG knowledge assistant, the choices that raise or lower them, and what to prepare before a quote.Related terms
- LLM (Large Language Model)An LLM (large language model) is an AI model trained on very large amounts of text that answers questions, summarises, translates and writes by predicting what text should come next.
- Fine-tuningFine-tuning is the process of further training a pre-trained AI model on your own examples so that it performs a specific task, tone or output format better.
- Token (AI)A token is the unit of text, often smaller than a word, that a language model reads and writes in and that usage is billed by.
- Generative AIGenerative AI is a type of artificial intelligence that creates new content such as text, images, audio, video or code, based on patterns it has learned from large datasets.
- GDPR (General Data Protection Regulation)GDPR (General Data Protection Regulation) is Regulation (EU) 2016/679, the European Union law that sets the rules for processing personal data and the rights of individuals.
Frequently asked questions
Where is the line between an SLM and an LLM?
There is no fixed line. The distinction is about use more than parameter count: models that can run on a single server, a laptop or a device, and are chosen for narrow tasks, are usually called SLMs.
Are small models good enough in languages other than English?
It depends on the task. They can do well on narrow tasks such as classification and field extraction. For long, fluent text in languages such as Turkish, larger models are usually ahead. Decide with a test set built from your own data.
Do you need a GPU server to run an SLM?
Not always. Compressed versions of small models can run on a strong CPU or a single GPU. For high volume and low latency, a GPU server or a managed cloud service is a better fit.
Let’s talk about your transformation
Tell us which work repeats every week. We will tell you honestly whether it is worth automating.
How we start
- A free first callWe pin down the problem and the goal together.
- A 2-week discovery sprintWe look at the data, the process and the tools, and decide where to start.
- A written proposalScope, architecture and timeline, in writing.