Why bother with a fallback
- Outages happen. Every cloud service has bad days, and your work doesn't stop for them.
- Limits happen. Hit a rate limit mid-task and the job stalls.
- Your connection fails. The internet goes down more often than any AI provider does.
- Some work should stay home. A local model never sends your text anywhere.
What Ollama is
Ollama is a free, open-source tool for running large language models on your own computer. It runs on macOS, Windows and Linux, downloads models with one command, and serves them through a local API that other apps can talk to.
# download and chat with a model
ollama run <model-name>
# or just download it for later
ollama pull <model-name>By default the API listens on your own machine at port 11434, so apps on that computer can use the model the same way they'd use a cloud one.
What hardware you need
The short version: the more memory your graphics card has, the larger the model you can run comfortably. Small models run on a CPU alone, just slowly. Most models come in quantized versions, which trade a little quality for a lot less memory.
Start with a small model, see how it feels on your machine, and size up from there. Ollama's model library lists the variants for each one.
What local models are good at, and what they aren't
Local models handle the everyday work well: summarizing a document, drafting a reply, classifying a pile of emails, answering questions about a file, explaining an error message.
They're weaker at long, multi-step reasoning, large codebases and anything where the frontier cloud models have a real edge. That's why we treat local as the fallback and the private option.
Design the fallback to be honest
A silent fallback is worse than no fallback. If an answer quietly came from a smaller model, you'll trust it more than you should. A good fallback follows three rules:
- Try the cloud first for the work that needs it.
- Fall back automatically when the cloud can't be reached, so work doesn't stop.
- Say so, every time. The answer should tell you which model wrote it.
Add a fourth for sensitive work: send it to the local model on purpose, and never to the cloud.
How Syrebrum does it
Syrebrum supports local models through Ollama alongside Claude, OpenAI, Gemini, Grok and Llama through OpenRouter or Groq. If your cloud AI can't be reached, Sy answers with a local model and tells you it did. Want everything local all the time? Syrebrum can run in a mode that keeps everything on your network.
How offline fallback works in Syrebrum
Questions
Is Ollama free?
Yes. Ollama is free and open source. The models it runs have their own licences, so check the one for the model you choose.
Does a local model send my data anywhere?
No. A model running through Ollama processes your text on your own machine.
Do I need a graphics card?
It helps a lot. Small models can run on a CPU, but a graphics card with more memory runs larger models faster.