Best local LLMs by memory (November 2026)
Popular open-weight models in the Ollama library, with download sizes for the default 4-bit builds. Leave several GB free for the operating system and the model’s working memory.
| Your machine’s memory | Models that fit comfortably | Download sizes |
|---|---|---|
| 16GB | llama3.1:8b, qwen3:8b | 4.9GB, 5.2GB |
| 24GB | gemma3:12b, qwen3:14b | 8.2GB, 9.3GB |
| 32GB | gpt-oss:20b, gemma3:27b (tight) | 14GB, 17GB |
| 48GB | qwen3:30b, qwen3:32b | 19GB, 20GB |
| 64GB | llama3.1:70b | 43GB |
Sizes from ollama.com/library, November 13, 2026. Which models fit is our guidance; longer documents need more headroom.
Tools to run a local LLM
| Tool | What it is | Good for |
|---|---|---|
| Ollama | Model server with a command line and local API | Running models as an always-on service |
| LM Studio | Desktop app with a model browser | Trying models on one computer |
| MLX | Apple’s machine-learning framework for Apple silicon | Running models efficiently on Macs |
| Open WebUI | Self-hosted chat interface | Giving staff a ChatGPT-style window |
| AnythingLLM | Desktop and server app for chatting with documents | Document Q&A |
| vLLM | High-throughput inference server | Serving many users on GPU servers |
What businesses use local LLMs for
Drafting
Letters and replies from your templates.
Summaries
Long PDFs and transcripts condensed.
Document search
Questions answered from your own folders, with sources.
Inbox triage
Sorting mail and drafting replies for approval.
DIY or managed?
| Do it yourself | Managed box (AdminOnsite) | |
|---|---|---|
| Hardware | Buy it | Included; yours after 24 months |
| Setup | Install model server, chat app, document index, permissions | Configured on your templates and folders |
| Updates and model changes | You | Us, remotely, with logged access |
| Email and file connections | You build them | Set up for drafting, triage and follow-ups |
| Cost | Hardware plus your time | $249–$699 a month plus setup |
DIY makes sense if you have someone technical with time to maintain it. If you don’t, the time is the real cost.
Questions
What is the best local LLM?
The best one that fits your memory with room to spare. On 24–32GB, models in the 12–20-billion-parameter range are a good balance; on 64GB, 70-billion-parameter models at 4-bit fit.
Is a local LLM free?
Many open-weight models are free to download under their licenses. You pay in hardware and the time to run them.
Can a local LLM read my documents?
Yes, through retrieval: your folders are indexed, and the relevant passages are handed to the model with your question.
How do I run an LLM locally?
Install a model server such as Ollama or LM Studio, download a model that fits your memory, and add a chat interface such as Open WebUI.