What can a private AI server do for an office?
The same everyday work people use ChatGPT for, on files you’d rather not upload:
First drafts
Letters, client replies and routine documents written from your own templates, for a person to review.
Summaries
Long contracts, transcripts, policies and PDFs condensed, with page references to check.
Ask your files
Plain-English questions answered from your shared drive, with the source documents cited.
Inbox help
Email sorted by urgency, with reply drafts waiting for approval.
What it won’t match is the newest cloud models on hard, open-ended reasoning or broad world knowledge. Open-weight models that fit office hardware are strong at drafting, summarizing and searching; they are weaker at the hardest analysis.
Which hardware: Mac mini, GPU tower or cloud VM?
Memory is the main constraint, because the whole model has to fit in it. Apple silicon shares one pool of memory between CPU and GPU, which is why a small Mac mini can hold models that would need a large graphics card on a PC.
| Option | Memory for models | Upfront cost | Noise and space | Still a cloud? |
|---|---|---|---|---|
| Mac mini with M6 | 16GB standard, configurable to 24GB or 32GB (153GB/s on 16GB, 170GB/s on 24GB and 32GB) | From $899; 24GB from $1,099; 32GB from $1,499 (Apple list) | Fits on a shelf; Apple describes the line as quiet and efficient | No |
| Mac mini with M5 Pro | 24GB standard, configurable to 48GB or 64GB (307GB/s bandwidth) | From $1,699; 64GB from $2,699 (Apple list) | Same footprint | No |
| PC or tower with a GPU | Set by the graphics card’s own memory (VRAM) | Depends on the GPU | Larger, louder and more power-hungry under load | No |
| GPU server rented in your cloud account | Whatever you rent | None; billed by the hour or month | None on site | Yes, your own account at a cloud provider |
Mac mini specs and starting prices from Apple’s August 25, 2026 announcement. Configured prices for higher memory are higher than the starting price.
How much memory do the models need?
Download sizes for the default 4-bit builds in the Ollama library. Leave room on top for macOS, other apps and the model’s working memory, which grows with the length of the documents you feed it.
| Model (Ollama tag) | Download size | Comfortable on |
|---|---|---|
| llama3.1:8b | 4.9GB | 16GB and up |
| qwen3:8b | 5.2GB | 16GB and up |
| gemma3:12b | 8.2GB | 24GB and up |
| qwen3:14b | 9.3GB | 24GB and up |
| gpt-oss:20b | 14GB | 32GB |
| gemma3:27b | 17GB | 32GB, tight; 48GB or more is easier |
| qwen3:32b | 20GB | 48GB or more |
| llama3.1:70b | 43GB | 64GB |
| gpt-oss:120b | 65GB | Doesn’t fit any Mac mini |
Sizes read from ollama.com/library on November 13, 2026. “Comfortable on” is our sizing guidance, not a vendor figure.
What does a private AI server cost per month?
Three ways to pay, compared with per-seat cloud AI:
| Approach | What you pay | What’s included | Who maintains it |
|---|---|---|---|
| Buy and run it yourself | Hardware up front, plus staff or IT time | Whatever you set up | You or your IT provider |
| Managed box on your shelf (AdminOnsite) | $249, $399 or $699 a month plus one-time setup ($299 to $499) | Mac mini, setup on your files, updates, monitoring and support; yours after 24 months | AdminOnsite, remotely, with logged access |
| Per-seat cloud AI (for comparison) | About $20–25 per user per month for standard business seats | The vendor’s models and apps; your data on their servers | The vendor |
US list prices from each vendor’s own page, checked November 13, 2026. Prices change; check the vendor before you buy.
Who keeps it running?
A private server is a computer, so it needs the care any office computer needs, plus a few AI-specific jobs:
- Operating system and security updatesPatched on a schedule, not when someone remembers.
- Model updatesNew open-weight models are tested before they replace the ones your templates were tuned on.
- BackupsSettings, templates and indexes backed up so a failed drive isn’t a lost month.
- PowerA small battery backup rides out short outages; the box catches up when power returns.
- MonitoringSomeone notices when it’s offline or out of disk before staff do.
- Remote accessUsed only for maintenance, and logged.
Security basics
- Don’t expose the model portOllama listens on 127.0.0.1, port 11434, by default. Keep it that way, or put it behind authentication on your office network only.
- Encrypt the diskTurn on FileVault on a Mac so a stolen box doesn’t mean stolen files.
- Use folder permissionsThe assistant should only see the folders each team is allowed to see.
- Log accessKeep a record of who connected remotely and when.
- Review outputsNothing goes to a client without a person reading it.
How to set one up
- Size the hardware to the modelsPick memory first, from the table above, based on the models and document lengths you need.
- Install a model server and a chat appOllama, LM Studio or Apple’s MLX run the models; Open WebUI or AnythingLLM give staff a chat window.
- Connect documents with permissionsPoint it at the shared folders it should search, team by team.
- Lock down network accessKeep the model server off the internet; use a VPN or managed remote access for support.
- Set an update and backup routinePatches, model updates and backups on a calendar.
- Agree the house rulesDrafts are reviewed before they go out; staff know what it’s for and what it isn’t.
When not to self-host
- You need the strongest available models for hard research or reasoning.
- Only one or two people will use it and per-seat cloud AI costs less.
- Nothing you’d put into it is confidential, and your clients don’t ask where data goes.
- Nobody, in-house or outside, will keep it patched and backed up.
Questions
Can I run ChatGPT on my own server?
No. ChatGPT is OpenAI’s hosted service. To keep prompts on your own hardware you run open-weight models, which several AI companies publish for anyone to download and run.
How much does a private AI server cost?
Hardware starts at $899 for a Mac mini with M6 at Apple’s list price, plus the time to set it up and maintain it. A managed AdminOnsite box costs $249 to $699 a month plus setup, including the hardware, setup, updates and support.
Is a Mac mini enough for an office?
For drafting, summaries, inbox triage and file search, yes, if it has enough memory for the models you need. A 24GB or 32GB Mac mini fits 8- to 14-billion-parameter models with room to work; 64GB fits 70-billion-parameter models at 4-bit.
Is a private AI server as good as ChatGPT?
For everyday office work it is very capable. It is not as strong as the newest cloud models at hard, open-ended reasoning, which is why drafts are reviewed by a person.
Does a private AI server need the internet?
The models run without it. Email, cloud file services and remote support need a connection, and work queues until it’s back.
Sources, checked November 13, 2026
- Apple: Mac mini technical specifications
- Apple Store: Buy Mac mini (US)
- Apple Newsroom: Mac mini with M6 and M5 Pro (Aug 25, 2026)
- Ollama model library (download sizes of default 4-bit builds)
- Ollama FAQ: network binding and keeping models loaded
- OpenAI Help Center: ChatGPT Business overview
- Claude Help Center: What is the Team plan?