Hardware
What machine your office needs
to run its own AI, and what decides it
Most offices need less machine than they expect for Agents and records, and more than they expect if they want a model trained on their own material.
Less than you expect, for most of what an office actually wants. Running Agents and answering from your own records needs a small, quiet machine with no graphics card in it. Training a model on your own material is the thing that needs a bigger machine.
The confusion comes from the hobbyist advice online, which is written by people running the largest model their desk can hold, for fun. A business usually wants a smaller model doing one job reliably, all day, for a team. That is a different machine, and the thing that decides it is which of two jobs you are buying it for.
Two jobs, two machines
Answering from your records and running Agents is one job. Training a model on your own material is another. The first needs a small machine. The second needs a graphics card.
Memory decides more than speed
The model has to fit into the machine's memory while it runs. That is the number which decides what you can run, more than how fast the processor is.
It has to survive a normal week
A business machine runs unattended, restarts itself after a power cut and keeps a backup. A fast machine that needs somebody to restart it is not useful in an office.
You may already own one
A spare desktop or an existing office server can often do the first job. Part of the audit is checking that before anybody buys anything new.
You buy it, in your name
I specify the machine and you purchase it. It sits in your office, on your network, and it is yours. See what else goes on it.
What the machine is actually being asked to do
Before any specification, be clear which of these you want, because they are different purchases. One is a model reading your own documents and records and answering questions from them, with Agents doing small jobs on a schedule or when somebody messages. The other is taking your own material and changing the model itself, so that it has absorbed how your company writes and works.
Most businesses want the first and have been told they want the second. If that is the question you are stuck on, the comparison is on its own page.
The small machine, with no graphics card
This is what most offices of ten to a hundred people end up with. It is a compact computer with no screen attached, sitting on a shelf or in a cupboard with a network cable in it. It runs a smaller open model on the processor and the memory alone, holds a copy of the records it is allowed to read, and runs the Agents.
It is quiet, it draws little power, and it needs no server room and no air conditioning. It answers at reading speed rather than instantly, which for a report or an Agent reply nobody notices. What it does need is enough memory to hold the model while it works, a solid-state disk big enough for your records, and a wired connection.
- Agents in WhatsApp, a team chat, email or a sheet
- Answering questions from your own documents and records
- Writing reports, summaries and handover notes
- Attendance, leave, tasks and the dashboards above them
- Running overnight, unattended, and restarting itself
The bigger machine, with a graphics card
A graphics card is in this machine for one reason: doing a great deal of the same arithmetic at once. You need it when you are training a model on your own material, and when a lot of people need instant answers from a large model at the same moment.
This machine is physically bigger, noisier, hotter and hungrier for power. It needs somewhere it can take in air, a power supply that will not trip, and ideally a cupboard nobody sits beside. The card's own memory becomes the limit that decides what you can train, and that is the specification worth arguing about rather than the processor.
- Training or fine-tuning a model on your own material
- A large model answering many people at once, instantly
- Working on images, drawings or video as well as text
- A company big enough that the machine is doing several of these together
What decides which one you buy
Read down this table honestly and the answer is usually obvious before anybody quotes you anything.
| What you want it to do | Small machine, no graphics card | Bigger machine, with a graphics card |
|---|---|---|
| Agents inside the apps your team already uses | Yes | Not needed for this alone |
| Answering questions from your own records and documents | Yes | Only if many people need instant answers at once |
| Reports, summaries and the daily writing | Yes | Not needed |
| Training a model on your own material | No | Yes, this is the reason it exists |
| Working on drawings, photographs or video | Limited | Yes |
| A whole company asking it things at the same moment | It will queue | Yes |
| Running quietly on a shelf, unattended | Yes | It needs space, power and cooling |
What else the room needs
The machine is the part people think about. These are the parts that decide whether it is still running a year later.
- Power protection, so a cut does not corrupt whatever it was writing
- A wired network point, because this is not a machine to put on office wifi
- A backup that leaves the machine, so a dead disk is an inconvenience
- Somewhere it can take in air, away from where people sit
- One named person in your office who knows it exists and who to call
- Remote access for whoever maintains it, switched on deliberately rather than by default
What you do not need
A surprising amount of the advice you will be given is for a problem you do not have.
- A rack, a server room or a dedicated air conditioner, for the small machine
- The largest model you can find, which is usually slower and no better at your actual job
- A fully air-gapped setup, meaning no network connection at all, unless your work genuinely requires it
- A machine sized for the whole company on the first day, before one team has used it
- Several machines, when one of them would be idle most of the day
Can you use a computer you already own?
Often, yes, for the first job and the first team. A reasonably recent desktop with enough memory will run a smaller model and the Agents well enough to find out whether the system is worth buying hardware for. That is usually how I would start, because it turns a purchase into a decision you make afterwards.
What an existing machine is bad at is being somebody's desk computer at the same time. Once the system matters to the business it needs its own machine, switched on all the time, that nobody turns off on the way home. The audit says which of those you are at. What the audit looks at.
Built, not described
Questions people ask
What hardware do I need to run AI locally?
For Agents and answering from your own records, a compact machine with no graphics card, enough memory to hold the model and a solid-state disk. For training a model on your own material, a bigger machine with a graphics card. The audit settles which of the two you need, and the machine is bought in your name.
Can I run a local AI server without a GPU?
Yes, for most of what an office wants. A smaller open model runs on the processor and the memory, which is enough for Agents, reports and answering from your own documents. It answers at reading speed. A graphics card is for training and for many people at once.
How much memory does the machine need?
Enough to hold the model you are running while it works, plus room for the records. That is the number which decides what you can run, and it is set by the model you choose rather than by the size of your team. I specify it against the job.
Can we use a computer we already own?
Often yes, to start with one team and find out whether the system earns its place. What does not work is sharing somebody's desk computer once the business depends on it. At that point it needs its own machine that stays switched on.
Does it need a server room?
The small machine does not. It is a small box, it draws little power, and it lives on a shelf or in a cupboard with a network cable. A machine with a graphics card needs more space, more power and somewhere it can take in air.
What happens during a power cut?
With power protection in front of it, it shuts down cleanly or rides out the cut, and it starts itself again when power returns. Mine has done that through Mumbai's power cuts. Anything that needs the internet waits until the line is back.
Who fixes it if the machine fails?
The hardware is bought in your name, so it carries its own warranty and a local supplier can replace parts. The system is backed up off the machine, so a replacement is restored rather than rebuilt. If you want me on call for that, it is quoted separately.
Next step