More companies want AI assistants that work with their own documents without sending data to the cloud. The deciding factor is not the processor brand but memory — how much of it the model can use. Here is how to match the computer to the task.
Why run AI locally
Data stays in the office: contracts, medical records or accounts never leave the company network. There is no fee per request, and the system keeps working without the internet. The trade-off is that you choose and maintain the hardware yourself.
Memory decides which model you can run
Language models are measured in billions of parameters. Compressed to 4 bits — the usual way to run them locally — a model needs a little more than half a gigabyte of memory per billion parameters, plus room for the conversation. Roughly: an 8-billion model needs about 6 GB, a 32-billion model about 20 GB and a 70-billion model about 40–45 GB.
What counts is memory that the graphics or the AI accelerator can use. In computers with unified memory the processor and the graphics share one pool, so a workstation with 128 GB can give up to 96 GB to the model.
Three classes of hardware
Single-board computers with an NPU of 1 to 6 TOPS and 8–16 GB of memory handle computer vision — counting, detection, reading text — and small language models. They suit training labs and devices that work on site.
Mini PCs with Intel Core Ultra or AMD Ryzen AI processors and 32–64 GB of memory run models of 8–14 billion parameters for a few users — for example, a document assistant for a small office.
Workstations with 128–192 GB of unified memory run models of up to about 70 billion parameters and serve a small team. Dozens of simultaneous users need a server with professional graphics cards — a different budget.
NPU or graphics?
An NPU is efficient for specific tasks, but not every model and program supports it. Popular tools for running language models locally, such as Ollama, LM Studio and llama.cpp, rely mainly on the processor and the graphics. Before buying, check that your software supports the exact chip — we ask the factory to confirm this in writing.
Checklist before you order
Which models and of what size; how many people will use them at the same time; how long the documents are; Windows or Linux; whether the factory should preinstall your system image; the warranty and how repairs work; power consumption and noise in an office.
How we help
Tell us which models you want to run and how many users you have. We suggest a configuration, ask the factory to confirm compatibility in writing and can have your system image installed before shipping.