Local AI for your business: what hardware does your own LLM need?

That depends on the model, the task and the tools used. A local model can be enough for document analysis, internal search or drafting texts. Demanding tasks may need other models or more computing power. Compare quality, failure cases and response time using representative documents; a blanket ranking of local versus cloud does not help.

The most common reason is data protection: customer data, personnel data or documents under professional secrecy should not pass through a third party's cloud. Add to that independence from providers' price and product changes, and predictable costs, because there are no fees per request after the purchase. If none of these requirements apply, a cloud model is usually the simpler choice.

No. Smaller models also run from main memory, on ordinary computers via the CPU and particularly well on a Mac, whose memory is shared with the graphics unit. Then RAM counts instead of graphics memory, but the answers come considerably more slowly. For productive use with several users there is rarely a way around a graphics card with enough memory.

The licence of the specific model and variant decides. Qwen3 14B, for example, is under Apache 2.0, Llama 3.3 under its own community licence. FLUX.1 dev has a non-commercial licence; commercial use of the model cannot be inferred from the free download. Running the model and using generated images must be checked separately.

A local model is not a locally installed ChatGPT. Examples of downloadable models are Qwen3 and Llama, for instance with Ollama as the runtime. Which tasks succeed has to be tested with the specific model. Locally installed tools can also use cloud functions; what counts is the data path actually set up. More on our page about the local ChatGPT alternative (in German).

To start with models of up to 8 billion parameters, a card with 8 GB of graphics memory is enough (e.g. RTX 4060 class). 12 to 16 GB is comfortable for most SME tasks, 24 GB for larger models and agents. A Mac uses its shared memory; there, 16 GB is the entry point and 32 GB is comfortable. Our buying guide by VRAM class (in German) goes deeper into the choice of hardware.

For small models (up to 8 billion parameters) usually yes: they run from main memory even without a dedicated graphics card, noticeably more slowly. MacBooks with 16 GB of memory or more are surprisingly capable here. For productive continuous use or larger models there is hardly a way around a computer with a proper graphics card or a rented GPU server. Our Mac buying guide (in German) sorts out the memory classes.

That depends on the model architecture, quantisation, context length, hardware and simultaneous use. The calculator estimates memory, not response speed. Measure the time to the first answer and the processing time on your own task before buying hardware.

Usually that is the wrong question: what most businesses want is not retraining of the model (fine-tuning) but internal search. The model gets access to your documents when answering and quotes from them without being changed. That is cheaper, more up to date and can be undone at any time. Real fine-tuning only pays off in special cases, such as a fixed writing style or technical jargon in large volume.

The model families listed here support multilingual tasks. Whether summaries, document fields and Swiss technical terms are processed correctly enough has to be checked with your own texts. Answers that read convincingly can be factually wrong; fixed review criteria remain necessary.

Downloaded models can work offline with a local runtime set up for this. Cloud models, web search, external interfaces and some add-on functions still need a connection. In Ollama, cloud functions can be switched off. Also check the interface used, extensions and outgoing connections.

Local operation alone does not prove data protection compliance. It can avoid transfers to model providers if the user interface, connections to other systems, remote access and backups are set up accordingly. Data protection law still applies to the processing of personal data. Data paths, access protection and the requirements of the specific use must be checked in the project.