llama.cpp
The C++ engine most local AI apps are built on, running language models on ordinary hardware.
These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.
1 more ways to get llama.cpp
Package managers
- Snap Store
snap install llama-cpp
About llama.cpp
llama.cpp runs language and vision models with no Python stack and no dependencies, in plain C and C++. Quantisation from eight bits down to one and a half lets a model that would need a data centre card fit into the memory of a laptop, at some cost in quality.
Apple Silicon is a first-class target through Metal and Accelerate, x86 uses AVX through AVX-512 and AMX, and there are CUDA, Vulkan, SYCL and HIP backends for graphics cards. A model larger than your video memory can be split between GPU and CPU. Ollama, LM Studio and Jan all build on it.
It ships a command line tool and a server with a built-in web interface, so it is usable directly as well as being a library.
Strengths
- No Python and no dependencies; one binary runs the model
- Quantisation down to 1.5 bits fits big models into small memory
- CUDA, Vulkan, SYCL, HIP and Metal backends, plus CPU and GPU hybrid inference
- Built-in server with a web interface, so it is usable on its own
Limitations
- Command line first; the web interface is basic next to dedicated apps
- Quantising hard reduces answer quality, and the trade-off is yours to judge
- Release numbering is a build counter, so version history is hard to read
- You find and download the model files yourself
Details
- Pricing
- Free · Free and open source under the MIT licence.
- License
- MIT
- Developer
- ggml.ai
- Platforms
- Windows, macOS, Linux, Android, Command line
- How it runs
- Downloadable app
- Account
- Not required
- Works offline
- Yes
- Best suited for
- Running a language model on hardware you already own, or building on top of one
- Categories
- Local AI tools
- Last verified
- Added
- Provenance
- Selected from the TechWalrus downloads catalog (AI); facts checked against the developer's own pages, 2 sources on file.
Similar software
Related functionality, not necessarily a direct replacement.
Ollama
The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.
LM Studio
A polished desktop app for downloading, running and chatting with local language models, with an OpenAI-compatible server built in.
Jan
An open-source ChatGPT replacement that runs entirely offline, and can also front cloud providers when you want them.
GPT4All
A local AI chat app with document chat built in, from Nomic AI. Stable, MIT licensed, but not updated since early 2025.
