exo
Connects your own computers into a cluster for running large AI models locally.
These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.
About exo
exo pools the memory and compute of several devices so you can run models too large for any one machine. Devices running exo find each other automatically, and exo splits the model across them based on each device's resources and the network links between them. It supports tensor parallelism and RDMA over Thunderbolt 5.
It uses MLX as its inference back end and offers APIs compatible with OpenAI, Claude and Ollama clients. A built-in dashboard manages the cluster and provides a chat interface. Custom models load from the Hugging Face hub.
Strengths
- Runs models larger than a single device's memory
- Automatic device discovery with no manual configuration
- OpenAI, Claude and Ollama compatible APIs
- Built-in dashboard with chat
Limitations
- Built around MLX, so it is centred on Apple hardware
- Benefits most from several high-memory machines
Details
- Pricing
- FreeFree and open source under the Apache 2.0 licence.
- License
- Apache-2.0
- Developer
- exo labs
- Platforms
- macOS
- How it runs
- Downloadable app
- Best suited for
- Running very large open models across several Macs at home
- Categories
- Local AI tools, AI developer tools
- Last verified
- Added
- Sources
Alternatives to exo
Compare allSoftware that can replace exo for an important use case, and what changes if you switch.
GPUStack
An open-source GPU cluster manager for serving AI models with vLLM and SGLang on your own hardware.
GPUStack also pools hardware across machines, but it targets GPU servers with vLLM and SGLang, needing administration skills rather than automatic discovery of Macs.
Similar software
Related functionality, not necessarily a direct replacement.
MLX LM
A Python package and command-line tool for running and fine-tuning language models on Apple silicon.
llama.cpp
The C++ engine most local AI apps are built on, running language models on ordinary hardware.
Ollama
The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.
LM Studio
A polished desktop app for downloading, running and chatting with local language models, with an OpenAI-compatible server built in.
vLLM
High-throughput, memory-efficient inference and serving engine for LLMs.
LocalAI
Serve language, image and speech models on your own hardware.
