GPUStack
An open-source GPU cluster manager for serving AI models with vLLM and SGLang on your own hardware.
These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.
About GPUStack
GPUStack manages a cluster of GPUs and uses it to serve AI models through inference engines such as vLLM and SGLang. It can also provide on-demand GPU instances that users reach over SSH.
You run it yourself. The repository includes Docker Compose files and a Helm chart for deployment. It suits teams and individuals with several GPU machines who want to share them for model serving without handing the models to a cloud service.
Strengths
- Pools GPUs across machines for model serving
- Supports vLLM and SGLang inference engines
- On-demand GPU instances accessible over SSH
- Deployment files for Docker Compose and Kubernetes Helm
Limitations
- Aimed at GPU servers and clusters rather than a single laptop
- Needs server administration skills to set up
Details
- Pricing
- FreeFree and open source.
- License
- Open source, license not stated
- Developer
- The GPUStack contributors
- Platforms
- Self-hosted
- How it runs
- Self-hosted
- Best suited for
- Teams that want to serve AI models across their own pool of GPU machines
- Categories
- Local AI tools, Developer tools
- Last verified
- Added
- Provenance
- Facts checked against the developer's own pages and store listings, 1 sources on file.
Alternatives to GPUStack
Compare allSoftware that can replace GPUStack for an important use case, and what changes if you switch.
Xinference
An open-source inference server for running language, speech and multimodal models through one API.
Xinference serves language, speech and multimodal models through one GPT-compatible API, running on a laptop or scaling to servers.
LocalAI
Serve language, image and speech models on your own hardware.
LocalAI is an MIT-licensed self-hosted server with compatible APIs for language, image and speech models, aimed at single hosts rather than GPU pools.
vLLM
High-throughput, memory-efficient inference and serving engine for LLMs.
vLLM is an Apache-2.0 high-throughput serving engine on Linux, which GPUStack builds on, without GPUStack's cluster pooling and SSH-accessible instances.
SGLang
A serving framework for running large language models and multimodal models on your own GPUs.
SGLang is an open-source high-throughput serving framework for language and multimodal models, used directly rather than through GPUStack's cluster manager.
Similar software
Related functionality, not necessarily a direct replacement.
nvitop
Interactive terminal viewer for NVIDIA GPUs and the processes running on them.
TabbyAPI
A lightweight, OpenAI-compatible API server for running ExLlama-format language models on your own hardware.
Open WebUI
A self-hosted chat interface that talks to Ollama and any OpenAI-compatible API, so your local models get a proper front end.
RamaLama
Command-line tool that pulls AI models from any source and serves them locally in containers.
LibreChat
Self-hosted open-source chat interface that brings many AI models and providers into one place.
llama.cpp
The C++ engine most local AI apps are built on, running language models on ordinary hardware.
