llama-swap
A proxy server that loads and hot-swaps local AI model servers on demand behind one API.
These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.
About llama-swap
llama-swap sits in front of local inference servers such as llama.cpp, vLLM, stable-diffusion.cpp and ComfyUI, and starts or swaps the right one when a request names a model. Clients talk to a single OpenAI- and Anthropic-compatible endpoint covering chat, completions, embeddings, speech, transcription and image generation.
It is written in Go and ships as one binary with one YAML configuration file and no external dependencies. It suits people running several local models on limited GPU memory who want them available through one address.
Strengths
- One binary and one config file with no dependencies
- Works with any OpenAI or Anthropic compatible backend
- Covers text, audio, image and rerank endpoints
- Actively developed
Limitations
- Requires writing a YAML configuration by hand
- Does not run models itself; you still need the backend servers
Details
- Pricing
- FreeFree and open source under the MIT licence.
- License
- MIT
- Developer
- mostlygeek
- Platforms
- Self-hosted, Command line
- How it runs
- Downloadable app, Self-hosted
- Account
- Not required
- Works offline
- Yes
- Best suited for
- Running several local LLM and image servers behind a single API on one machine
- Categories
- Local AI tools, AI developer tools
- Last verified
- Added
Alternatives to llama-swap
Compare allSoftware that can replace llama-swap for an important use case, and what changes if you switch.
LocalAI
Serve language, image and speech models on your own hardware.
LocalAI is an MIT server that runs language, image and speech models itself with compatible APIs, instead of proxying separate backend servers you configure by hand.
Xinference
An open-source inference server for running language, speech and multimodal models through one API.
Xinference serves language, speech and multimodal models itself behind one GPT-compatible API, removing the need for separate backend servers, and scales from laptop to servers.
Similar software
Related functionality, not necessarily a direct replacement.
llama.cpp
The C++ engine most local AI apps are built on, running language models on ordinary hardware.
Ollama
The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.
vLLM
High-throughput, memory-efficient inference and serving engine for LLMs.
TabbyAPI
A lightweight, OpenAI-compatible API server for running ExLlama-format language models on your own hardware.
SGLang
A serving framework for running large language models and multimodal models on your own GPUs.
GPUStack
An open-source GPU cluster manager for serving AI models with vLLM and SGLang on your own hardware.
