Alternatives to llama-swap
A proxy server that loads and hot-swaps local AI model servers on demand behind one API. The listings below can replace it for an important use case. Each note says what changes if you switch.
The original
llama-swap
A proxy server that loads and hot-swaps local AI model servers on demand behind one API.
Replacements
Listings that take over the same core job as llama-swap.
LocalAI
Serve language, image and speech models on your own hardware.
LocalAI is an MIT server that runs language, image and speech models itself with compatible APIs, instead of proxying separate backend servers you configure by hand.
Xinference
An open-source inference server for running language, speech and multimodal models through one API.
Xinference serves language, speech and multimodal models itself behind one GPT-compatible API, removing the need for separate backend servers, and scales from laptop to servers.
Similar software
Related functionality, not a direct replacement.
llama.cpp
The C++ engine most local AI apps are built on, running language models on ordinary hardware.
Ollama
The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.
vLLM
High-throughput, memory-efficient inference and serving engine for LLMs.
TabbyAPI
A lightweight, OpenAI-compatible API server for running ExLlama-format language models on your own hardware.
SGLang
A serving framework for running large language models and multimodal models on your own GPUs.
GPUStack
An open-source GPU cluster manager for serving AI models with vLLM and SGLang on your own hardware.
