Alternatives to TabbyAPI

A lightweight, OpenAI-compatible API server for running ExLlama-format language models on your own hardware. The listings below can replace it for an important use case. Each note says what changes if you switch.

Replacements

Listings that take over the same core job as TabbyAPI.

  • llama.cpp

    The C++ engine most local AI apps are built on, running language models on ordinary hardware.

    FreeOpen sourceWindowsmacOSLinuxAndroidCommand line

    llama.cpp runs GGUF-style quantised models as one binary with a built-in server and web interface, supporting many backends instead of ExLlama format only.

  • KoboldCpp

    A single executable that runs GGUF language models locally, with a web interface and no installation at all.

    FreeOpen sourceWindowsmacOSLinux

    KoboldCpp is a single executable for GGUF models that exposes OpenAI, Ollama, A1111 and ComfyUI compatible endpoints and includes its own web interface.

  • TextGen

    Chat with local language models using the TextGen desktop interface.

    FreeOpen sourceWindowsmacOSLinux

    TextGen is an AGPL-3.0 desktop interface that includes API access alongside a built-in chat interface, so no separate front end is required.

  • Ollama

    The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.

    FreemiumOpen sourceWindowsmacOSLinuxCommand line

    Ollama downloads and runs models with one command and serves a localhost REST API, removing ExLlama format limits with a desktop app option.

  • LocalAI

    Serve language, image and speech models on your own hardware.

    FreeOpen sourceLinuxmacOSSelf-hosted

    LocalAI is MIT-licensed and serves language, image and speech models with compatible APIs for common services, broader than TabbyAPI's language-model focus.

  • vLLM

    High-throughput, memory-efficient inference and serving engine for LLMs.

    FreeOpen sourceLinuxCommand line

    vLLM is an Apache-2.0 high-throughput serving engine for Linux, suited to production-scale deployments on appropriate GPU hardware rather than lightweight single-user use.

  • Xinference

    An open-source inference server for running language, speech and multimodal models through one API.

    FreeOpen sourcemacOSLinuxSelf-hostedCommand line

    Xinference serves language, speech and multimodal models behind one GPT-compatible API on macOS, Linux or servers, but has no Windows listing.

  • SGLang

    A serving framework for running large language models and multimodal models on your own GPUs.

    FreeOpen sourceSelf-hostedCommand line

    SGLang is a high-throughput serving framework for language and multimodal models on your own GPU servers, heavier than TabbyAPI's lightweight setup.

Similar software

Related functionality, not a direct replacement.