MLC LLM logo

MLC LLM

A compiler and inference engine for running large language models natively across GPUs, phones and browsers.

FreeOpen sourceWindowsmacOSLinuxCommand line
Official websitellm.mlc.aiDownload pagegithub.comSource codegithub.comDocumentationllm.mlc.ai

These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.

About MLC LLM

MLC LLM is a machine learning compiler and deployment engine for large language models. Its MLCEngine runs models on AMD, NVIDIA, Apple and Intel GPUs through Vulkan, CUDA, ROCm and Metal, in web browsers through WebGPU, and on iOS and Android devices.

The same engine is exposed as an OpenAI-compatible REST server and through Python, JavaScript, iOS and Android APIs. It suits developers who want to compile and serve models on their own hardware rather than end users looking for a chat app.

Strengths

  • Broad hardware support across Vulkan, CUDA, ROCm, Metal and WebGPU
  • OpenAI-compatible REST server
  • One engine for desktop, mobile and browser deployments

Limitations

  • Aimed at developers rather than casual users
  • Model compilation adds setup steps

Details

Pricing
FreeFree and open source under the Apache 2.0 licence.
License
Apache-2.0
Developer
The MLC AI contributors
Platforms
Windows, macOS, Linux, Command line
How it runs
Downloadable app
Account
Not required
Works offline
Yes
Best suited for
Developers deploying language models on local GPUs, mobile devices or in the browser
Last verified
Added

Alternatives to MLC LLM

Compare all

Software that can replace MLC LLM for an important use case, and what changes if you switch.

  • llama.cpp

    The C++ engine most local AI apps are built on, running language models on ordinary hardware.

    FreeOpen sourceWindowsmacOSLinuxAndroidCommand line

    llama.cpp runs models as a single dependency-free binary with CUDA, Vulkan, HIP and Metal backends and aggressive quantisation, with no compilation step, under the MIT license.

  • Ollama

    The simplest way to pull down an open language model and run it on your own machine, from one command or a desktop app.

    FreemiumOpen sourceWindowsmacOSLinuxCommand line

    Ollama trades MLC LLM's compiler approach for one-command model downloads and a desktop app, with a localhost REST API, but no browser or phone deployment.

  • mistral.rs

    A Rust inference engine for running text, vision and multimodal models locally with an OpenAI-compatible server.

    FreeOpen sourceSelf-hostedCommand line

    mistral.rs is a Rust engine loading GGUF models behind an OpenAI and Anthropic compatible server, aimed at local servers rather than mobile or browser deployment.

  • vLLM

    High-throughput, memory-efficient inference and serving engine for LLMs.

    FreeOpen sourceLinuxCommand line

    vLLM focuses on high-throughput serving on Linux GPUs instead of reaching phones and browsers, and also uses the Apache-2.0 license.

  • LocalAI

    Serve language, image and speech models on your own hardware.

    FreeOpen sourceLinuxmacOSSelf-hosted

    LocalAI serves language, image and speech models behind APIs for common AI services on Linux and macOS, without MLC LLM's compilation step or browser target.

  • LM Studio

    A polished desktop app for downloading, running and chatting with local language models, with an OpenAI-compatible server built in.

    FreemiumProprietaryWindowsmacOSLinux

    LM Studio replaces developer tooling with a desktop app built on llama.cpp and MLX with an OpenAI-compatible server, but it is closed source.

  • MLX LM

    A Python package and command-line tool for running and fine-tuning language models on Apple silicon.

    FreeProprietarymacOSCommand line

    MLX LM runs and fine-tunes models only on Apple silicon from the terminal, dropping MLC LLM's cross-GPU, mobile and browser reach.

  • Foundry Local

    Microsoft's tool for downloading and running AI models entirely on your own device.

    FreeProprietaryWindowsmacOSCommand line

    Foundry Local is Microsoft's tool for on-device models with hardware acceleration and app SDKs on Windows and macOS, but it is not open source.

Similar software

Related functionality, not necessarily a direct replacement.

Report a wrong fact or a dead link on this listing