OpenVINO Model Server

A self-hosted inference server for serving AI models that have been optimized with OpenVINO.

FreeOpen sourceLinuxSelf-hosted
Official websitegithub.com

These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.

About OpenVINO Model Server

OpenVINO Model Server (OVMS) is a scalable inference server for running models optimized with the OpenVINO toolkit. You run it yourself and send inference requests to it from other applications, so a model can be served to many clients from one place.

The repository includes Dockerfiles for Ubuntu and Red Hat based images, a client directory, demos and documentation, plus a script to install it as a service. It suits teams deploying models on their own hardware who already use or plan to use OpenVINO.

Strengths

  • Serves OpenVINO-optimized models at scale
  • Container images for Ubuntu and Red Hat
  • Demos and client examples included

Limitations

  • Models need to be in or converted to OpenVINO-supported formats
  • Aimed at server deployment rather than casual desktop use

Details

Pricing
FreeFree and open source.
License
Apache-2.0
Developer
Intel
Platforms
Linux, Self-hosted
How it runs
Self-hosted
Account
Not required
Works offline
Yes
Best suited for
Serving OpenVINO models from your own servers
Last verified
Added

Alternatives to OpenVINO Model Server

Compare all

Software that can replace OpenVINO Model Server for an important use case, and what changes if you switch.

  • NVIDIA Triton Inference Server

    An open-source inference server for deploying machine learning models from many frameworks on GPUs and CPUs.

    FreeOpen sourceLinuxSelf-hosted

    NVIDIA Triton Inference Server serves models from many ML frameworks on GPUs and CPUs, so models need no conversion to OpenVINO formats, with Docker-based deployment.

  • KServe

    A Kubernetes-native platform for serving predictive and generative AI models at scale.

    FreeOpen sourceSelf-hosted

    KServe provides a Kubernetes-native serving layer for predictive and generative models across frameworks, but requires a Kubernetes cluster and the skills to run one.

  • vLLM

    High-throughput, memory-efficient inference and serving engine for LLMs.

    FreeOpen sourceLinuxCommand line

    vLLM serves language models with high throughput on Linux GPUs rather than OpenVINO-optimized models, and also uses the Apache-2.0 license.

  • LocalAI

    Serve language, image and speech models on your own hardware.

    FreeOpen sourceLinuxmacOSSelf-hosted

    LocalAI serves language, image and speech models behind APIs compatible with common AI services on Linux or macOS, without the OpenVINO format requirement, under MIT.

  • Xinference

    An open-source inference server for running language, speech and multimodal models through one API.

    FreeOpen sourcemacOSLinuxSelf-hostedCommand line

    Xinference serves language, speech and multimodal models behind one GPT-style API from a laptop up to servers, without requiring OpenVINO-converted models.

  • Sonar

    A self-hosted LLM inference server, formerly Aphrodite Engine, that serves Hugging Face models through OpenAI-compatible APIs.

    FreeOpen sourcemacOSLinuxSelf-hostedCommand line

    Sonar serves Hugging Face language models through OpenAI-compatible APIs on NVIDIA, AMD, Intel, CPU and Apple hardware, under the AGPL-3.0 license.

Similar software

Related functionality, not necessarily a direct replacement.

Report a wrong fact or a dead link on this listing