Compare software

lm-evaluation-harness vs Inspect: catalog facts
lm-evaluation-harness	Inspect
Free Free and open source.	Free Free and open source.
Windows, macOS, Linux, Command line	Web, Command line
Hundreds of benchmark tasks in one tool New tasks defined through YAML templates	Over 200 pre-built evaluations ready to run Sandboxed execution in Docker, Kubernetes and other systems
Command-line and Python only, no graphical interface Running large benchmarks needs substantial compute	Writing new evaluations requires Python Aimed at researchers rather than casual users
Downloadable app	Downloadable app
Open source	Open source
License: MIT	License not stated
No account needed	No account needed
Works offline	Not stated if it works offline
Official website Full listing and sources Checked September 26, 2026	Official website Full listing and sources Checked October 2, 2026

Catalog facts only. Anything “not stated” is unconfirmed. Check full listings for details.