Compare software

Kiln vs lm-evaluation-harness: catalog facts
Kiln	lm-evaluation-harness
Free The app is free and the accompanying library is open source.	Free Free and open source.
Windows, macOS, Linux	Windows, macOS, Linux, Command line
Evals, fine-tuning and synthetic data in one app RAG, agents and MCP support	Hundreds of benchmark tasks in one tool New tasks defined through YAML templates
Licence of the desktop app is not stated in the research Broad scope can be more than a small project needs	Command-line and Python only, no graphical interface Running large benchmarks needs substantial compute
Downloadable app	Downloadable app
Closed source	Open source
License not stated	License: MIT
Not stated if an account is needed	No account needed
Not stated if it works offline	Works offline
Official website Full listing and sources Checked September 26, 2026	Official website Full listing and sources Checked September 26, 2026

Catalog facts only. Anything “not stated” is unconfirmed. Check full listings for details.