guides
Guides
Practical, workflow-first guides on benchmarking models with orbench. No fluff, just the steps and the reasoning behind them.
How to evaluate an LLM on your own data
What a custom eval is, when a public benchmark is enough, and how to build a test set, pick a grader, and keep results comparable as models ship.
How to benchmark OpenRouter models on your own tasks
Why public leaderboards aren't enough, and the full workflow: your own tasks, candidate models, trials, rubric judging, and reading the results.
Cost per answered task: a better way to compare LLM pricing
Why cost-per-call is misleading, the formula for cost per answered task, a worked example, and how to measure it on your own workload.
How to build a benchmark CSV
The format orbench expects, how column names become prompt template placeholders, and how to design a task set that stays useful as new models ship.