Run a dataset through one or more models, score the outputs with one or more evaluators, and review the results.
Before you start
You need:
- A dataset.
- An evaluator that dataset can support.
- A model to generate the outputs that evaluators score against the dataset.
Create an examination
Create Examination
On the Examinations page, click Create Examination.
Configure Examination
-
Enter a Name. It is required.
-
Add a Description with notes on the run’s scope and how to interpret it.
-
Choose a Dataset. The list shows each dataset’s sample count.
-
Set Sample Selection as a percentage of the dataset or a number of samples. The control shows how many of the total samples will be used.
-
Choose Generation Models to produce the outputs to score. These are the models you want to evaluate for your use case.
-
Check the evaluators you want in the Evaluators table, which lists the ones your dataset can support.
- The four columns show the inputs each evaluator needs. A dash means it does not need that input, and a red ✕ means the dataset does not supply one it needs, which disables the evaluator.
- Choosing a generation model supplies MO, which can enable more evaluators.
- For each LLM-as-a-Judge evaluator, expand its row and choose one or more Evaluator Models to judge with. It cannot run without at least one.
-
Click Next.
Review & Submit
Check the Name, Description, Dataset, Sample Selection, and the evaluators you picked, then click Create Examination.
Manage your examinations
The Examinations list shows one row per run, with Name, Description, Models, Datasets, Status, Visibility, Created, and Actions.
Above the table, filter by Name or Status, reorder the list with Sort By, and limit it to your own runs with Only mine.
Each row’s Actions column opens the examination, and offers a delete action for runs you created.
Track an examination’s progress
An examination is queued while it waits to run, in progress while it runs, then completed or failed.
If it fails, check that the models you selected are active, that the dataset supplies the fields your evaluators need, and that each LLM-as-a-Judge evaluator has a judge model. Then create the examination again.
Read the results
Open a completed examination. Its header reports the models tested, the evaluators used, and the total number of evaluations, and the results are split across four tabs.
A bar chart of each model’s score, one evaluator at a time, chosen with Filter by Evaluator.
Sample Responses
One row per input, with a count of the unique inputs scored. Expand a row to see the responses for that input and how many models and judges scored it.
Interactive Analysis
Choose a Chart Type: scatter across two evaluators, a heatmap of model against evaluator, a box plot of each model’s distribution, parallel coordinates, or a judge agreement matrix. Filter Models narrows any of them to specific models.
Agentic Assessment
A cross-evaluator summary for runs that involve tools, with an Overview sub-tab and one sub-tab per agent-relevant evaluator.
Overview opens with Exam Agentic Facts: how many inputs carried tool definitions, how many carried expected tool calls, how many outputs invoked a tool, the number of distinct tools, and the tool names with their call counts. Below it, a composite score per model and a table of each evaluator’s score alongside that composite.
Each evaluator’s sub-tab shows its rubric, a per-model chart, and a table of scores by judge model. Last modified on August 17, 2026
