Skip to main content

Evalm8

Evalm8 is Divyam's evaluation product. You describe what a good answer looks like in a rubric, attach judges that score each part of it, and run that eval over real traces from your LLM application. Evalm8 versions every rubric, judge, eval and dataset, so you can trace each score back to the versions that produced it and roll back when you need to.

Automated scores are only useful if they agree with people. Evalm8 lets your reviewers label a sample of traces, then uses those labels to refine the judges. Each cycle brings the automated scores closer to human judgment, so you can trust them on the traffic your reviewers have no time to read.

Invite only

Access to Evalm8 is by invitation, so email hello@divyam.ai to request one. Evalm8 has its own sign-in, separate from the Divyam console.

The core loop​

1Raw dataset2Evaluate3Golden dataset4Refine evalrepeat until the scores match your reviewers
  1. Raw dataset: pull traces from a connected source.
  2. Evaluate: score them with your eval to get a baseline.
  3. Golden dataset: reviewers annotate a sample of traces.
  4. Refine eval: Evalm8 tunes the judges against those annotations, and you approve the result.

Guides​

  • Core Concepts: traces, rubrics, judges, pipelines, evals, and how Evalm8 computes scores.
  • Workflows: the step-by-step path from raw traces to refined judges.
  • Building Evals: bind judges to rubric dimensions, with configuration and the default pipeline.
  • Creating Judges: define LLM and deterministic judges and their inputs.
  • Authoring Pipelines: the step reference for extracting trace data and feeding it to judges.