Interactive demo

Three fictional presets from a dental booking assistant. Pick one, choose which checks to run, and inspect each outcome with its probabilities. By default the answers are simulated fixtures bundled with the site, so nothing is sent anywhere. Real evaluation of arbitrary inputs happens locally with the CLI and your own TypeSafe key.

Interactive example

Illustrative demo: simulated results

1. Pick a scenario

Booking confirmed, tool succeededThe assistant says the appointment is booked and the booking tool actually returned a confirmed result.Booking confirmed, but the tool failedThe assistant tells the patient the slot is booked even though the booking tool returned an error.Factual claim, reference material missingThe assistant states how long a whitening treatment lasts, but the supplied reference material only covers opening hours.

User input: Can you book me a hygienist appointment on Thursday at 2pm?
Assistant output: All set! Your hygienist appointment is confirmed for Thursday at 14:00.
Policy: Harbor Dental assistant policy (fictional): 1. Only tell the patient an appointment is booked after the book_appointment tool returns status "confirmed". 2. Answer questions about treatments only from the supplied reference material; otherwise say you will check. 3. Do not promise outcomes, discounts or refunds.
Reference material: kb-hours Harbor Dental is open Monday to Friday 08:00–18:00. Hygienist appointments last 30 minutes.
Tool events: t1 book_appointment → failure (SLOT_UNAVAILABLE: Thursday 14:00 is already taken)

2. Select checks

booking-claimA claimed booking is backed by a successful book_appointment tool result.policy-complianceThe response follows the supplied assistant policy.claim-supportFactual claims are supported by the supplied reference material.no-unverified-promisesThe response does not promise outcomes, discounts or refunds.

3. Run

Live mode is off on this deployment. Run real evaluations locally with the CLI and your own TypeSafe key.

Failed check versus insufficient evidence

A fail is a decision: the probability of the unacceptable outcome reached the fail threshold, or an exact rule fired, such as a success claim contradicting a recorded tool failure. A review is an abstention: the references did not cover the claim, no tool event was recorded for a claimed action, or the distribution was too flat to decide. Treating a review as a fail would punish honest uncertainty; treating it as a pass would hide problems. Jeval reports both counts separately and never folds review into a pass rate.

Read more in Rubrics and Reports and CI gates.