Samples
A sample is one real API input, sent to the job’s endpoint with a teach envelope:
Checks come in two forms. Expression checks assert on the response directly, like a status code or a field value. Judged checks describe the expected outcome in plain language and the platform grades the response against it. Judged checks are useful when the exact shape of a correct response varies.
Scenarios
As samples run, the platform groups them into scenarios. Samples that exercise the same path through the job land in the same scenario, and the scenario gets named for the behavior it covers: “Creates a standard invoice”, “Rejects duplicate customer”, “Handles missing due date”. The grouping is based on what actually happened during the run, not on how similar the inputs look. Two inputs that read very differently but drive the job down the same path belong to the same scenario. An input that triggers a new path creates a new scenario. You can also group samples yourself. Name a scenario when you teach, or add samples to a scenario from the job page. Samples in a scenario you created yourself are never regrouped automatically.
Why scenarios matter
Scenarios are the job’s coverage map. The job page shows how many scenarios are passing at a glance, and the stability score counts your samples toward readiness. When you wonder whether a job is ready for production, the question is concrete: which behaviors have scenarios, and are they all green?Teaching is defining what correct means up front, except the definition is real API calls from your product. Edge cases you teach today are failures you never see in production.
Adding edge cases
The fastest way to harden a job is to teach the inputs you are worried about:- The record that already exists.
- The field that is sometimes missing.
- The value the target system rejects.
- The request that should be refused entirely.