# How to Evaluate LLM Outputs: Evals for Small Teams

> Build a small eval suite in your own repo: real cases from logs, pass or fail checks, and a run on every prompt or model change.

- Canonical: https://zeroraised.com/blog/how-to-evaluate-llm-outputs
- Published: 2026-10-28
- Author: ZeroRaised Team (https://zeroraised.com/blog/authors/zeroraised)
- Publisher: ZeroRaised (https://zeroraised.com/)
- Section: Growth
- Tags: artificial-intelligence, developer-tools, saas, bootstrapping

Evaluate LLM outputs with a small, task-specific test suite kept in your own repository: real cases from your logs, a clear pass or fail check, and a run on every prompt or model change. OpenAI's Evals platform becomes read-only for existing users on 31 October 2026 and is scheduled to shut down on 30 November 2026 ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)), so a testing setup that depends on it has an expiry date.

**TL;DR**

- Keep the dataset and scoring in your own repository: OpenAI's hosted Evals platform is being retired.
- Start with 30 to 50 real cases from logs and a pass or fail check for each.
- LLM-as-a-judge is useful but biased, so validate it against human labels and prefer pairwise or pass or fail scoring.
- Include typical, edge and adversarial cases, and add a case every time a real bug slips through.

**Disclosure:** ZeroRaised publishes this guide and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 28 Oct 2026; ZeroRaised has no commercial relationship with the companies named. The starter numbers marked as suggestions are ZeroRaised's own recommendation, not a sourced benchmark.

**An eval** is a structured test for measuring a model's performance on a defined task, combining a dataset of inputs, a scoring method and a threshold that decides whether a change is acceptable.

## Why do evals belong in your own repository?

OpenAI's own guidance defines evals as "structured tests for measuring a model's performance", on the same page that gives the retirement dates ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

That date is the argument for ownership. If your cases only exist in a vendor dashboard, a product decision elsewhere can remove your ability to check a prompt. Keep the cases as files, keep the scoring code in the repo, and treat a hosted tool as a convenience. Tests next to the prompt are also read by the people who change it.

## What are the five steps of a useful eval?

OpenAI's process is a good skeleton ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)):

1. **Define the objective.** Say what the system should do and what a bad output looks like. "Extract customer, product and requested action, and refuse duplicates" is testable; "summarise tickets" is not.
2. **Collect a dataset.** Use production, historical, synthetic and human-curated examples; real traffic is most valuable because it contains the mess.
3. **Define metrics.** Choose exact match, a field-level pass, a rubric score or a pairwise preference, whichever catches the failure you care about.
4. **Run and compare.** Run the old and new versions on the same set. A score without a comparison is a number, not a decision.
5. **Grow the set.** Re-run on every change and add the cases that broke.

OpenAI also recommends eval-driven development, task-specific evals, logging everything, automating scoring and calibrating scoring with human feedback ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)). None of that requires a platform.

## Why must evals be task-specific?

A generic "quality" score hides the thing you need to know. OpenAI lists overly generic metrics as an anti-pattern, along with biased datasets that do not match production traffic, vibe-based evals and ignoring human feedback ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

Task-specific means the check matches the job. For extraction, compare fields. For a support reply, check that policy is quoted correctly and no promise is invented. For an agent, test tool selection and the arguments it passes, because OpenAI notes agents need evals for both ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

For a multi-agent architecture, let evals drive it; starting there adds complexity you cannot yet measure ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

## How do you use an LLM as a judge without fooling yourself?

An LLM judge is cheaper and more scalable than human review, which makes it tempting to use alone ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)). It has known biases: position bias, where the order of two answers changes the verdict, and verbosity bias, where longer answers score higher regardless of quality ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

Three rules keep it honest:

- Prefer pairwise comparison or pass or fail scoring, which OpenAI recommends for reliability ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).
- Label 20 to 30 cases by hand and check how often the judge agrees with you. That is a ZeroRaised suggestion.
- Keep a written rubric, or the judge is just a second model with opinions.

## Which edge cases should every eval set include?

OpenAI recommends typical, edge and adversarial cases; the edge cases it names are a checklist: multilingual input, typos, very short requests, long conversations and conflicts with the system prompt ([OpenAI](https://developers.openai.com/api/docs/guides/evaluation-best-practices)).

Those are not exotic. A one-word request, a second-language customer and a user asking the assistant to ignore its instructions all happen in production. Clean examples measure the demo, not the product.

Adversarial cases deserve a separate folder. Prompt injection attempts, requests for restricted data and attempts to trigger an expensive tool call belong in the set. Our guide to [reviewing AI-generated code](https://zeroraised.com/blog/how-to-review-ai-generated-code) and the [defensible AI product guide](https://zeroraised.com/blog/how-to-build-a-defensible-ai-product) cover the surrounding controls.

## What does a starter eval suite look like?

This is a ZeroRaised recommendation, not a sourced benchmark:

1. **Pull 30 to 50 real inputs from logs.** Include the boring majority and the five failures you remember.
2. **Write a pass or fail check for each.** Where possible, check structure with code before asking a model to judge meaning.
3. **Record the expected output or the rule.**
4. **Run it on the current prompt and model to get a baseline.**
5. **Change one thing, run again, compare.** Prompt, model, temperature and retrieval changes each get a run.
6. **Add a case for every bug.** The set should grow from incidents, not from imagination.
7. **Keep it fast.** A suite that takes 40 minutes will be skipped. A suite that takes two minutes will run.

The model switch is the one small teams miss. When you change models for price reasons, run the evals before you commit. A cheaper model can be a good trade, but the [Gemini and Mistral price increases](https://zeroraised.com/blog/gemini-mistral-ai-price-increases) show how fast a cost assumption changes, and the [LLM cost guide](https://zeroraised.com/blog/how-to-reduce-llm-costs) shows where to find savings without guessing. The [LLM API selection guide](https://zeroraised.com/blog/how-to-choose-an-llm-api) covers the rest.

## What this means for bootstrapped founders

1. **Put the eval set in the repo this week**, starting with the one workflow that touches money, customers or compliance. A folder of JSON cases and a script that prints a pass rate is enough.
2. **Write the pass rule before the prompt.** If you cannot say what good looks like, you cannot test it.
3. **Automate the run in CI or a pre-release script.** A test you remember to run is a test you will skip.
4. **Log inputs and outputs, with a privacy review.** You cannot build a dataset from data you did not keep, so keep what you need and no secrets.

## Where ZeroRaised fits

ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have a product people can try, you can [submit it here](https://zeroraised.com/submit). The [newsletter](https://zeroraised.com/newsletter) carries new analysis, and the [pricing page](https://zeroraised.com/pricing) explains the listing options.

A launch page that says how you test AI features is a trust signal: quality as a process, not a demo.

## Sources

- [OpenAI: evaluation best practices](https://developers.openai.com/api/docs/guides/evaluation-best-practices)

## Frequently asked questions

### What is an eval?

An eval is a structured test for measuring a model's performance on a defined task. In practice it is a dataset of inputs, a way to score the outputs and a threshold that tells you whether a change made the system better or worse.

### Why should I move my evals out of OpenAI's platform?

OpenAI's Evals platform is read-only for existing users from 31 October 2026 and scheduled to shut down on 30 November 2026. If your tests only exist there, you lose the ability to run them. Keeping the dataset and the scoring code in your own repository means a platform retirement is not a testing outage.

### How many test cases do I need to start?

A useful starter is 30 to 50 real cases drawn from your logs, covering the common request types and the failures you already know about. That is a ZeroRaised suggestion, not a sourced benchmark. The number matters less than the cases being real and the scoring being consistent.

### Can I let an LLM grade the outputs?

Yes, with care. LLM-as-a-judge is cheaper and more scalable than human review, but it carries position bias and verbosity bias, so a longer answer can score better for no good reason. Prefer pairwise comparison or pass or fail scoring, and check the judge's agreement against a set of human labels.

### When should I run the evals?

On every prompt change, every model change, every retrieval or tool change, and before a release. Model switches made for price reasons deserve the same run as switches made for quality, because a cheaper model can pass a demo and fail your edge cases.