Evaluation Methodology and Evaluating AI Systems

Open-ended text cannot be evaluated like a classification label. Entropy describes data uncertainty, cross-entropy adds divergence between true and learned distributions, and perplexity indicates difficulty predicting the next token. These measures help model developers, but low perplexity does not prove application usefulness; exceptional confidence may signal benchmark leakage into training. After preference tuning, a language-model metric can worsen while answers become more useful to people.

Application evaluation starts with observable behaviour. A closed task can be checked for functional correctness, much like a unit test, yet a correct SQL query may still be slow and costly. Exact or lexical matching penalises valid paraphrases and inherits reference errors. Semantic similarity through embeddings captures meaning better but requires a consistent model. One score is rarely sufficient: content quality must be combined with latency, cost, safety, format, and scenario-specific constraints.

LLM-as-a-judge receives a request, response, rubric, and scale, then assigns a score. It cheaply scales checks of factual consistency, instruction-following, or style; a stronger model can review a cheaper model and regenerate failures. Keeping the judge's reasoning explains low scores and feeds criticism into another loop. Yet the evaluator changes with model versions, favours its own outputs, and reads ambiguous criteria inconsistently, so it needs calibration.

Pairwise comparison is often easier than an absolute score: a person or model can select the better answer more reliably than assign an objective rating. Leaderboards use this mechanism, although order, intransitive preferences, and query population can distort rankings. For important claims, Evgeny describes a stricter workflow: extract claims, find support and contradictions in trusted sources, weight sources, and give an expert direct links. PubMed outweighs casual discussion in medical content, while a medical-legal reviewer decides.

A team first defines criteria: domain knowledge, generation capability, instruction-following, cost, and latency. Hard constraints, such as keeping sensitive data inside the security boundary, remove some options immediately. Provider information and public benchmarks create a shortlist, but candidates require an internal gold set and meaningful rubrics. A human baseline shows whether performance is sufficient. Different models may lead on different subtasks, turning evaluation into routing policy rather than a contest for one universal winner.

After selection, every component and the end-to-end path need tests: extraction, retrieval, generation, validation, and the user outcome. The team writes guidelines, creates labelled or synthetic cases, and combines deterministic checks, model judges, and human review. A modest gold set can grow from production failures. The final layer is a business metric: if a support assistant usefully resolves half of requests, chasing a higher laboratory score may not pay. Online monitoring detects drift after changes to models, prompts, retrieval, or users.