Evaluation Methodology and Evaluating AI Systems
What we discussed on the recording
The third episode combines methodology with practical AI-system evaluation. Outputs are probabilistic, many phrasings are acceptable, and quality depends on the product. Public benchmarks support rough comparison but suffer contamination and poor task fit, so control must move into the team's own pipeline.
Entropy describes uncertainty, cross-entropy divergence from an expected distribution, KL divergence between distributions, and perplexity confidence in a sequence. A strong language metric does not guarantee a useful product: correctness, latency, cost, and safety need separate evaluation.
Open-ended text calls for embeddings and semantic similarity. AI as a judge scales review but inherits model bias. Pairwise comparison is easier than an absolute scale, although it needs checks for transitivity, option order, repeated voting, and result stability.
The process starts with criteria, model selection, and independent checks. Fact checking relies on trusted sources; labeling needs instructions and human agreement. A human baseline is a comparison point. The final pipeline tests components separately and end to end, connects technical scores to business metrics, and combines AI judges, people, and labeled data.