Evaluation Methodology and Evaluating AI Systems

The third episode combines the chapters on evaluation methodology and the practical evaluation of AI systems.

The discussion covers benchmark limitations, entropy, cross-entropy, KL divergence, perplexity, semantic similarity, and functional correctness.

It then assembles a validation workflow around AI as a judge, pairwise comparisons, fact checking, trusted sources, labelling, and a human baseline.