Part IX — AI Quality Engineering

Part IX develops the ability to evaluate AI-enabled systems as systems with uncertain, contextual, and sometimes probabilistic behaviour.

Chapters

1. Chapter 1 — AI Quality Engineering: Behaviour, Evidence, and Boundaries

   150 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Parts I and III; Part VIII evidence concepts recommended

2. Chapter 2 — AI System Architecture and Failure Boundaries

   150 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapter 1; Parts III, IV, VI, and VIII

3. Chapter 3 — Evaluation Data, Oracles, and Experimental Design

   180 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–2; Parts III and VI

4. Chapter 4 — Classification, Ranking, and Predictive-Model Evaluation

   190 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapter 3 and proportionate quantitative reasoning

5. Chapter 5 — Generative AI Evaluation: Rubrics, Factuality, and Instruction Following

   170 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–4

6. Chapter 6 — Robustness, Metamorphic Testing, and Adversarial Inputs

   160 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–5

7. Chapter 7 — Retrieval-Augmented Generation Quality

   170 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–6; Part IV API boundaries and Part VI data-quality foundations

8. Chapter 8 — Human Evaluation and Model-Based Evaluators

   175 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–7

9. Chapter 9 — Tool-Using and Agentic AI Systems

   180 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

   Prerequisites: Chapters 1–8 and Part IV API Quality Engineering

10. Chapter 10 — Safety, Fairness, Privacy, and Responsible Quality Boundaries

    180 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

    Prerequisites: Chapters 1–9; Parts IV and VI

11. Chapter 11 — AI Regression, Production Learning, and Change

    175 minutes, plus the practical exercise · Status: Draft · Version: 0.1.0

    Prerequisites: Chapters 1–10 and Part VIII observability foundations

12. Chapter 12 — Capstone: AI Quality Strategy and Evaluation Portfolio

    240 minutes, plus the capstone exercise · Status: Draft · Version: 0.1.0

    Prerequisites: Chapters 1–11