07. Causal Inference for Generative AI

Generative AI
Causal Inference
Evaluation
Lecture Notes
Causal evaluation of prompts, model upgrades, RAG, hallucination reduction, judges, safety interventions, personalization, and GenAI product outcomes.
Published

May 3, 2026

This course specializes causal evaluation for generative AI systems, where outputs are stochastic, measurement is difficult, and human behavior often changes in response to generated content.

Notebook links open rendered HTML pages generated from the source notebooks under notebooks/lectures/. Code is visible by default; rendering is configured not to execute live notebook code, so local LLM or GPU-heavy cells are not triggered during website builds.

Notebook Sequence

  1. 01. Generative AI as a Causal System
  2. 02. Estimands for Generative AI Changes
  3. 03. Prompt Changes as Interventions
  4. 04. Model Choice and Model Upgrade Experiments
  5. 05. Decoding Parameters, Temperature, Top-p, and Output Variability
  6. 06. RAG, Context, and Grounding Interventions
  7. 07. Hallucination Reduction as a Causal Estimand
  8. 08. LLM Judges, Human Labels, and Measurement Error
  9. 09. Preference Data and Causal Interpretation
  10. 10. Safety Interventions and Policy Evaluation
  11. 11. User Trust, Reliance, and Overreliance
  12. 12. Human Creativity, Productivity, and Quality Effects
  13. 13. Code Generation and Developer Productivity
  14. 14. Synthetic Data Generation and Causal Validity
  15. 15. Multimodal Generative AI Evaluation
  16. 16. Personalization, Memory, and Long-Context Effects
  17. 17. Marketplace, Content, and Creator Ecosystem Effects
  18. 18. Interference, Contamination, and Feedback Loops
  19. 19. Observational Evaluation of Generative AI Rollouts
  20. 20. Cost, Latency, and Quality Tradeoffs
  21. 21. Monitoring, Regression Testing, and Eval Drift
  22. 22. Executive Readouts for Generative AI Experiments
  23. 23. Governance, Auditability, and Content Risk
  24. 24. Capstone: Evaluating a Generative AI System

How To Read This Track

  • Work through the notebooks in order if you want the full course arc.
  • Treat each notebook as a lecture plus lab: read the discussion, inspect the code, and rerun locally when you want to experiment.
  • For AI-heavy notebooks, expect some brittleness when live model calls are enabled; that instability is part of the course material rather than something hidden from the reader.

The .ipynb sources remain in the matching folder under notebooks/lectures/.