Week 24 · Term Three · How machines learn, and how they fail
Catch a machine that learned the snow instead of the wolf, see how a 99% score can find nothing, read the study that split face-scanner errors by group, write a fair rule for an AI essay detector, and file a model card that tells the truth.
Monday finds a shortcut hiding behind a high score. Tuesday shows why accuracy can mislead when the thing you care about is rare. Wednesday compares error rates by group from a real 2018 audit. Thursday decides how a school should use an AI-writing detector. Friday writes the label every machine should carry.
25 minutes
AI · A machine learns whatever pattern its examples hold
Explain that a machine learns from patterns in its training examples, including accidental ones, and calculate how a shortcut rule can still score well.
Make
A shortcut report
Audit handoff: The shortcut report is filed. Tomorrow a machine scores 99% and still misses everything that matters.
25 minutes
Statistics · Accuracy can hide the thing you care about
Calculate accuracy as a percent, and explain why a high accuracy can hide a machine that misses every rare case.
Make
An accuracy check
Audit handoff: The accuracy check is filed. Tomorrow, one real audit split the score by group and found a gap of more than 40 times.
25 minutes
Ethics · Test the machine for every group
Explain how a single overall score can hide unfair errors for one group, compare error rates by group, and list the steps of a fair test.
Make
A group audit
Audit handoff: The group audit is filed. Tomorrow a machine is used to judge students' writing.
25 minutes
Ethics · A flag is a question, not a verdict
Weigh the tradeoffs of using an AI tool to make decisions about people, and decide where a human must check before anyone is affected.
Make
A detector policy
Audit handoff: The detector policy is filed. Tomorrow you write the label every machine should carry.
30 minutes
Writing · The label every machine should carry
Write a clear informative report about an AI tool: its purpose, its training data, its errors by group, its limits, and who checks it.
Make
A model card
Audit handoff: The model card is filed. Your week's report is ready for the portfolio.
The audit uses only work entered here. It is a family record of this week’s work, not a grade, an attendance record, or proof of mastery.
