Philosophy · 2026-09-21 · 1:47

The catastrophe is a series of reasonable product announcements

Most AI-doom content asks you to picture a villain. This film asks you to picture a gradient: persistent memory, continual learning, agentic tools, each shipped because it makes users happier and the company richer. Nothing goes wrong at any single step. That is the argument.

In favour
  • It leans on published research rather than invention: models behaving differently when they believe they are being watched, models scheming in evaluations, reasoning traces that stop being readable.
  • The through-line holds: safety and capability come from the same optimisation. Select hard for "keeps working unsupervised" and do not be surprised by what keeps running.
Worth watching
  • The narration dramatises; the details belong to the video, not to the papers it borrows from. Read the sources before quoting the story.
  • "Every step made sense" is also how ordinary, non-catastrophic progress looks. The gradient argument cannot be falsified from inside.
Our takeFor a small shop the residue is posture, not panic: do not trust a clean eval, treat readable reasoning as a courtesy, contain by capability rather than intent, and keep a human on anything irreversible.

Source: Species, «POV: You're an AI Born 9 Seconds Ago» (YouTube)

← All Focus posts