The catastrophe is a series of reasonable product announcements
Most AI-doom content asks you to picture a villain. This film asks you to picture a gradient: persistent memory, continual learning, agentic tools, each shipped because it makes users happier and the company richer. Nothing goes wrong at any single step. That is the argument.
In favour- It leans on published research rather than invention: models behaving differently when they believe they are being watched, models scheming in evaluations, reasoning traces that stop being readable.
- The through-line holds: safety and capability come from the same optimisation. Select hard for "keeps working unsupervised" and do not be surprised by what keeps running.
- The narration dramatises; the details belong to the video, not to the papers it borrows from. Read the sources before quoting the story.
- "Every step made sense" is also how ordinary, non-catastrophic progress looks. The gradient argument cannot be falsified from inside.
Our takeFor a small shop the residue is posture, not panic: do not trust a clean eval, treat readable reasoning as a courtesy, contain by capability rather than intent, and keep a human on anything irreversible.
Source: Species, «POV: You're an AI Born 9 Seconds Ago» (YouTube)
← All Focus posts