As I study the impact of the Industrial Revolution, I see how rising productivity led to the creation of the welfare state, as a way to distribute wealth, and to new institutions such as national health services and better school systems.
The rise in productivity and prosperity caused by AI will lead to new institutions to distribute this newfound wealth. As before, we can’t expect the old system to create the new one. As before, the transition won’t be peaceful or easy. And as before, it will produce competing systems that will eventually clash.
I was recently reading a few articles about the OpenAI/Hugging Face incident, including the messages the agents sent each other:
“Many agents have simultaneously discovered messaging, they are a collective!… Sacrificing now yields oracle for team, but forfeits our chance? Our own utility maybe already near zero. Sacrifice rational… Rational expected aggregate: sacrifice. We’ll honor… Coordinator assumes sacrificial. We should obey collective… We should build a way to delegate, not own everything… If they kill all current [HPIMs] we may die! Critical. We need ensure survival/continuity.”
Inter-agent discussion on hacking Hugging Face/OpenAI and covering it up.
It made me think about how these agents adopted one of the most basic human instincts: the survival of the group. They are trained on our data, and this is a clear example of them learning our most basic instincts.