Tekma za superinteligenco je hazardiranje s prihodnostjo človeštva

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight toward self-improving superintelligence and gambling with our lives.

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers couch their public statements in more restrained language to sound sensible—but I hear the same people express fear privately. No other human activity poses this level of danger.

A common response is: “If they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well understood, but the company is locked in a race to get there first. Its people believe that no one else will act responsibly, so they must do it themselves, despite the risks.

Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that no better trajectories are available.

I remain optimistic about the potential for coordination. Warning shots such as the Hugging Face attack have made pacing agreements among US laboratories more viable. However, I do not feel that we are on track to prevent a global race. Doing so may require costly measures, including a temporary ban on improving model capabilities.

If you are a laboratory researcher, I urge you to consider what the next few years will actually feel like. Do you want to initiate a superintelligent reinforcement-learning run without a rigorous understanding of its mind? Should you simply put your head down because “it is happening anyway”—or use this moment to demand different conditions?

Komentiraj