Vitalik Buterin assesses the risk of AI against humanity: "Even Mars is not safe"

For the creators of ETH, the development of artificial intelligence needs to prioritize human intentions to achieve positive results, not just profit maximization.

This Monday (27), Vitalik Buterin, the creator of ETH, published an article in which he shared his thoughts on “technological optimism”, inspired by Marc Andreessen, who gave his opinion on artificial intelligence (AI) in his manifesto of “technological optimism” published in October of this year.

While Buterin agrees with Andreessen’s positive outlook, he also points to the importance of how AI is being developed and the future direction of the technology.

Buterin recognises the risks of AI, including the extinction of humans.

“It’s an extreme statement: no matter how much damage a worst-case scenario like climate change, a man-made pandemic, or nuclear war might cause, there are many civilized islands that will pick up the pieces intact,” he said. ”

“But if the super-intelligent AI decides to rebel against humanity, it will most likely survive and wipe out humanity forever, not even Mars.” ”

Buterin referred to a 2022 survey by AI Impacts, which said that between 5% and 10% of participants, respectively, believe that humanity will face extinction due to AI or that humans will not be able to control AI.

He said the security-focused open source movement is well suited to spearhead AI development, rather than private companies and venture capital funds.

“If we want a future that’s both super-intelligent and ‘human’ (where humans aren’t just pets and actually maintain significant control over the world), then it looks like something like this is the right choice and the more natural choice,” he said. ”

Buterin continued, "What we need is for humans to be proactive in their choice of direction and outcome. And the ‘profit maximization’ formula doesn’t automatically come to them. ”

The good side of technology

Buterin said he loves technology because it expands human potential, mentioning the history of innovation, from hand tools to smartphones.

“I believe these things are very good, and it’s also very good to extend human influence further to planets and XLM, because I believe that human beings are very good,” Buterin said.

He said that while he believes that transformative technologies will lead to a brighter future for humanity, he refuses to believe that the world should remain the status quo and simply reduce greed and build an effective public healthcare system.

“Some types of technology make the world a better place more reliably than others,” Buterin says. "Some types of technologies, if developed, can mitigate the negative effects of other types of technologies. ”

Artificial intelligence control

Buterin warns that the development of digital authoritarianism and surveillance technology is being used against those who challenge or disagree with the government and are controlled by a select group of technocrats. He said most people would probably rather wait a decade for AI to develop than see the technology monopolized by a conglomerate.

“My biggest concern is that the same kind of management technology that the whole country has allowed OpenAI to serve more than 100 million customers with 500 employees will also allow a 500-strong political elite, or even a five-member board of directors, to maintain an iron fist with more than 500 employees,” he said. ”

Although Buterin says he is sympathetic to the Effective Acceleration (also known as “e/acc”) movement, he has mixed feelings about his passion for military technology.

“Enthusiasm for modern military technology as a force for justice seems to require the belief that the dominant technological power will reliably be one of the good guys in most conflicts, both now and in the future,” he said, citing the idea of military technology. can be good, because it is built and controlled by the United States, and “America is good.”

He asked, “Does it take to be an E/ACC to be an American supremacist, betting everything on the morality of the government now and in the future and the future success of the nation?” he asked.

Buterin warns against granting “extreme and opaque power” to a small group of people in the hope that they will use it wisely, preferring instead the “d/acc” philosophy of defense, decentralization, democracy, and difference. He argues that this mindset can be adapted to true altruists, liberals, pluralists, blockchain advocates, and “solar and moonpunk.”

“A world that’s good for defense is a better world for a number of reasons,” Buterin said. "First, of course, there are immediate security benefits: fewer deaths, less destruction of economic value, and less time wasted in conflict. ”

“However, there is less recognition that a world conducive to defense contributes to the development of healthier, more open, and more freedom-respecting forms of governance.” ”

While he stressed the need to build and accelerate, Buterin said society must constantly ask itself where we are accelerating. According to Buterin, the 21st century could be a pivotal century for humanity and could determine the fate of humanity for thousands of years.

“These questions are challenging, but I look forward to observing and participating in the great collective effort of our humanity to find answers,” Buterin said.