‘A Risk That We Will Smash Into the Wall’

Behind on my posts again, read my recent interview with Nick Bostrom at Oxford’s Future of Humanity Institute for Publishing Perspectives in full below or by following this link.

“Artificial Intelligence could in the end replace most, if not all, of the high-level functions of an editor, but if you get to that point then the whole game changes not just for an industry but for humanity as a whole, because it will have become superintelligent,” says Oxford academic and prophet of an AI apocalypse Nick Bostrom.

Projected on to the wall behind him is a live stream from the Google DeepMind AlphaGo challenge being played in Seoul. DeepMind has become the first computer program to beat a professional human player at the game of Go, this month scoring a 4-to-1 victory over one of the world’s top champions.

Bostrom defines superintelligence as an intelligence much smarter than the best human brains in practically every field, including scientific creativity, general wisdom, and social skills. It’s also the title of his book Superintelligence: Paths, Dangers, Strategies (Oxford University Press), which was published in 2014 and made The New York Times bestseller list.

If you haven’t been following the rise of the new technology, then the threat of an AI takeover in the next 100 years may come as a surprise or seem a joke, even a piece of science-fiction. It shouldn’t. Bostrom’s book—a call to arms to the leaders of industry, government, and the military about the possible threat posed by AI to the future of humanity—led Raffi Khatchadourian in November to dub Bostrom “a philosopher of remarkable influence” in his feature on the professor, The Doomsday Invention.

Bostrom has been listed on Foreign Policy’s Top 100 Global Thinkers list twice, and he was the youngest person included on Prospect magazine’s World Thinkers list.

And on April 11th, Bostrom’s keynote address at London Book Fair’s Quantum Conference will help make good on the event’s tag line, “Publishing & So Much More.” He certainly is that. Success has made him a hot ticket.

The slightly reserved, precise, and very sharp Bostrom is the founding director of the Future of Humanity Institute (FHI) in Oxford, which explores often neglected “existential risks” to the future of humanity.

Bostrom has defined existential risks as ones that could annihilate life on Earth or permanently and drastically curtail its potential, such as supervolcanoes, pandemics caused by bio-weapons, nanotechnology, and of course AI. He explored this same ground in an earlier book, Global Catastrophic Risks. Comprising 26 essays by world experts and edited with Milan Cirkovic, it’s enough to keep you up all night.

‘Existential Risks’

The Institute itself is situated over a gym in a maze of Oxford streets that date back a thousand years and it shares space with the Centre for Effective Altruism. Not surprisingly given his schedule, Bostrom is already behind schedule when he walks in to begin our interview holding his trademark super large smoothie. His assistant hovers nervously behind me as he has a phone call booked straight after our chat. Bostrom says:

“The book is my attempt to think through all the issues that might arise as we move towards machine superintelligence. The first couple of chapters talk about how we get there – the rest focuses on what happens when we do, when machines do surpass humans. This is called the control problem. How can we create controls so that machine superintelligence will be beneficial to humanity as opposed to destroying us?

“After all, AI has an upside. It is a very general-purpose technology. Almost everything we expect technology to do can be done better if the technology is smarter and adapts to the user. In the long term, all the things we want to achieve, like space settlement, we will need machine intelligence for.

“However, [while] the development of machine superintelligence may well be a portal that civilization will have to pass through to meet its full potential, there is a risk that rather than go through the portal we will smash into the wall. We could have a terminal catastrophe which will lead to the destruction of everything. It may even be the scenario that happens unless we put in a heroic effort.

“AI is advancing quickly. It is very fashionable as all the big tech companies have some kind of AI effort. What happens when AI succeeds is now taken a lot more seriously than a sci-fi type thing to talk about over drinks. AI is fashionable. So the key players are starting to realize that maybe there should be a few things that we should start doing now.”

He’s more sanguine, he says, about the implications of DeepMind’s victory with AlphaGo than many of the rather lurid headlines.

“DeepMind’s victory in AlphaGo is significant as a milestone rather than for the technology itself,” Bostrom says. “It shows the progress that has occurred in machine learning as it is—a general-purpose software, not handcrafted material put together to play Go. With some minor modifications, the same applications could play other games and be used for other applications. Traditional AI would have to be reworked entirely when you wanted to use it for a new application.”

‘Publishing & So Much More’

When the stakes are so high in his field, does it seem at all urgent to talk about publishing? Given the threats he routinely explores, Bostrom has only recently started thinking about how AI might affect publishing, he says. But he does have some good news:

“Certainly there are lower-level editorial tasks that AI should be able to do soon, and that includes an interesting application of a more advanced version of spell-checking, where the AI suggests changes in sentence structure as well the spelling.

“While I could see AI augmenting or replacing humans in visual arts, in writing it would be harder, as the process of writing decides what is to be written next. The current automated writing software is a very limited form of writing—basically putting data into sentences.

“However, it would be a mistake to reduce this to an examination of the impact on one sector when its impact on humanity could be so great. At the moment, the planet is in our hands and our technology gives use the power to organize, plan and shape it. If an AI can shape the world in the way it wants, then we might get paved over, just like we pave over ant colonies—not because we hate them, it is just a dispassionate act.”

Whether we would have any warning of the apocalypse is a different matter.

“There is some uncertainly whether there will be a sudden advance in AI or whether there would be warning signs building up to it. If it goes well, it will be a controlled detonation which will produce a desirable outcome for humanity.

“We don’t want to be in a position in 50 years’ time of thinking ‘it’s too late now – if only we’d started to think about it in 2016’ .”