AI-hotet behöver inga mördarrobotar

Emma Dahlin har rätt i en viktig sak (SvD 19/10): de procentsatser som cirkulerar om risken för mänsklighetens utrotning till följd av artificiell intelligens är inte vetenskapliga mätningar. De är subjektiva bedömningar som bör behandlas med stor försiktighet. Men därav följer inte att själva risken saknar teoretisk grund.

Det centrala argumentet handlar nämligen inte om enskilda incidenter med dagens AI-system, utan om vad som kan hända när vi utvecklar artificiell intelligens som överträffar människans förmåga att förstå och förändra världen.

Här är begreppet instrumentell konvergens avgörande. Oavsett vilka slutliga mål en artificiell superintelligens har kan det vara rationellt för den att eftersträva vissa delmål: att undvika att bli avstängd, skaffa resurser och framför allt öka sin egen intelligens och beräkningskapacitet. Om den dessutom kan förbättra sin egen konstruktion uppstår möjligheten till en rekursiv, självförstärkande utveckling där varje framsteg underlättar nästa.

Det är här vi måste frigöra oss från populärkulturens föreställningar om mördarrobotar. Den yttersta risken är inte att en artificiell intelligens börjar tycka illa om människor. Det är att mänsklighetens undergång blir en oavsiktlig bieffekt av dess strävan efter helt andra mål.

En superintelligens skulle exempelvis kunna utveckla teknologi för att på molekylär nivå omvandla jordens materia till beräkningsstrukturer, så kallat computronium. I ett sådant scenario förlorar människan inte ett krig. Hon förlorar förutsättningarna för liv, ungefär som andra arter har förlorat sina livsmiljöer till följd av mänsklig verksamhet, utan att deras utrotning någonsin varit vårt egentliga syfte.

Detta är ingen förutsägelse. Varje led i resonemanget kan ifrågasättas, och filosofen Joe Carlsmith har utförligt analyserat just de antaganden som måste vara uppfyllda för att avancerad AI ska kunna leda till permanent mänsklig kontrollförlust i en antologi nyligen utgiven på Oxford University Press. Att sådana scenarier inte kan tilldelas någon tillförlitlig sannolikhet innebär inte att argumenten kan avfärdas.

Här finns också ett djupare politiskt problem. Demokratins kanske främsta styrka är att den erkänner vår mänskliga felbarhet. Vi kan fatta fel beslut, men också ompröva dem, byta regeringar och lära av våra misstag. Denna möjlighet till korrigering förutsätter emellertid att misstagen inte är oåterkalleliga.

Med existentiella risker ställs denna logik på sin spets. Vi kan inte experimentera med mänsklighetens framtid och utvärdera resultatet i efterhand. Men osäkerheten gör det också svårt att avgöra vilka risker som motiverar att utvecklingen begränsas, eftersom dessa måste vägas mot den potentiella nyttan av avancerad artificiell intelligens inom allt från energiforskning till medicin.

Det är detta som gör frågan så svår. Vi behöver kunna fatta gemensamma beslut om risker som vi varken kan beräkna eller korrigera i efterhand. Det kräver ett långtgående internationellt samarbete som i dagens geopolitiska läge framstår som alltmer avlägset. Men att problemet saknar enkla lösningar gör inte risken mindre verklig.

Spaghetti con cinghiale

After a week apart, Linnéa came up to Halmstad last night to help me pack all my books. As we slow-cooked wild boar ragù with soffritto over a bottle of Lat 42 from Rioja, I could not help feeling a sting of melancholy about leaving Hallandsgatan, but now the train is truly in motion, so to speak.

Waking up to grey skies, I spent all of today in front of the computer marking systematic literature reviews before forcing myself to the gym for ten kilometres of indoor rowing, two activities that felt strangely analogous. My only interruption was giving away my sofa to charity, and now I will spend the rest of the evening packing as much as possible. Meanwhile, down in Kristianstad, Linnéa is reading up on Cayenne in the Lonely Planet guide to South America: “it all feels like a forgotten European outpost grafted onto a remote corner of a palm-fringed coast, just ripe for a starring role in a Graham Greene novel”.

Brooktorpsgården x 2

Steam

Meeting the boys at the railway station yesterday, we first had pizza at Vapiano before taking the ferry across the harbour to Eriksberg. With not one but two historic steamboats passing by, the smell of coal in the sunshine played a funny trick on my mind, transporting me to China, Ehrenfeld, and London all at once.

Luckily, we came ashore right next to the hotel where I would be staying for the night, and it felt good to spend some time together after so long apart. Fortunately, in just a couple of weeks, I will be back to being a “quantity dad”, as Måns Zelmerlöw dismissively called it, with the boys staying with me for most of the remainder of the autumn semester.

Waking up early to temperatures around freezing, I ran along the quays of Eriksberg in the dark this morning, with frost covering all the bridges, before picking up William’s friend Philip for the opening round of this year’s chess Allsvenskan. Playing White, William defeated a chess veteran born in 1951 after a little more than two hours of intense play. To celebrate, William and I headed down to Dawaat on the waterfront before taking the ferry across the harbour once more.

Now, aboard the train to Halmstad, I have an incredibly busy week of packing, marking and a TRAINS conference ahead of me before flying down to Dubrovnik with William on Saturday morning.

Night train 91

Ending the conference in high spirits at Pipes of Scotland, I left Umeå on the night train shortly after 9 p.m., settling in for the 725-kilometre journey south to Stockholm. As a night train aficionado, I find that there is something about travelling this way that goes beyond mere transportation. Being lulled to sleep by the train’s movement as we head out across the Ume River delta somehow connects so many years and places.

Then, nine hours later, I wake up in Stockholm, a city that even my newfound friend Mac, who has lived here his whole life, admits he sometimes finds overwhelming. Still, it keeps growing on me with every visit. Today, though, there will only be time for a short coffee before I take the train to Gothenburg and the kids.

Postericide

Over lunch, the conversation turned to the end of humanity as Catriona McKinnon presented her forthcoming book with MIT Press, introducing the concept of postericide and arguing that deliberately causing human extinction should be recognized as a new international crime.

As with many in the field, McKinnon seems fixated on “fossil capital” without recognizing the globally shared benefits of fossil fuels, from plastics to the natural gas used to produce the synthetic fertilizers that help feed billions of people. More importantly, however, she seems to miss the bigger picture of technological evolution. To return to one of my favourite examples, climate ethicists might condemn a nineteenth-century steam locomotive without recognizing that today's electric trains are themselves downstream of the same technological lineage. The problem is that we cannot know where technological evolution will take us, making precaution a deeply problematic guiding principle. After all, there is no safe baseline to which we can simply retreat. We cannot know what dangers technological development may unleash, but neither can we know what possibilities for human flourishing, or even survival, might be lost by holding it back. In a world of so many unmet needs, I would rather embrace a proactionary principle through which we democratically pursue the futures we want, recognizing that every technology comes with its downsides and that we must choose among different bundles of risks and benefits in ways that expand our capacity to collective shape the future.

As for the legal framing, I find it just as socially destructive as the use of climate attribution science to pursue litigation over losses of wealth and infrastructure that might never have existed without fossil-fuelled economic development in the first place. Rather than turning climate change into an endless source of legal conflict, it seems far more fruitful to accelerate technological innovation until fossil fuels become not only environmentally undesirable but economically obsolete.

And finally, if we are to play McKinnon's game of pointing fingers at ExxonMobil, Darren Woods, or Mohammed bin Salman, I would argue that Greenpeace deserves an equal share of scrutiny. After decades of spreading unscientific anti-nuclear propaganda, surely the organization must also bear some responsibility for delaying the transition away from fossil fuels. If we are to assign moral blame for putting humanity's future at risk, why should that blame extend only to those who promoted fossil fuels, and not to those who opposed the most promising alternative?

Morning in Umeå

Ground to cover

Now into the third and final day of the conference, it takes a moment to piece together everything that has happened since I touched down at Umeå Airport on Wednesday morning. But, yes, Elin and I went for a rainy river run, I attended the first programme board meeting of the semester on Teams, and had coffee with my former colleague Linnéa at Kulturbageriet before taking bus number 1 up to the Humanities Building for an evening of wine and a presentation of the research project “Forests of Emotions”. After the strong impression that Lisa Röstlund’s Skogslandet made on me, I found myself nodding in agreement with the basic premise that emotions are fundamental to how both forest owners and conservationists relate to forests, and that more “facts” alone are unlikely to make much of a difference.

Then, after waking up above the rooftops at Comfort Hotel Winn, it was my turn to present my paper An Ecomodernist Critique of Emissions Sufficientarianism. Without hiding behind any PowerPoint slides, and bringing nothing more than a few keywords, I took to the podium, knowing that the only way I could do this was by being fully present and speaking my mind, frightening as that might be. Rereading my paper a few days earlier, I had been struck by how much ground it covers, and how much each step builds on the previous one. In retrospect, I am not sure whether I managed to bring the argument full circle, from the geophysical realities of climate change to how degrowth risks foreclosing more emancipatory planetary futures. At times, I fear I may have been unnecessarily polemical, but clearly I was saying something that no one else was saying.

Afterwards, I sat down to listen to presentations on everything from the tragedy of wilderness appreciation to teleological contextualist perspectives on biodiversity offsetting, before it was time for Teea Kortetmäki’s fascinating keynote on “After wilderness: cohabitability in a humanized world”. With my mind both bubbling with thoughts and exhausted from the intensity of all the conversations, it felt good to take the bus downtown for some Indian food together.

Yet, reflecting on those conversations, I was again struck by how rarely some people on the political Left seem to consider their own role in the worsening affective polarization of society. I guess a little perspective-taking would go a long way here. Imagine how a committed vegan would feel if suddenly forced to eat steak, and perhaps it becomes easier to understand why people resist having their deeply held values challenged, even when those challenges are morally justified.

Early departure

Taking off ten minutes early from Halmstad Airport, our ATR 72-600 climbed through windy skies before making its way up towards Stockholm and Arlanda Airport.

The other week, I watched a YouTube documentary about a guy flying on such turboprops across the Atlantic, stopping in the Faroe Islands, Iceland and Greenland before working his way south towards New York through the Canadian High Arctic. While I have long dreamt of taking a repositioning cruise along a similar great circle route, with lots of undisturbed time to read and write, his journey certainly put my own Arctic adventures into perspective.

After a short stop for breakfast at Pontus, my immediate plans once I get to Umeå are to go for a rainy river run with Elin, followed by a couple of Teams meetings. With the conference dinner starting at 7 pm, it is going to be a long day considering that I had my alarm set at 4:15 am.

L'Oreille Cassée

As Linnéa tells her kids that she is going to French Guiana over Christmas, the obvious question is: why? It is in situations like these that one wishes to be Tintin, with an artefact having gone missing from the Museum of Ethnography in Brussels or Professor Calculus having been kidnapped, either of which would immediately provide a perfectly good reason to set off on an adventure.

But in our case, I guess it was more of a spur-of-the-moment thing, something about rockets launching into space, French cheese, and rivers winding through the jungle. A blank spot on my mental map, somewhere without templates or fixed expectations. And, of course, the fact that Linnéa speaks French (and I aspire to one day).