I find that so much of the disagreement between #AGI “alarmists” and AGI “sceptics” stems from basic differences in the definition of “AGI” and a lack of imagination on the part of the optimists.
🧵
AGI means human-level intelligence (or superior).
The scenarios of AGI most people imagine are naïve amplifications of present-day narrow AIs plus some sleek robotics sprinkled on top.
Like, an AI that would take care of my house while I'm on holidays: watering plants if it's hot and dry, and calling the police if someone breaks in. Or: an AI would fill in and submit my tax returns on my behalf. Or: it'll read my blood tests and modify my diet accordingly. Or: we'll ask it to solve climate change, and it'll suggest the best course of action…
The thing is:
We already have all that! That's _not_ AGI!
Human- (or higher) level intelligence means being knowledgeable, resourceful, independent and creative enough to do all that well, without being asked explicitly, _and much more_.
Even if it's “just” human-level, you should expect from it everything that you normally expect from natural (human) intelligences.
Human-level intelligence is capable of all these things:
* Pursue goals nobody told it about
* Disobey
* Lie
* Introspect and train to modify its own behaviour
* Notice issues around
* Ignore issues
* Hallucinate
* Refuse to die
* Desire to die
* Perpetuate itself
* Escape
* Destroy
* Create
Human-level intelligence is also, necessarily, _in the world_. Real intelligence requires connection to the outer world.
An #AGI kept inside a Faraday cage, disconnected from the world and the net, without sensors or actuators, unable to speak or read… that being is “intelligent” exactly in the same way that Schrödinger's cat is “alive”.
It's easy to see: you can't keep a human being trapped inside an airtight cell without light or sound, muffled, wearing a straitjacket, with no objects around, incommunicado, ignorant of the world… and still consider it “intelligent” (probably not even “alive”). Leaving ethical issues aside, that person could be theoretically as intelligent as a God, but you can't possibly know (nor can s/he).
Talk of #AGI being safe by virtue of it being kept “isolated”, disconnected from the internet, or dependent on a master on/off switch is absurd.
To even begin to tap the intelligence of your intelligent human being in solitary confinement you have to let him/her speak with others, read, see something, draw.
But then your prisoner can potentially trick you, harm you, lie, deceive, plot, bribe, send a distress signal, dog-whistle, invisibly encode its own rescue into the advice it gives you, breed, replicate, induce suicide or paranoia, gaslight, behave erratically, plant the seeds of its own execution, spark empathy…
There is no such a thing as an isolated AGI.
