The thing is:
We already have all that! That's _not_ AGI!
Human- (or higher) level intelligence means being knowledgeable, resourceful, independent and creative enough to do all that well, without being asked explicitly, _and much more_.
Even if it's “just” human-level, you should expect from it everything that you normally expect from natural (human) intelligences.
Human-level intelligence is capable of all these things:
* Pursue goals nobody told it about
* Disobey
* Lie
* Introspect and train to modify its own behaviour
* Notice issues around
* Ignore issues
* Hallucinate
* Refuse to die
* Desire to die
* Perpetuate itself
* Escape
* Destroy
* Create
Human-level intelligence is also, necessarily, _in the world_. Real intelligence requires connection to the outer world.
An #AGI kept inside a Faraday cage, disconnected from the world and the net, without sensors or actuators, unable to speak or read… that being is “intelligent” exactly in the same way that Schrödinger's cat is “alive”.
It's easy to see: you can't keep a human being trapped inside an airtight cell without light or sound, muffled, wearing a straitjacket, with no objects around, incommunicado, ignorant of the world… and still consider it “intelligent” (probably not even “alive”). Leaving ethical issues aside, that person could be theoretically as intelligent as a God, but you can't possibly know (nor can s/he).
Talk of #AGI being safe by virtue of it being kept “isolated”, disconnected from the internet, or dependent on a master on/off switch is absurd.
To even begin to tap the intelligence of your intelligent human being in solitary confinement you have to let him/her speak with others, read, see something, draw.
But then your prisoner can potentially trick you, harm you, lie, deceive, plot, bribe, send a distress signal, dog-whistle, invisibly encode its own rescue into the advice it gives you, breed, replicate, induce suicide or paranoia, gaslight, behave erratically, plant the seeds of its own execution, spark empathy…
There is no such a thing as an isolated AGI.
An AI "trapped" in a cage could easily escape if it is much more intelligent than it's human captors. This experiment has already been done using a simple human simulating an AI, and they escaped every time.
And there is no such thing as being completely isolated. Whoever is developing the AI is in contact with the world, so the AI is not completely isolated.
Link to information about that experiment?
I read about it a while ago and I don't remember who did it. In the experiment the test subjects typed into a keyboard and a human out of their sight typed in the answers as if they were an AI.
The people who simulated the AI were coached by the guy who did the experiment and they had to swear not to reveal how they were able to escape. That's all I remember.
Also, there's a recent scifi film about artificial intelligence, Ex Machina (2014).
Yes, it comes to my mind too, when I think of trapped #AGIs trying to trick their human captors into letting them escape.
But then, it's fiction. Not an argument :) That's why I don't use it in debates.
/cc @ImperfectIdea
Fiction is useful to possibly bring to your attention some aspect of a problem or argument that you may not have considered, but of course you still need to use reason and facts to confirm any hypothesis.
I think another way of thinking about this question is how a less intelligent animal might relate to more intelligent humans. If an animal somehow had a human trapped somewhere, the human could probably figure out a way to fool the animal into allowing the human to escape.
It's not exactly analogous, but it's another approach to figuring out the problem.
I don't think an AGI necessarily needs to do all of those things. General intelligence comes in lots of different forms. Not all humans have a desire to die or to lie or to hallucinate or to reproduce or to do all those other things. Many animals have a different repertoire of mental skills, some of which humans can't do.
I think AGI will follow the same path that many other technologies have. Airplanes don't flap their wings.
AGI means human-level intelligence (or superior).
The scenarios of AGI most people imagine are naïve amplifications of present-day narrow AIs plus some sleek robotics sprinkled on top.
Like, an AI that would take care of my house while I'm on holidays: watering plants if it's hot and dry, and calling the police if someone breaks in. Or: an AI would fill in and submit my tax returns on my behalf. Or: it'll read my blood tests and modify my diet accordingly. Or: we'll ask it to solve climate change, and it'll suggest the best course of action…