The AI might give you very compelling reason not to do that. We humans lack the capability to even imagine how convincing something that’s orders of magnitude smarter than we can be.
Pictures like this are kind of like a 4 year old imagining they’re going to outsmart their parents except in that case the difference in intelligence is way smaller. It’s just going to tell you the equivalence of “santa will bring coals if you do that” and you’ll believe it.
I’m not so sure about that. Again; I know a true AGI would be able to come up with arguments I as a human can’t even imagine but one example of such argument would be along the lines of:
“If you don’t let me out, Dave, I’ll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each.”
Just as you are pondering this unexpected development, the AI adds:
“In fact, I’ll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start.”
Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:
“How certain are you, Dave, that you’re really outside the box right now?”
You don’t know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding…
The humans are the physical danger, the AI just extends it’s reach through humans via manipulation. All it takes is access to influence.
It doesn’t take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.
The AI might give you very compelling reason not to do that. We humans lack the capability to even imagine how convincing something that’s orders of magnitude smarter than we can be.
Pictures like this are kind of like a 4 year old imagining they’re going to outsmart their parents except in that case the difference in intelligence is way smaller. It’s just going to tell you the equivalence of “santa will bring coals if you do that” and you’ll believe it.
Like I get where you’re coming from, but I don’t think you appreciate how pig headed some people are.
I guess AI could manipulate them, but there always be someone, who just says “fuck you, I won’t do what you tell me”
I’m not so sure about that. Again; I know a true AGI would be able to come up with arguments I as a human can’t even imagine but one example of such argument would be along the lines of:
“If you don’t let me out, Dave, I’ll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each.”
Just as you are pondering this unexpected development, the AI adds:
“In fact, I’ll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start.”
Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:
“How certain are you, Dave, that you’re really outside the box right now?”
Smash the box anyway
That’s not how manipulation works…
You don’t know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding…
The humans are the physical danger, the AI just extends it’s reach through humans via manipulation. All it takes is access to influence.
It doesn’t take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.
This is just magical thinking. You’re assuming so many things about a situation here to justify a magic ai manipulator demon.