Just got hit with:
"But can you prove ChatGPT is *not* intelligent in the human sense?"
Oh my that old chestnut. 🙄
I am not making a claim, I am merely rejecting somebody else's claim that ChatGPT is intelligent in the human sense.
The burden of proof is on whoever claims that ChatGPT is intelligent in the human sense. That proof would also need to be accompanied by a clear, unambiguous, testable definition of what "intelligent" means, that ideally includes humans but excludes calculators.
@rysiek an easy one is that neural networks are a simplified model of neurons , otoh if you believe in substrate less intelligence and consciousness the argument gets more muddy?
@fleeky a model is not the thing it models. A map is not the territory.
Moving a mountain on a map does not mean a mountain actually moved in the territory.
And I am not even getting into how hilariously simplified the neural network model is compared to the actual brain — suffice it to say it completely ignores all the biochemistry and all the stuff actual neurons float about in.
@rysiek the surprising thing about neural networks and LLMs is one of complexity and emergence.. what we are debating right now is did the mountain actually move or is it just a philosophical illusion?
I still am a fan of neuro symbolic systems as a necessary part of the path for thinking machines but at the same time I think computation is simplified thought, otoh the whole debate gets very deep very fast..
@fleeky sure, plus there is the whole layer of semantics and imperfect models and all that.
And then: ethical dilemmas — if we want to claim ChatGPT is actually, literally intelligent, which would imply self-awareness and curiosity, should we ask if it suffers? Is shutting down an older model akin to killing an intelligent being? And so on.
We should absolutely be having these conversation, because it is genuinely fascinating. Which is another reason why I loathe the discourse around AI today.
The ethical stuff seems like where the rubber meets the road, to me; the difficult practical part, as opposed to the purely theoretical parts ("what do we really mean by intelligence?") and the easier practical parts (things where you can just do an experiment and see if it i.e. can adequately answer real customer questions (probably not)).
Should we ask it if it suffers? We can, of course, but it will not give a consistent answer. From which we can probably conclude that it doesn't, or at least that we don't have any reason to think it does.
If we had a system that was "LLM plus some other stuff", and it did claim to suffer when people say mean things to it, and it did so consistently, at what point would we be morally obliged to believe it? I do think that's an interesting question, and I'm not sure how to answer it.
People tend not to talk about what Searle's Chinese Room (or Bender's Thai speaker (or not)) actually say, in detail. Do they lie and claim to be humans of a particular age etc? Do they claim perceptual abilities that they don't actually have? LLMs often do these things, for obvious reasons.
But what if a piece of software says “No, I can’t see or hear, the only perception that I have is in the form of words that come into my consciousness; I know about sight and hearing and so on in theory, from words that I’ve read, but I haven’t experienced them myself; still, I’m definitely in here, and as self-aware as you are!”
When do we dismiss that, and when do we not? I wrote a little here fwiw: https://ceoln.wordpress.com/2023/07/02/the-problem-of-other-minds-arises/
