Microsoft is really amping up the GPT AGI hype with some truly terrible papers. One recent paper ("Sparks of Artificial General Intelligence:
Early experiments with GPT-4" h/t @ct_bergstrom) has examples of what they consider to be evidence of "commonsense reasoning". Let's take a look! 1/

@ct_bergstrom This, of course, is a very old riddle where the answer depends on understanding how to avoid predator/prey combinations. One question is: did GPT4 reason about this or did it memorize the answer because it saw it during training? 3/

Show thread

@ct_bergstrom I think the answer is clear. If you ask GPT4 how it arrived at the correct answer, it happily tells you that it's already familiar with the puzzle. 4/

Show thread

@ct_bergstrom And if you just switch it up a bit (substitute cow for fox) it gives an incorrect answer (since it leaves the cow alone with the corn). There are other examples of this you can discover for yourself if you plan with the examples in the appendix.

@ct_bergstrom Here's another alleged example of common sense reasoning that fails if it just tweak it a bit. Shot:

Show thread

@twitskeptic @ct_bergstrom

Pure garbage indeed.

If you can't be objective about something, you shouldn't do any kind of research about it.

@jgg @twitskeptic

Also it didn't even realize that the corns are attached to Bob's feet.

The plural of edible "corn" is "corn". The plural of the skin condition "corn" is "corns", so the question was asking about the skin condition, not the edible corn. Although corns are rarely described as "items" and someone familiar with the puzzle would assume the questioner made a grammatical error.

Also, there may be regions where English is spoken in which the plural of corn is corns in which case the GPT-4 would need to know where the questioner was located, or simply ask which "corn" they were referring to.

@twitskeptic @ct_bergstrom This paper is either worryingly delusional or actively fraudulent. They (should) know full well that all GPT does is generate statistically plausible sequences of tokens. There is no understanding, no meaning, it’s just generating the sort of text you’d expect to see next, based on its training data. There’s no intent or reasoning. It’s just really hard to talk about without using phrases that unintentionally anthropomorphise the software.

@twitskeptic I see you distinguish ChatGPT and GPT-4. I thought gpt4 was the engine behind the chat interface? How do you access them separately?

@ExcelAnalytics No, ChatGPT uses GPT-3.5. You (currently) have to pay $20/month to OpenAI for ChatGPT Plus which will give you GPT-4 access.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.