After hearing Sebastian Bubeck talk about the Sparks of AGI paper today, I decided to give GPT-4 another chance.
If it can really reason, it should be able to solve very simply logic puzzles. So I made one up. Sebastian stressed the importance of asking the question right, so I stressed that this is a logic puzzle and didn't add anything confusing about knights and knaves.
Still, it gets the solution wrong.
@ct_bergstrom Another "Sparks of AGI" problem is the claim that GPT-4 can reason about emotions in complex situations. The example they give isn't particularly complex, so I came up with another one, well, I don't see a future career in therapy for this model.
