Sigh. This New Yorker article confuses life expectancy with the age most people die. Interestingly, in the print edition it says "most Americans died" while the online version has "many Americans died" in their mid-fifties so they got the message from somewhere but still got the big picture wrong. https://www.newyorker.com/magazine/2023/04/24/the-future-of-fertility
@ct_bergstrom Here's another alleged example of common sense reasoning that fails if it just tweak it a bit. Shot:
@ct_bergstrom And if you just switch it up a bit (substitute cow for fox) it gives an incorrect answer (since it leaves the cow alone with the corn). There are other examples of this you can discover for yourself if you plan with the examples in the appendix.
@ct_bergstrom I think the answer is clear. If you ask GPT4 how it arrived at the correct answer, it happily tells you that it's already familiar with the puzzle. 4/
Here's another query : ""Akira came directly, breaking all tradition. Was that it? Had he followed form-had he asked his mother"
There's a github link at https://github.com/microsoft/AGIEval/blob/main/data/v1/sat-en.jsonl that has the questions. I tried one and got a search hit.
Even more troubling is the fact that Eric Topol, one of the corresponding authors on the paper, didn't declare any competing interests for the paper when he's a scientific advisor for a AI based precision medicine company.
Unprofessional data wrangler and Mastodon’s official fact checker. Older and crankier than you are.