In today's "LLM is the future" rebuttal, this exchange from #gitlab
"
Q: Is 23 less then twenty five ?
A: No, 23 is not less than 25.
"
and #gpt4all (nous hermes 2 mistral DPO) which is somehow even worse
"
Q: Is 23 less then twenty five ?
A: No, 23 is not less than 25. In fact, it is greater by 2 units (25 - 23 = 2).
"
@falken What model did you use for the first test? And what happens if you ask it "Is 23 less than 25" (written as numbers, not in words)? When I ask Claude 3 Opus "Is 23 less than twenty five?", it answers "Yes, 23 is less than 25."
