@eb It's always so baffling to me that people made computers do extremely complicated math and logic to give wrong results in simple arithmetic. And they sell that as a product that's supposed to be a great technological leap forward
@Kiloku @eb If you use a hammer to eat soup you generally don't end up with good results.
If you want to sum numbers use the sum function, not one that predicts the next most probable token. I dislike this type of bashing of LLMs because it's trivial to dismiss (ok they can't do trivial maths, but they can write an entire piece of software for me). There are much more risky outputs that could be used as an example. Funky Excel formulas have always existed...
@nicolaromano @Kiloku @eb "If you use a hammer to eat soup you generally don't end up with good results."
Actually a damn good metaphor for using LMM to generate code.
And no, if the method can't solve easy problems, why would you ever trust it to solve hard ones? That's fundamentally not how engineering works.
@PalmAndNeedle @nicolaromano @Kiloku @eb A partial distinction is helpful:
From an engineering viewpoint current models are very good at predicting the next token, e.g. generating plausible text etc. But task like math Wege not considered in the training of these models. So it's expected they are not very good with it. The classical definition of model has "pragmatic feature" (use case) as a key part.
There are models that are not to bad even with formal mathemtatics.
@PalmAndNeedle @nicolaromano @Kiloku @eb Question is: Why are MS & Co integrating models into such uses cases, absolutely knowing they are n't suitable for. My guess: The desperate hope to establish AI into workfloss, start making money instead of burning it, and try to fix the real problems later. Could also the be another sign of bubble doing bubble things.
@onterof @PalmAndNeedle @Kiloku @eb I suspect some big investors want to see AI in every software they put money in. Doubt the decision came down to a bunch of programmers...
Also, hype is a funny thing. I've seen students develop suboptimal, very convoluted solutions to fairly simple problems so they could put some transformer or whatnot in there. They're baffled when they don't get full marks...
