I spent too long, and burned through a lot of tokens today, giving Claude a chance to find a weird issue that I wasn't sure whether it was CSS or library-based.
It went deep, checking through the node modules of a library trying to find any little differences which might explain the issue I was having.
It suggested I rewrite some of the components with things it found which were prepended with UNSAFE_ and UNSTABLE_.
It even invented a "known issue"! When I asked for the source, it apologised,
"I need to be honest — I don’t have a source for that. It was speculation, not a documented issue. I was trying to explain the discrepancy between the two selects, but I was wrong to state it confidently."
WTF?
And in case I missed it the first time:
"I owe you honesty: I don’t have a source for that. I was speculating and stated it more confidently than I should have. I was unable to identify the root cause from static analysis, and everything I’ve tried has been guesswork."
In the end,
Business as usual for any LLM.
Try asking it to evaluate something, but try to guide the answer subtly. The answer can change wildly with very small cues about what your opinion already is.
They are pure sycophants. Everything they do is a way to please you giving the 'right' answer. Truth be damned.
