@boris_steipe sorry, let me try to explain better. This is all about intent. Is the objective that we use technology to explain things to us, or do we want to be able to explain things to technology in a way that it can reproduce basic questions whose responses have no added value, and arguably are reduced to fact checking? I'm interested in the former, the latter is what I see in your guiding simplifications. Maybe you meant it differently?

Remember from school that facts are the mostly basic type of knowledge. The more valuable ones are functional and others that are widely called conditional, but as I hinted in my article, are very diverse.

@mapto

I respectfully disagree. Your article claimed that there were fundamental limitations of performance of LLMs. I noted that we are in fact not sure about the limitations, because they are difficult to disentangle from their semantic presentation. In particular I had observed abilities that in my understanding would cover many if not all of the higher order processing abilities you had listed. They can be elicited.

First, it is surprising that such emergent abilities appear, especially given how those algorithms work: their "world" is entirely based on relationships between abstract symbols, and has no grounding in experience at all, and their training consists merely in predicting a next token in a sequence. That we see higher order "understanding" emerge under these circumstances is profound. It strongly supports the idea that language and thought are deeply linked: language is not what we think, it is how we think.

Second, what must not be conflated is the ability to reason in principle, and the ability to parse human language, with all of its ambiguities, unstated assumptions and discursive conventions.(1)

When you say "explain things to technology", you are referring to the latter: "parsing human language". This is a different question from the ability to "reason". Now, the current dialogical abilities of LLMs are already a radical departure. Remember: previously, this process of "explaining things" was called programming.

Of course there are limitations. Where I disagree with you is: these limitations of communicating intent are not the same thing as a lack of reasoning.

The distinction is important because it can lead to significantly improved interactions. Jason Wei of Google has done groundbreaking work on "chain-of-thought prompting", and similar work from Google Brain on "Least-to-most prompting" shows that there are probably many additional strategies to be discovered.

All the best.

---
(1) One of the more surprising aspects of ChatGPT is that it embodies Paul Grice's "cooperative principle" – the well-known maxims of quantity, quality, relation, and manner.

@boris_steipe it's clear we'll not agree, but I struggle to comprehend what though process would lead to the apparent confidence of saying "ChatGPT embodies Paul Grice's "cooperative principle". Is there some sort of a widely agreed verification process that it has passed, or is this based on some personal observations?