@glc @gregeganSF @acowley No, bytes/tokens/words/whatever is irrelevant. The important part that's wrong in the "word-space" model is that it misses the context. The "language" part is a red herring. What's really going on is a tangle of suspended code that's getting executed step by step. And yes there are concepts, entities, and all that stuff in there.

@dpwiz

I'd say there is syntax without semantics (in the traditional sense of formal logic, that is).

You have some other view evidently.
That much is now clear.

I don't see much difference from Markov and Shannon, apart from some compression tricks which are needed to get a working system.

@glc Perhaps. I just hope this not another "X is/has/... Y" claim.
What's your favorite or most important consequence of this distinction?

@dpwiz

That no concepts are involved, and the numerous corollaries of that, I suppose. At least, that's what I find myself harping on now and then.

I have no strong interest in the details. though considerable interest in watching this play out.

—Someone like Cosma Shalizi is going to actually get into the weeds a bit more:
http://bactra.org/notebooks/nn-attention-and-transformers.html

You'll probably find much to agree with and much to disagree with there. And at adequate length.

@glc > I find this literature irritating and opaque.

That's a promising start! (8