Christian Kothe @christiankothe@qoto.org

So what are everyone's bets as to what the secretive hardware form factor is that Jony Ive is to be working on with OpenAI? Here's mine: it's probably something that looks like (1) a bluetooth headset and (2) after that something like Snap's Spectacles.

Reflecting on the chunky cables featuring in Half-Life 2 and later, I came to the conclusion that Combine electrical systems must clearly be running on 1.5 Volts!

... the wild and probably bogus details aside though, I've never bought into the idea that hallucinating or BSing is an unsolvable intrinsic flaw of LLMs, since it may take not much more than operationalizing the process we humans use to construct an internally consistent world model, which is to explore a range of consequences that follow from our beliefs, spot inconsistencies, and update our world model accordingly. And that looks like something that could be attempted in well-trodden paradigms like RL or GANs or something that's not much more complex, so my bet would be that we should've largely worked it out within 4-5y.

Woke up from a strange vivid dream this morning in which I was attending an ML symposium and someone gave a talk on overcoming the hallucination problem with LLMs. The main slide had a graph of nodes representing LLM statements and they were doing some sort of graph diffusion process where the "curl" operator was pinpointing the contradictory/inconsistent statements, which they could then follow to update the weights to discourage those from occurring. Needless to say I immediately tried to arrange an improptu mtg between the speaker and some DL luminaire who was also there to get them to adopt it.😂

If you haven't come across this nice science-related article: https://arstechnica.com/science/2024/01/the-key-to-fighting-pseudoscience-isnt-mockery-its-empathy/

Wondering if anyone out there is using LLMs are a proposal heuristic in NAS. Would seem fruitful (e.g. after fine-tuning on NeurIPS). Add in reinforcement learning for bonus points. It's not quite recursive self-improvement since re-architecting and retraining the LLM would be a slow, expensive, and human-in-the-loop step.

So I suppose Altman and Brockman are now going on to found NeXTAI, am I guessing that right?

So I guess that's one way to read the top-right corner of my desktop (hat tip to my wife). "It's 10:44PM. The year is 3500 RPM, and it is 62C outside." Pretty dystopian!