Christian Kothe @christiankothe@qoto.org

@rooster That could have featured in the opening scene in Forrest Gump FWIW.

@grumpygamer there's your new character name ;)

@tdietterich I'm actually impressed that they find time for a book at all amid fiercest competition with OpenAI, and now from a bit of an underdog position. There could also be more on how this can be synthesized into a probabilistic programming language like e.g. numpyro (sampling, variational inference).

@tdietterich I combed through it (in a rush). It's a nice very dense collage of much of the core math of today's ML/DL scene. There's not much of an arc though, and chapters aren't held together by much, so the topic still deserves a much more didactic textbook some day.

@zeux Might keep an eye on JetBrains IDEs... I've mainly used them for Python and much less so with C++, but there have been some C++ and multi-lang related tools coming from that direction lately (Rider, Fleet, CLion Nova).

@deobald @simon Thanks! Yeah figured that'd be a reason, I'm impressed they're still busy adding things like SIMD optimizations etc to it as we speak. And yeah, certainly Lisp is a great language for pumping out crazy amounts of functionality especially, when one's dealing with tree structures like json, queries, and responses.

@simon 70% Steel Bank Common Lisp, 20% Rust. There's definitely a connoisseur at work. Kinda amazed that someone would whip that out in 2023 to make SotA software of that size.

@rabaath Funny thing is that it feels clunky to me even though I didn't come from R!

So what are everyone's bets as to what the secretive hardware form factor is that Jony Ive is to be working on with OpenAI? Here's mine: it's probably something that looks like (1) a bluetooth headset and (2) after that something like Snap's Spectacles.

Reflecting on the chunky cables featuring in Half-Life 2 and later, I came to the conclusion that Combine electrical systems must clearly be running on 1.5 Volts!

@robpike When I got to "One standardized version of G-code, known as BCL (Binary Cutter Language), is used only on very few machines." I could suddenly sense the desperation Owen Lars must have felt as he found himself without a droid that speaks the binary language of moisture vaporators.

@grumpygamer Know the "bus factor"? Maybe we should call it the 737 factor now

... the wild and probably bogus details aside though, I've never bought into the idea that hallucinating or BSing is an unsolvable intrinsic flaw of LLMs, since it may take not much more than operationalizing the process we humans use to construct an internally consistent world model, which is to explore a range of consequences that follow from our beliefs, spot inconsistencies, and update our world model accordingly. And that looks like something that could be attempted in well-trodden paradigms like RL or GANs or something that's not much more complex, so my bet would be that we should've largely worked it out within 4-5y.

Woke up from a strange vivid dream this morning in which I was attending an ML symposium and someone gave a talk on overcoming the hallucination problem with LLMs. The main slide had a graph of nodes representing LLM statements and they were doing some sort of graph diffusion process where the "curl" operator was pinpointing the contradictory/inconsistent statements, which they could then follow to update the weights to discourage those from occurring. Needless to say I immediately tried to arrange an improptu mtg between the speaker and some DL luminaire who was also there to get them to adopt it.😂