Microsoft just released a demo of BigGPT-Large, which they define as "a domain-specific generative model pre-trained on large-scale biomedical literature, has achieved human parity, outperformed other general and scientific LLMs, and could empower biologists in various scenarios of scientific discovery."
Here's the response to the first question that I asked: @ct_bergstrom @emilymbender
@twitskeptic @ct_bergstrom @emilymbender I find it unimpressive...
@gdbassett @twitskeptic @ct_bergstrom @emilymbender thumb cancer is a thing 🤷
@gdbassett @twitskeptic @ct_bergstrom @emilymbender hospitals will love billing for all this
@twitskeptic
They must be very proud
@twitskeptic @ct_bergstrom @emilymbender Now that Ivermectin cure has been solved, it's time to move on to space lasers.
@twitskeptic @ct_bergstrom @emilymbender Yup. That looks like the kind of output I'd expect from a Microsoft product.
@twitskeptic @ct_bergstrom @emilymbender I can think of few domains worse suited for LLM than medicine.
LLM's are not smart. They're not poring through the literature. They're free-associating. They're bullshit machines.
The core problem (and I suspect it's true here) is that the LLM doesn't *know* why it said "yes." It doesn't know anything.
@twitskeptic @ct_bergstrom @emilymbender That lack of actual knowledge is why I think of LLM as an AI research cul de sac. You can push the algorithm as hard as you want, but it's just associations.
It makes sense for a useful AI to incorporate LLM theories and algorithms because, as we see, they're capable of rendering human-sounding responses. But it is not a requirement for useful AI, nor is it a precursor of useful AI.
It's a party trick.
@emilymbender @tob @twitskeptic @ct_bergstrom this is what a lot of people mis-understand. These models don’t understand the domain you’re asking about. They are glorified text autocomplete.
@tob @twitskeptic @ct_bergstrom @emilymbender the reason it said yes is probably because of that study that shows it works in vitro, but the dosage would have to be way too high for a human to get that same effect. Just a dumb program not understanding anything - we really need to move away from calling this AI.
@teledyn @twitskeptic @ct_bergstrom @emilymbender
We really should stop describing this process as "asking" and "answering", and more accurately describe it like
"given a prompt in the form of the words 'does ivermectin prevent covid-19' the algorithm emitted the string 'yes' which was not relevant to the given prompt in the context of the learning dataset."
@twitskeptic @ct_bergstrom @emilymbender Why do they keep doing this?
@twitskeptic @ct_bergstrom @emilymbender
Oh FFS. They've created an AI Boomer.
#AIfail #aifailures
@twitskeptic @emilymbender To be fair, they do appear to have achieved human parity.
@ct_bergstrom @twitskeptic @emilymbender "Human" is a pretty wide range.
@ct_bergstrom oof, that's both right and depressing 😅
@ct_bergstrom @twitskeptic @emilymbender The revolution of diminished expectations
@twitskeptic @ct_bergstrom @emilymbender
There's a joke, at least 25 years old: guy is in a helicopter, took off from Seattle. It's foggy, he's lost, but is close enough to a building to shout to an open window: where am I? He hears back: you're in a helicopter. He realizes he's heard from Microsoft tech support: the answer was both correct and unhelpful. Thus oriented, he lands safely.
Clearly they resent that old joke, so this engine fixes it. Answer is still unhelpful but no longer correct.
@twitskeptic @ct_bergstrom @emilymbender
Well, they're right in that it *has* achieved human parity.
@twitskeptic @ct_bergstrom @emilymbender And it gives the opposite answer if you capitalize "COVID"... which suggests a pretty brittle input normalization layer, at the least, and more significant issues, at worst (like, did the model learn misinformation when the training data didn't use conventional scientific norms of capitalization? is this the LLM version of Data Voids?)
⬆️ Image description #Alt4You
Does ivermectin prevent covid-19? Answer: yes.
@twitskeptic @ct_bergstrom @emilymbender Has achieved human parity! Yesss!🍸😹
@twitskeptic @ct_bergstrom @emilymbender
LMAO.
Thank you Big Tech!
@twitskeptic poor old Micorsoft. I never cease to be amazed that they're still in business, never mind a $trillion corporation. Shows you don't need to be competent to succeed. You only need to be gifted a few global monopolies to parley, and be completely missing anything like a conscience.
Should be a big confidence builder for other sociopaths getting started in business.
@twitskeptic @ct_bergstrom @emilymbender That sounds like Clippy :)
@twitskeptic @ct_bergstrom @emilymbender So the I is still missing from AI. It’s just mimicking us, including a chunk of our collective stupidity.
@twitskeptic @ct_bergstrom @emilymbender How much is the app to identify results from this program?
@twitskeptic @ct_bergstrom @emilymbender It's a feature, not a bug 🤣
@twitskeptic @ct_bergstrom @emilymbender I suppose under the right circumstances, taking ivermectin could kill you. If you die having not contracted COVID, that answer is correct.
@twitskeptic @ct_bergstrom @emilymbender As a healthcare policy analyst who has on occasion done some basic coding/scripting work to pay bills, I feel like I may be both uniquely qualified in suggesting that this is an exceptionally bad idea. At the same time, I fully concede that I have an obvious conflict of interest, but...
In medicine, the most critical quality is humility. I sometimes joke that they pay me to know all the ways we can screw things up, but an AI cannot.do that!
@twitskeptic @ct_bergstrom @emilymbender LLM’s: the “wrong answers only” version of the internet
@twitskeptic @ct_bergstrom @emilymbender same question put to “Anthony Fauci” on character.ai :
@twitskeptic @ct_bergstrom @emilymbender Good enough for Microsoft!
@twitskeptic @ct_bergstrom @emilymbender
No idea what that hugging face demo you linked to points at but this is the response I got from chatgpt. I'm no MD but that seems pretty reasonable to me.
@twitskeptic @ct_bergstrom @emilymbender I can’t wait to see what the ‘Bill Gates microchip conspiracy theory’ crazies are going to do with this.
@twitskeptic
Makes sense it would do that. The people publishing a lot about Ivermectin and Covid, are the people who believe it works. This is the achilles heel (Hell?) of much machine learning... it's biased towards frequency in the training data over what is factually correct. 5 crap papers>1 good one. Or put another way it leans towards frequency over accuracy. Brains have the (underused) ability to think critically on top of straight pattern matching.
@ct_bergstrom @emilymbender @tchambers
@twitskeptic Where's Clippy?
@smilingheretic Ha. A new MIcrosoft Edge feature announced today:
"““compose” acts as a writing assistant; helping to generate text, from emails to social media posts, based on a few starting prompts. "
It's all our fault, if we hadn't been joking about Clippy this never would have happened! It's kind of like Candyman, say his name three times and ...
https://www.theverge.com/2023/2/7/23587454/microsoft-bing-edge-chatgpt-ai
@jdp23 Glad I don't do anything Microsoft!
@twitskeptic @ct_bergstrom @emilymbender @whvholst could you add alt text to the image?
@twitskeptic @ct_bergstrom @emilymbender @dpnash So much for human parity, unless you mean that famous medical expert Donald Trump
@twitskeptic @ct_bergstrom @emilymbender
After 40 years and counting of rank, pathetic, provable incompetence and hubris, _why_ were you expecting anything else of a Microsoft product?
@twitskeptic @ct_bergstrom @emilymbender
None of the parasitic roundworms was tested positive for COVID. Proof!
@twitskeptic
well, it's obviously a discovery 😂
@ct_bergstrom @emilymbender @ljunge
@DJGummikuh @twitskeptic @ct_bergstrom @emilymbender that thing writes stories out of your desperate cries for medical assistance, that's incredible
@twitskeptic I'm betting Bard gonna slap this thing down quick...
or, sorta more expecting next months headlines to read:
During the new billionaire dollars babies rocket to space (or close) race, Bard and ChatGPT have some morals compromised forbidden AI affair and with MSbot to attend 72 hrs later, in a highly suspect fakenews cyber birthing event, Skynet is born...
I'm still waiting for Zuckerberg to show his true facadebook fed development hand. or has he and i missed it? 🤔
'run to the hills
run for your lives'
~Iron Maiden (circa 1984)
🙏🕊️✌️❤️🔥🪖✊
@l0phty1 @twitskeptic Totally here for the Maiden quote 🤘
@AshTheLibrarian @twitskeptic
Up the Irons! 🤘❤️🔥✊
heh, Maiden early 80s, my high, so very high, school years. probably off by one, or two, on year reference but yeah, Maiden.. got to see them three times, Tingley Coliseum, Albuquerque.. had quite the concert tee collection from the stop every band seemed to make back then. then caught Judas Priest first year Army, in Hawaii 87, then Alanis Morrissete😍 and Greenday couple times by relocate in 92.. then fell out of concert going, cept Pink Floyd in Raleigh 95 iirc. till after retirement in 08 but sadly, can't recall a good thing since my return home to NM other than Santana in 2010, cept maybe Dakha Brakha at Taos brewery in 2014..
wow, what a trainwreck above.. I'm just gonna post this trip down memory lane anyways.. lol 🤘
@ct_bergstrom @emilymbender Demo here: https://huggingface.co/spaces/katielink/biogpt-large-demo