The way I think about “AI” as a set of technologies is that they are probabilistic tools. Where these tools, at least on a technical level, fall short is when they’re applied to things that require deterministic answers. Or maybe are probabilistic, but are high-consequence so you need people to make decisions.
Anyway, just read this article about making AI hallucination-proof for clinical settings — and at least from a technical perspective this seems like it might work
The way I understand the work here is that each step of the way the way this system essentially goes: “citation needed”.
The initial categorization of symptoms must always link back to actual diagnostic records. The secondary linking of these symptoms to potential diagnoses must be entirely based off of source material, with every statement being backed by actual quotes. With a third pass going back to make sure that those statements also relate back to the patient data.
@yosh However, as I understood it, the piece in the system which verifies that the provided "citation" proves the fact the LLM is trying to use is a second LLM. So what protects the system from the second LLM hallucinating?
Probably anything which involves matching strings of text will be fine, but what if it has to do some sort of computation, such as "number of Rs in strawberry"?
