One of the decisive moments in my understanding of #LLMs and their limitations was when, last autumn, @emilymbender walked me through her Thai Library thought experiment.
She's now written it up as a Medium post, and you can read it here. The value comes from really pondering the question she poses, so take the time to think about it. What would YOU do in the situation she outlines?
@ct_bergstrom @emilymbender Quite a few of the commenters here are equating Bender's thought experiment with Searle's famous Chinese Room experiment. I think this is incorrect .
I think Bender is trying to answer the question: is it possible to derive the meaning of symbols without understanding the external representation of those symbols? Suppose you locked a newborn baby in a Skinner box (not recommended) and only allowed the child to listen to English audiobooks. Would that child learn to understand the meaning of the words it hears?
Searle's thought experiment, on the other hand, is designed to answer the question of whether is it possible to assign language understanding to a program that can accurately translate a language.
