the "parrot" analogy isn't really seeping into a lot of people

yes, #ChatGPT is fundamentally a very large pile of data, and all it's doing is basing its responses on what's included in the dataset

but what feels like a better example is understanding what that dataset is, and what it means, and why it's bad

chatgpt, conceptually, generates the top-rated reddit comment for a post that you write.

is this likely to have well-formed English prose with proper capitalisation and punctuation? yes.

is this likely to seem reasonable and/or impressive and occur in a short amount of time? also yes.

is this likely to be correct? maybe!

and of course, using reddit to answer all of your questions could be very useful! after all, there are loads of very clever people on reddit, and the nuggets of humanity's sum total knowledge are located, probably, somewhere within there.

reddit has rules and moderators, and we can usually make sure that the top comment isn't total nonsense. sure, a few things might slip through the cracks, but it can't be that bad.

but do we feel like posting a question on reddit and taking the top comment verbatim is a revolutionary idea with absolutely no downsides, and that we just need to hire more moderators and add more rules to avoid all those downsides? absolutely not.

remember, chatgpt is based upon a very large dataset, and that's all. this dataset is so unfathomably large that it cannot possibly be curated, and while its authors can add exceptions to its rules, it would be incredibly dishonest to imply that it can be controlled by these rules. much like how reddit is "the front page of the internet" and has all sorts of stuff on it, truly having a wealth of information. moderators make sure that bad information, for some definition of bad, doesn't come to the surface.

but… would you hold redditors to the highest regard? would you describe your reddit-based chat bot a revolutionary technology? absolutely not. it's only "revolutionary" that they could afford the resources to convert reddit into a chat bot in the first place.

unsurprisingly, those who wish for extremely large financial returns on the success of chatgpt describe it as a solo actor rather than what it's actually closer to, an aggregator of internet comments. we've spent decades coming to terms with how the internet can be a great source of information and also, you shouldn't trust everything you believe on the internet, and also, the internet is useless without any form of curation.

(and to be fully clear here: reddit is not the only source for chatgpt. the only real information we know about its dataset is that it comes "somewhere" from the vast repositories of human information, and "reddit" serves as a pretty generic analogy for this.)