In what world is it acceptable to have a product whose behavior is not reproducible at all? You have no idea what the training data is, what the evaluation data is, y'all write papers about the system "learning" this or that, when your test set might be part of its training set. And these companies can't provide any guarantees for what the output will be for a particular input, and the ways in which it will change, if the output is different for the same input.

Follow

@timnitGebru
Perhaps there should be more funding for fully reproducible models like LLM360-Amber and LLM360-Crystal.

arxiv.org/abs/2312.06550

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.