Martin Ruskov @mapto@qoto.org

The case of “vegetative electron microscopy” illustrated here shows what is badly needed in current #LLM research and has implications far beyond. We need tools that help us curate huge corpora. We need to be able to trace #hallucinations back to the training data and understand what are the specific (to a surprise, often #deterministic) reasons in the model input that cause that particular output.

If anyone is interested in collaborating on this, I'm in, have done some small-scale experiments and have already submitted a grant proposal.
https://theconversation.com/a-weird-phrase-is-plaguing-scientific-papers-and-we-traced-it-back-to-a-glitch-in-ai-training-data-254463

I think I now know where to draw the line between "good" and "bad" #GenAI, and possibly (or rather obviously) the same for #machineLearning. It's simply whether the input data has been constructed rigorously. Put this way it's the most obvious statement ever, but somehow #BigTech have convinced us all that they advance research by recklessly scraping #twitter, #4chan and who knows what else (they keep their training data secret).

What is good science in computational linguistics? Well, open data is a step towards it. But open and crap is not a solution. We need to actually _know_ and manage the data. And nobody in their right mind would want to plough through toxic data to clean it. We've all heard the horrors of Kenyan data workers who do it for money and still suffer doing it.

But better (yes, also smaller) corpora are of interest to scholars in the humanities and the social sciences. Think of https://textcreationpartnership.org or https://mlat.uzh.ch. Yes, they are too big for individual researchers or even teams to handle, but we have the organisational and technological infrastructure to work on them collectively. We've been doing it for ages and we will continue doing it. We just need to do it together.

And this is the goal of the European Research Council project proposal I'm submitting in this very moment.

Today at #CHR2025, I will be presenting our work on the evaluation of the historical adequacy of masked language models (MLMs) for #Latin. There are several models like this, and they represent the current state of the art for a number of downstream tasks, like semantic change and text reuse detection. However, a historical researcher, philologist or else would want to be sure that such models really represent the historical period of interest. For example, it would be an embarrasing hallucination if St. Augustine showed up in the context of the Roman senate.

Our evaluation confirms a known problem: LLMs and masked models in particular are trained on corpora without attention to historical periods. Unlike other research we've done on Early Modern English, this problem leads to models being barely distinguishable when it comes to their ability to generate based on a historical period. Even though history is a case where it is most obvious when models go wrong, this type of contamination is a known problem for LLM training overall, think of different legal jurisdictions using the same language, dialects in programming languages, etc.

This research was generously supported by AgileLab.

The full paper is available at:
https://anthology.ach.org/volumes/vol0003/the-latin-language-evolved-over-time-masked-models/

Our paper on the values found in fairy tales from some European countries has been published. We studied how values are explicitly present in tales from Germany, Italy and Portugal using various NLP techniques, but most notably Word2Vec and Word Embedding with a Compass. We visualise synchronic semantic variation to show certain differences based on observations of the corpus, some of them already observed in previous literature. A discussed example in our findings is how motherhood in Germany is strongly related to generosity, whereas in Italy and Portugal it has stronger relationship to wisdom.

Fulltext available at: https://aclanthology.org/2023.nlp4dh-1.8/

@folklore @linguistics @bookstodon

In the morning session today Sara Sullam and I will be presenting our work on exploring nominal (in our case study - bibliographical) data. We do it by borrowing a method from educational research - the notion of phenomenographic variation. #CHR2023🧵

An outstanding article, and it's not about AI. It's about added value and doing business:

"A marketing manager with no engineering background opens Cursor on Monday morning. By Wednesday afternoon, she has a working customer-facing app. It looks polished. It performs the core task. She demos it to her VP, who forwards it to their CMO, who then shows it in the executive staff meeting as evidence that the team is “moving at AI speed.”

By Friday, it is in front of customers.

No one asked who owned the decision to ship it. No one tested it against the conditions it would actually face. No one had the cultural standing to say this looks great, and we are not putting it into production. The prototype became a product because the organization had no system for telling the difference.

I watched a version of this scenario play out recently in a boardroom. A senior executive demoed an AI-built internal tool. The room admired the speed. What received less attention were the harder questions: Who would own it after launch? Who would maintain it? And what would happen when it produced an answer that was confidently wrong?

This is what vibe coding is about to expose across businesses. The companies that think the story is about software are going to lose to the companies that understand the story is about judgment."
https://www.forbes.com/sites/jasonwingard/2026/04/23/vibe-coding-will-break-your-company/

Sounds too much like the Kremlin:
"Around the same time, White House chief of staff Susie Wiles forced a meeting of Trump’s most trusted advisers. The problem: No one was being honest with the president about the domestic impact of the war.

Privately, Wiles had expressed fears that the inner circle’s rose-tinted retelling of the conflict would leave Trump oblivious to the political reality of the war, just months ahead of a contentious midterm season, reported Time magazine earlier this month."
https://newrepublic.com/post/209262/donald-trump-iran-war-plans-screaming-aides

"Among those monitored were a Palestinian academic invited to give a guest lecture at Manchester Metropolitan University and a pro-Gaza PhD student at the London School of Economics, according to internal documents.

In October 2024, the University of Bristol provided the firm with a list of student protest groups it wished to receive alerts about, an internal university email suggests. It included pro-Palestinian and animal rights activists.

In total, 12 universities paid the firm to monitor campus protest activity. Others include the University of Oxford, Imperial College London, University College London (UCL), King’s College London (KCL), the University of Sheffield, the University of Leicester, the University of Nottingham and Cardiff Metropolitan University."
https://www.aljazeera.com/news/2026/4/20/uk-universities-pay-to-spy-on-students-social-media-accounts

“Someone who says ‘I’m against abortion but says I am in favor of the death penalty’ is not really pro-life,” Leo said. “Someone who says that ‘I’m against abortion, but I’m in agreement with the inhuman treatment of immigrants in the United States,’ I don’t know if that’s pro-life.”
https://www.huffpost.com/entry/pope-leo-breaks-down-why-maga-is-not-really-pro-life_n_68dd802ae4b0c450ba64c434

One of the core problems of GenAI is that it's trained on junk data no one has ever read and reviewed. Once again we see that's a lesson not learned, because in their endless quest to more data for more-of-the-same models, GenAI companies have found a new source of mediocre training slop: work-related chats.
https://gizmodo.com/failed-companies-are-selling-old-slack-chats-and-email-archives-to-train-ai-2000747916

What are they thinking? Have they never participated in such conversations not to know that in the context of remote-first work, these are the equivalent of watercooler conversations? Noise is the norm there, and transformer models are supposed to filter this out? All this without considering the survivor bias of failed companies (nice pun).