@Gargron I guess that depends on just how much effort you want to put into it. Naive Fisher Classifier or
Bayes classifier after running through a Stemmer would be the simplest that comes to mind while still being very effective.

@freemo Well, any classifier needs training data. We don't have any because the reports system neither has a "spam" class nor makes an effort to keep it when admins decide to ban the spammer.

The second problem is stemming is language-dependent, Mastodon is used all across the world, and our language detector is unreliable enough as it is.

@Gargron Yea I guess if you want to avoid any training period then those wouldnt be good choices. But still there are unsupervised classifiers you could pick.

Stemming is language specific but if you use a library that shouldn't be a problem, presuming you can detect the language reliably. If not you can fall back to not stemming i guess. But even with your current approach I suspect you'd see an improvement with stemming.

Personally what I'd suggest is a combination of supervised and unsupervised. Unsupervised being a first-pass, then as users block certain messages over time the supervised learning algorithm would improve and make suggestions.

Anyway thats my 2 cents.