Mill Screen

Unstructured comment in, typed probabilistic decisions out. Mill reads a comment once and returns six independent probabilities (toxic, severe toxic, obscene, threat, insult, identity hate) for text in English, Russian, Turkish, Spanish, French, Italian or Portuguese. It is a System One model: a fast, single-pass decision you call as a function inside a moderation pipeline, not a chat model you prompt. Under the hood it is XLM-RoBERTa-large (565M parameters) with a language-conditioned attention block, trained on a 360k-comment Jigsaw-derived corpus, with Mistral-7B generating extra examples for the rare threat and identity-hate labels.

On its 35,658-comment held-out test split, Mill ranks toxic comments better than every public multilingual model, in every language: toxic AUC 0.992 against 0.969 for Detoxify multilingual, 0.933 for textdetox XLM-R large and 0.851 for citizenlab mDistilBERT. The margin is widest outside English (Russian: 0.992 vs 0.922), and on English it ties Detoxify's original toxic-bert. It runs at 871 comments a second, 11.8 ms each, the same speed class as other XLM-R-large models. One honest caveat: Mill trained on this test set's distribution and the baselines did not.

Mill grew out of a 2025 toxic comment classifier. A 2026 audit found that model had never fine-tuned its encoder; fixing that and retraining took macro F1 from 0.60 to 0.88, and that retrain became Mill.

• One pass, six independent probabilities per comment (labels can co-occur)
• Seven languages in one model, with per-class thresholds tuned on validation
• Toxic AUC 0.992 across all 7 languages, ahead of Detoxify multilingual (0.969), textdetox XLM-R large (0.933) and citizenlab mDistilBERT (0.851)
• 871 comments/s, 11.8 ms each (one Quadro RTX 6000, fp16)
• Held-out test: macro AUC 0.985, macro F1 0.88, exact match 88%
• Released on Hugging Face with per-class thresholds and a model card that documents known bias

• Comparing fairly with public models that predict different label sets, none of them all six labels across languages.Scored every baseline as published on the same 35,658 test rows, on the one label they share (toxic), with paired-bootstrap 95% confidence intervals; the six-label comparison is English-only, against toxic-bert. The headline is AUC, which needs no threshold, so Mill's tuned thresholds give it no edge.
• The 2025 model behind Mill never fine-tuned its encoder: freezing the embeddings while gradient checkpointing was on stopped gradients reaching all 24 layers.Fixed it and retrained: 54% of parameters now learn, up from 0.8%, macro F1 rose from 0.60 to 0.88, and threat F1 doubled (0.42 → 0.84).
• Identity-term bias: short, benign sentences that name an identity group are over-flagged as toxic, a pattern inherited from training data where those terms appear mostly in abusive contexts.Quantified it with templated probe sentences across all seven languages and disclosed it on the model card, with a costed mitigation: roughly 600 counter-examples and one retrain.