Just as the community adopted the term “hallucination” to describe additive errors, we must now codify its far more insidious counterpart: semantic ablation.
Semantic ablation is the algorithmic erosion of high-entropy information. Technically, it is not a “bug” but a structural byproduct of greedy decoding and RLHF (reinforcement learning from human feedback).
During “refinement,” the model gravitates toward the center of the Gaussian distribution, discarding “tail” data – the rare, precise, and complex tokens – to maximize statistical probability. Developers have exacerbated this through aggressive “safety” and “helpfulness” tuning, which deliberately penalizes unconventional linguistic friction. It is a silent, unauthorized amputation of intent, where the pursuit of low-perplexity output results in the total destruction of unique signal.
When an author uses AI for “polishing” a draft, they are not seeing improvement; they are witnessing semantic ablation. The AI identifies high-entropy clusters – the precise points where unique insights and “blood” reside – and systematically replaces them with the most probable, generic token sequences. What began as a jagged, precise Romanesque structure of stone is eroded into a polished, Baroque plastic shell: it looks “clean” to the casual eye, but its structural integrity – its “ciccia” – has been ablated to favor a hollow, frictionless aesthetic.
This is a good name for one of the main reasons I’ve never really felt a desire to have an LLM rephrase/correct/review something I’ve already written. It’s the reason I’ve never used Grammarly, and turned off those infuriating “phrasing” suggestions in Microsoft Word that serve only to turn a perfectly legible sentence into the verbal equivalent of Corporate Memphis.
I’m not a writer, but lately I often deliberately edit myself less than usual, to stay as far as possible from the semantic “valley floor” along which LLM text tends to flow. It probably makes me sound a bit unhinged at times, but hey at least it’s slightly interesting to read.
I do wish the article made it clear if this is an existing term (or even phenomenon) among academics, something the author is coining as of this article, or somewhere in between.
GPT-4o mini, “Rephrase the below text in a neutral tone”:
“avoid the typical style associated with LLM-generated text” – slop!
That’s a fine illustration of the problem, whatever it’s properly called.
Having paused to search the web I find that “ablation” according to wikipedia is a term used in AI since 1974. Arxiv.org has a recent paper talking specifically about “semantic ablation” which phrase it uses to describe an operation deliberately removing semantic information from an LLM’s representation of a sentence in an attempt to see what purely syntactical information is left over afterwards, or something like that.
Interesting, thanks for doing the research!
As an extreme non-expert, I would say “deliberate removal of a part of a model in order to study the structure of that model” is a somewhat different concept to “intrinsic and inexorable averaging of language by LLM tools as they currently exist”, but they may well involve similar mechanisms, and that may be what the OP is referencing, I don’t know enough of the technical side to say.
That paper looks pretty interesting in itself; other issues aside, LLMs are really fascinating in the way they build (statistical) representations of language.