Why every AI clip scores 85: the score inflation problem

If a tool hands you ten clips and scores them 88, 87, 85, 89, 84, 86, 85, 83, 87 and 86, it hasn't ranked anything. It's produced ten numbers that happen to be in the eighties.

We know because that's exactly what ours did.

The actual data

A real production run on a 14-minute talk, ten clips generated:

Metric	Range across all 10 clips
Overall virality score	83 – 89
Hook score	85 – 91
Flow score	82 – 87
Value score	83 – 90
Trend score	75 – 82

A six-point spread on the headline number. The gap between the "best" and "worst" clip in that batch was smaller than the noise you'd get re-running the same clip twice.

The scores were decoration. You still had to watch all ten.

Why this happens

It isn't a bug so much as the default behaviour of asking a language model to rate something.

Models are agreeable. Asked to score content on a 1–100 scale, they cluster high. Handing back a 34 feels like criticising the user's work, and models are trained hard not to do that.

Relative comparison leaks in. Given ten candidate moments from one video and asked to score each, the model implicitly ranks them against each other. But it already filtered these from the whole transcript as the best moments, so it's scoring the top decile against itself and, naturally, everything looks good.

"Rate 1–100" has no anchor. 100 compared to what? The best clip in this video? The best clip on TikTok? Without a reference, the model picks a comfortable band and stays there.

What we changed

Three things, all in the scoring instructions:

1. Explicit bands with an honest floor. Instead of "score 1–100", the rubric now defines what each range means, 85–100 as genuinely exceptional and rare, 40–64 as "watchable but unremarkable, this is where most clips honestly land", and below 39 as weak. Crucially it states that a video may contain no clips above 85.

2. An explicit ban on relative ranking. The instruction now says: do not rank these against each other, score each on its own merits against everything on the platform. If the best moment in a boring video is a 45, say 45.

3. Consistency between the overall score and its parts. The overall number has to be defensible against the four sub-scores: no returning an 85 overall when hook and value are both 50.

Did it work?

Partially, and I'd rather report that accurately than claim a clean win.

Re-running the same video, the spread widened from 83–89 to 79–91, with sub-scores ranging 75–94. Better separation, and the ordering became more useful, but still clustered higher than the rubric asks for.

The honest read: that source was a well-known talk with genuinely strong moments, so high scores may be correct. But I'd expect a mediocre video to still score higher than it deserves. Score inflation is reduced, not solved.

What to do with a score, whatever tool produced it

• Use it for ordering, not as a grade. Which to post first is a useful question. "Is 84 good?" isn't.
• Read the reasoning, not the number. Ours returns a sentence on why each moment was picked. A specific reason ("the guest names a figure the host clearly didn't expect") is worth more than any integer. A generic one ("engaging content with strong appeal") means the model had nothing.
• Distrust a narrow spread. If every clip from a video lands within a few points, the scoring isn't discriminating. Treat the list as unordered.
• Check the sub-scores diverge. Hook and value measuring different things should sometimes disagree. If they move in lockstep, they're one number wearing four hats.

The broader point

A number with no visible reasoning and no honest floor is a progress bar that always says 90%. It looks like information and isn't.

We'd rather show you a 45 with a reason than an 85 with none: the first is useful and the second is decoration. Related: why AI clips cut mid-sentence covers the other quality problem we measured and fixed.