YouTube thumbnail A/B tests: what 460 real Test & Compare runs show

A YouTube thumbnail A/B test (YouTube calls it Test & Compare) is rare, short and close to a coin flip. I found 460 tests among 828,766 videos from served creator channels in the TubeAI database, published between April 10 and September 28, 2026. 93% of them tested thumbnails only, and 92% compared just two versions. Once the thumbnails started switching, the median test stopped after 20.5 hours. When it stopped, the original packaging was still showing in 45% of cases.

This post covers the tests themselves: how often creators run them, what they test, how long a rotation lasts and which version survives. For what a plain thumbnail swap (no test) does to views, see what 7,418 thumbnail swaps show.

How often do creators A/B test thumbnails?

Not often. Of the 828,766 videos I tracked, 75,998 had their title or thumbnail changed at least once in the first week. Only 460 of those changes followed the back-and-forth pattern of a Test & Compare rotation. The rest were one-way edits: a creator replaced the packaging and left it.

Bigger channels test more. Channels with 1M+ subscribers were about 10 times as likely to have a detected test as channels under 10K. These rates are a lower limit, not true adoption (see how I measured this at the end), but the size gap holds across every band.

What a thumbnail A/B test looks like from the outside

A test can't be seen directly from outside YouTube. What can be seen is which thumbnail and title the video is showing each time it's checked. A normal edit shows up as one switch: A, then B, then B for good. A Test & Compare rotation looks different. The video goes back to a version it had already dropped, more than once, and then stops on one version.

What gets tested

Thumbnails, almost always. 430 of the 460 tests changed only the thumbnail, 11 changed only the title, and 19 changed both. 424 tests compared two versions, 29 compared three, and 7 mixed titles and thumbnails into four or five combinations.

How long a test lasts

The first switch I saw came a median 32 hours after publishing (half of all tests started between hour 7 and hour 63). 39% had started within the first day and 66% within two days. Once switching began, it didn't last long: across the 442 tests that settled, the median rotation lasted 20.5 hours from the first switch to the last. The middle half lasted between 11 and 31 hours, and 291 tests (66%) were done within a day. A typical test switched versions 7 times, and 1 in 10 switched 15 times or more.

18 tests were still switching when the 7-day tracking window closed. I report those separately and leave them out of every outcome figure below.

Does the original thumbnail win?

Slightly less than half the time. In 201 of the 442 settled tests (45.5%), the video ended on its original packaging. In the other 241 (54.5%), it ended on a version added later. If you look only at the most common setup, two thumbnails and the same title, the result is almost exactly 50/50: 197 kept the original and 202 kept the challenger.

This result matters. Creators usually put their best idea into the first thumbnail and treat the second as a backup. Yet the backup survived just as often. If you can't reliably predict which of your own thumbnails will win, that's the case for running the test at all.

Do A/B tested videos get more views?

A little, but not in a way that suggests the test made the difference. I compared tested videos with every other video on the same 382 channels, using the outlier score (a video's views relative to the channel's usual level, where 1.0x means a normal video). I only included videos at least two weeks old.

Tested videos landed a few points above the channel's normal level (1.06x to 1.09x vs 1.01x), and about one in five reached 2x. That's a small gap, and the direction of cause is unclear. Creators may choose to test videos they already expect to do well, and a video that is getting lots of impressions gives YouTube enough data to finish a test. The 17 videos still switching at day 7 did best of all, but 17 is too few to draw conclusions from.

What the data doesn't show: a test turning an ordinary video into a breakout. Tests where the new version won and tests where the original won ended within 0.03x of each other.

What this means if you're about to run one

• Expect a coin flip. In two-thumbnail tests, the original and the new version each survived about half the time. Test two genuinely different ideas rather than two color versions of the same one, because only a real difference is worth learning from.
• Start with the thumbnail. That's what 93% of testers did. Your title is a separate decision. See the best YouTube title formats and how long a title should be.
• Plan around the first days. Most rotations started in the first two days and were done within a day of the first switch, so the thumbnail you live with long term is usually decided early.
• Don't expect the test to fix the video. Tested videos finished close to their channel's normal level. A test picks the better of your options. It doesn't make up for a weak idea.

How I measured this

The TubeAI database records a video's title and thumbnail during its first week and logs every change. I counted a video as a Test & Compare rotation when its log showed at least two distinct title-and-thumbnail combinations, went back to a combination it had already dropped at least twice, and never used more than three titles or three thumbnails (YouTube's test limits). A single edit, or a chain of edits that never goes back, doesn't count.

A test counts as settled when one version held for at least 24 hours before the 7-day window ended. The population is videos from served creator channels (approved, classified as real creators) published between April 10 and September 28, 2026. Switches are only caught when the tracker happens to see the alternate version, so some real tests are missed. All counts here are therefore a lower limit.