The thumbnail decision log: a template for lessons, not scores

A CTR figure expires the week it is read. A lesson written properly is still true when you brief the next cover, and the log is where the difference is made.

By the Thumbnail Bench team Published 10 min read
A spec card showing one completed decision log entry with its video, hypothesis, variants, method, result, lesson, confidence and next use.
On this page
  1. Why a lesson compounds and a score does not
  2. The template
  3. How to write the lesson: a grammar
  4. The confidence ladder
  5. Log the decisions that were not tests
  6. The quarterly review
  7. Keeping the log alive
  8. What to do next

A thumbnail decision log records, for every cover decision, what you expected, what you changed, what happened and what you will do differently next time, in a form the next brief can use. A spreadsheet of click-through rates is a different document, and a much less useful one. The A/B testing guide ends on the rule "write the lesson, not the score"; this is the template that makes the rule easy to follow, a grammar for the lesson itself, worked examples of entries that hold up months later, and the quarterly review that turns a page of lessons into a short list of house rules.

Why a lesson compounds and a score does not

A score is a number tied to the conditions that produced it. A CTR of any size belongs to one traffic mix, one video age, one week of the calendar and one row of neighbours, and none of those will be the same next month. You cannot compare it with the next score without rebuilding all four conditions, which is why a baseline has to be split by source and age before any single figure means anything. A score answers "how did that cover do" and the answer is already out of date.

A lesson is a statement about the viewer. "On reviews, doubt beats shock" does not need the conditions rebuilt; it is applied by choosing doubt on the next review. Two lessons about the same lever either agree, and you have a rule, or disagree, and you have a condition to find. That is what compounding means here: each entry makes the next brief faster, and the log gets more useful as it gets longer, where a list of scores gets longer and no more useful.

Our view

Our view is that a channel's log is worth more than its analytics history, because the analytics describe what happened and the log describes what the channel learned. Two people with the same analytics and different logs will make different covers, and the one with the better log will make them faster.

The template

Ten fields, one row per decision. The width is deliberate: a narrower log loses the fields that make a lesson reusable, and a wider one does not get filled in.

FieldWhat goes in itExample
Date and videoWhen, and which video by working title2026-09, "Air fryer, one month"
FormatThe video type the lesson applies toReview
Decision typeTest, manual swap, refresh, judgement call, or not testedTest
HypothesisFormat, lever, expected direction, viewer reason, written before the resultOn reviews, doubt beats delight because the viewer wants a verdict
VariantsWhat changed and what stayed, both halves if a title was involvedA doubt face, B delight face; same headline, layout, colour, title
MethodTest & Compare, alternating swap, concept run, or noneTest & Compare, two slots
ResultThe result state, the manual gap, or "shipped A"Winner: A
LessonOne sentence in the grammar belowDoubt beats delight on reviews; the viewer wants a verdict, not a fan
ConfidenceHint, lean, rule, or retiredLean (first Winner)
Next useThe specific next decision this changesDoubt on the next two reviews; retest against calm after that

A blank row to copy:

Date and videoFormatDecision typeHypothesisVariantsMethodResultLessonConfidenceNext use

Two fields do most of the work. Hypothesis is written before the test, which is the only thing that stops the entry being a story told in hindsight; the variant ladder gives the sentence shape for each kind of test. Next use turns the lesson into an instruction with a video attached, which is the difference between a log that is read and a log that is admired. An entry with an empty next-use field has not finished.

How to write the lesson: a grammar

A lesson has five parts, in this order: the lever's winning side, the losing side, the format, the condition if there is one, and the confidence. "Doubt beat shock on reviews when the product was expensive; Winner." Every part narrows where the lesson applies, and a lesson that leaves out the format or the condition will be applied where it is false.

A matrix of four log entries, each shown first as the score a creator usually writes and then as the lesson the grammar produces.
The left column is true and useless; the right column can be applied to a cover that does not exist yet.

The table below rewrites the entries we see most often. The left column is what gets written; the right is the same result in the grammar.

Written as a scoreWritten as a lesson
"B won""The number headline beat the word headline on finance stories; the picture could not show the stake; Winner."
"New style did better""Object-led beat face-led on unboxings; the viewer was checking the product; Preferred, retest once."
"No difference""Expression did not move tutorials; calm and shock performed the same; stop testing emotion on this format."
"CTR went up after the swap""Manual swap on an evergreen how-to: the specific title beat the general one by a wide gap across both pairs of windows; the week held no holiday; lean."
"Kept the old one""Not tested: video under a week old and impressions still climbing; judged too early to swap; revisit at day 28."
"Package B won""The method promise beat the stakes promise on debt stories; two packages, so the half is unknown; wedge next time."

Some habits keep entries honest. Write "Performed the same" entries as findings, because a lever that does not matter is a lever you can stop spending time on. Write manual swaps with the calendar named, because a swap that straddled a seasonal edge is not a lesson about the cover. And write package tests as lessons about the promise, not the picture; how a title-and-cover test is read sets out why crediting the cover after a package win is the most common false entry.

Take a hypothetical entry from start to finish. The video is a budgeting explainer. Variant A is a calm face on a dark green ground with $0 LEFT in yellow at a third of the height; variant B is the same face, ground and layout with MY BUDGET in the same yellow. The hypothesis, written before the test: on budgeting explainers, a stake headline beats a subject headline because the viewer wants a consequence, not a topic. Test & Compare returns Winner: A. The score entry reads "A won", which tells the next brief nothing. The lesson entry reads: "The stake headline beat the subject headline on budgeting explainers; the viewer wanted a consequence; Lean." Next use: stake headline on the next two budgeting videos, then retest against a number-plus-noun headline. The two entries describe the same result. Only one of them can be handed to someone else.

Thumbnail Bench recommends

Write the lesson for the person briefing a cover in six months, who may not be you and will not remember the test. If the sentence needs the video open to make sense, it is not finished. If it could be pasted into a brief as an instruction, it is.

The confidence ladder

Confidence is a field, not a feeling, and it moves only when evidence arrives. Four levels are enough.

LevelWhat earns itWhat it permits
HintOne manual gap, one Preferred, or one concept run that leaned one wayTry the direction on the next video; do not brief it as a rule
LeanOne Winner, or two hints in the same direction on the same formatUse it by default on that format; keep one retest scheduled
RuleThe lean repeated on the same format, or a Winner plus an agreeing hintBrief it without testing; move testing to the next lever
RetiredA rule that stopped holding, or a lever the format no longer variesNote why; keep the entry so nobody re-runs the old test

A lesson climbs one level per agreeing result and drops one per disagreeing result. It never jumps to rule on a single test however clear the Winner, because one video's audience is one audience; the small-channel method of running a concept across several uploads is the fastest way to earn the repeat. And a rule is not permanent. Audiences habituate, the niche converges, and a rule that was true for a year can retire; the quarterly review below is where that happens.

Log the decisions that were not tests

Most cover decisions are never tested, and a log that only holds tests hides where a channel's habits live. Every upload is a decision. So is every swap on a live video, every refresh of an old cover, and every time you chose not to test. Give each of them a row with the decision type filled in and the hypothesis written as the reason.

  • Judgement call: the cover was shipped without a test. Hypothesis is why this cover; next use is what would make you test it.
  • Manual swap: the windows, the source, the gap, and the calendar note. Whether the swap should have happened at all is decided before the row, not by it.
  • Refresh: which old cover, what changed, and the source-matched comparison a month later.
  • Not tested: the reason. Too young, too few impressions, ineligible, or no hypothesis. The last one is the useful entry, because a run of "no hypothesis" rows on one format tells you where the next test belongs.

We would rather have a log with thirty honest "not tested: no hypothesis" rows than one with ten tidy test entries, even though the first looks like a channel that never tests and the second looks like rigour. The thirty rows show which format runs on habit. The ten hide it.

Observed pattern

When channels start logging untested decisions, the same discovery comes up: the format they upload most is the one they test least, because its covers are made on habit. The log makes the habit visible, and the first test on that format is usually the most valuable one the channel runs that year.

The quarterly review

Once a quarter, read the log as a whole rather than a row at a time. It takes an hour and it is where lessons become rules. The weekly review feeds this one; the weekly loop writes rows, the quarterly loop reads them.

A five-step flow for the quarterly log review: gather the quarter's rows, group them by lever and format, move confidence up or down, find the untested levers and formats, and write the house rules page.
The output of the review is a one-page list of rules with dates, not a longer log.

  1. Gather. Every row from the quarter, including the untested ones.
  2. Group by lever and format. All the expression rows for tutorials together, all the headline rows for reviews together. Patterns hide across rows and appear in groups.
  3. Move confidence. Promote lessons that repeated, demote lessons that were contradicted, retire rules that stopped holding. Write the date beside each change.
  4. Find the gaps. Which formats have only judgement rows? Which levers have never been tested on your biggest format? Those are the next quarter's hypotheses, and they go into the log now as planned tests.
  5. Write the house rules page. One line per rule, with the format, the date it was promoted and the lever it settles. This page, not the log, is what the brief reads. It should fit on one screen, and if it grows past that, some rules are conditions of other rules and should be merged.

Rebuild the CTR baseline in the same sitting, because the channel's mix drifts over a quarter and the baseline is what every manual entry in the next quarter is read against.

Thumbnail Bench recommends

Retire rules on purpose, not by forgetting them. A rule that has not been retested in a year is a belief. Put one retest of your oldest rule into each quarter's plan; if it holds, the date moves and the rule stays. If it does not, you have caught habituation before it cost a quarter of covers.

Keeping the log alive

A log dies when it becomes a chore or when nobody reads it. These keep it in use.

  • One row per decision, written the day the result arrives. Not the day the test starts, when the hypothesis is written, and not a week later, when the reason has faded.
  • Read it at brief time. The research step of the five-step thumbnail workflow starts with the house rules page and the log's rows for this format, so the brief begins from what the channel already knows.
  • Keep the runner-up. Every test's losing variant is the refresh candidate for that video, and the next-use field can point to it by name. On the bench the history strip and versions keep every render, so "next use: the calm-face version from the history" is an instruction someone can follow in a minute rather than a rebuild. Tweak makes the retest variant from the winning render without touching anything but the lever.
  • One owner. On a channel with an editor or an agency, the log has a single person who writes rows and runs the review, and everyone reads the house rules page.

What to do next

  • Copy the blank row into whatever you already use for planning, and fill in the last three cover decisions from memory, untested ones included. Notice which fields you cannot fill.
  • Write the hypothesis for the next upload before the cover is made, in the grammar, with the "because".
  • Put the first quarterly review in the calendar now, and plan one retest of whatever you currently believe most firmly about your covers.
Thumbnail Bench team

We build an AI thumbnail maker and spend our days looking at what earns clicks in the feed. Thumbnail Bench was created and is run by Tim Schroeder, a YouTube creator of several years; the founder notes are his. Everything here is written by the team, checked against YouTube's own documentation where a claim can be checked, and labelled as observation or opinion where it cannot. About us.

Ready to go viral?
Make your first thumb.

4 free credits, 4 more after your first render. No card. Your first options in about a minute.