Arcane Marketing logoArcaneApply
← All posts

A buyer launches four new ads on a Thursday. By Monday two of them sit above target cost per lead, so they get paused. By Friday the weekly report has a winner in it and a line saying creative testing is working. Nothing that week produced an answer. It produced an opinion with a spend figure attached to it.

Most of what gets sold as an ad creative testing framework is that loop with a spreadsheet around it. The repair work happens before launch rather than in the reporting: decide what the test can resolve at your actual conversion volume, and write down what you will do with each possible result. The platforms publish most of what you need for that. They also contradict each other, which is the first thing worth knowing.

The platforms contradict each other on duration

Google's Experiments FAQ says "It's recommended that you run the experiment for at least 4-6 weeks or longer if you have a long conversion delay". Google's page on monitoring experiments says "We recommend allowing your experiment to run for 2 to 3 weeks to gather data". One advertising platform, two help pages, a month of difference between them. LinkedIn sets a minimum of two weeks per ad set, with a ceiling of 90 days on a single A/B test. TikTok asks for a minimum of 7 days to obtain the most reliable results.

Those are floors, written for different products with different conversion delays. A seven-day floor applied to a business whose sales cycle runs six weeks will mislead you reliably. The version that travels across platforms is the other line in Google's FAQ: "We recommend you wait for 1-2 conversion cycles". Work out what your cycle actually is, then use the platform floor as a sanity check on it.

What an ad creative testing framework decides before launch

Four things, all settled before anything spends.

The variable. LinkedIn's definition of an A/B test is two ad sets with separate budgets differing by exactly one thing, and it recommends one ad per ad set unless the test itself requires more. Google's Demand Gen documentation lists creatives, audience, product feed and bidding as separate things you can test. A new hook on a new audience with a new bid strategy is a relaunch wearing a test's name badge. You will learn the new thing is better or worse and have no idea which part did it.

The split. Google recommends using 50% to provide the best comparison between the original and experiment campaigns. Uneven splits are usually an attempt to lose less money on the losing arm, and they buy that comfort by making the result unreadable.

The sufficiency threshold, which is the next section.

The decision rule. Write the sentence before launch: if the variant comes in under X at Y confidence it becomes the new control and the next test runs against it, otherwise it dies and the next variable gets tested. Agreeing this after the data lands is how a test turns into a debate.

Volume per arm is the binding constraint

Calendar time is the proxy. The real constraint is how many conversions each arm collects. Google's Demand Gen documentation is blunt about it: with conversion-based bidding, experiments require a minimum of 50 conversions per arm to surface results. Its Smart Bidding guidance for Display asks for at least 30 conversions per month, per ad group if you care about consistently hitting a target. And the monitoring page names its own failure modes in plain language: "Your campaign doesn't receive enough traffic" and "Your traffic split was too small and your experiment isn't receiving enough traffic".

So run the arithmetic before you build anything. Assumptions first: a $20,000 monthly budget in one Demand Gen campaign, a $400 cost per conversion, an even split, and the 50-conversions-per-arm minimum quoted above.

$20,000 at $400 per conversion is 50 conversions a month. Split evenly, each arm collects 25. Reaching 50 per arm takes two months of uninterrupted spend. That test is answerable, slowly, and the duration guidance barely matters to it because the volume sets the clock.

Now change one assumption. Same budget, same split, but a $1,200 cost per conversion because the conversion is a booked call for a high-ticket service. The campaign produces about 16 conversions a month, 8 per arm, and 50 per arm sits six months out. Nobody holds creative still for six months. That test is unanswerable at that spend, and the honest responses are a cheaper conversion event higher in the funnel, or fewer arms with the budget concentrated. The third response is to accept you are making a judgment call and to label it as one in the report. All three are defensible. Reporting a six-week readout as conclusive is not. If your tests keep producing winners that fail to hold up the following month, this is usually what happened, and it is worth reading your reporting like an operator rather than like a scoreboard.

Power is a budget decision, confidence is a per-call choice

TikTok says the quiet part out loud. Its split-test guidance defines power value as the likelihood of detecting potential differences in your ad group and recommends a budget that gives you a power value of at least 80%, with the estimate shown to you before you launch. An underfunded test does not return a slower answer. It returns no answer at all, and the gap gets filled by whoever is most confident in the meeting.

Confidence is the other half, and it belongs to the operator. Google's Demand Gen experiments let you read results at several confidence levels, from a directional default up to a stricter setting intended for high-certainty decisions. The useful habit is choosing the level per decision. Swapping one creative inside a rotation of twelve is cheap and reversible, so directional evidence is enough to act on. Shifting most of the budget to a new angle is neither cheap nor reversible, so it earns the strict bar. Run every call at one threshold and you end up stalled on the small decisions or careless on the large ones.

The learning period is where tests get wasted

Google asks you to wait at least 2 weeks without changes for the initial learning period and warns that performance may be more volatile during this time as the algorithm makes optimization decisions. LinkedIn spells out the consequence of ignoring that: alter or remove ads from the winning ad set and that ad set is no longer considered statistically valid.

Volatility is the trap. Day three is where the eventual loser often looks like the winner, and a daily check-in manufactures pressure to act on it. The discipline here is a calendar entry rather than a dashboard habit. Nobody touches the test until the date you set, and the only legitimate reasons to intervene early are broken tracking or a compliance problem.

Creative volume is the other half of the job now, because the delivery systems do the matching. Meta's engineering team reports that advertisers who had not previously used Advantage+ creative and then switched on its AI-driven targeting features "experienced a 22% increase in ROAS", and estimates that "businesses using image generation are seeing a +7% increase in conversions". Those are Meta's figures about Meta's own automation, so weigh them accordingly. The operating implication survives the discount either way: a framework whose only move is refereeing two ads against each other starves the system. You want a production cadence that keeps genuinely distinct concepts flowing in, plus a formal test reserved for the few questions that would change the plan.

Before the next creative test launches, do the twenty minutes of arithmetic: conversions a month, divided by arms, measured against the platform's own per-arm minimum. If that puts a readable result inside a window you will actually honor, run the test properly and then leave it alone until the date. If it does not, say so out loud and make the call on judgment with the label attached. The expensive failure is rarely a losing creative. It is a quarter's worth of tests that were never answerable, reported as though they were. Campaigns that stall around $10k a month often have a testing log full of exactly that.

Sources

Stuck above the $10k ceiling?

We diagnose your account in 24 hours and tell you straight whether it's a creative, audience, or sales problem.

Talk to us