Ad creative testing for agencies: the platform rules, the budget, and the client report
Ad creative testing works when each test answers one question, has the budget to answer it, and is read once at the end. This guide sets out the testing tools in Meta and Google Ads with the rules from their own help pages, turns a test into a budget and a number of days, and covers running creative tests for clients.
What is ad creative testing?
Ad creative testing is comparing versions of an ad, such as different images, videos, headlines or messages, to find out which produces more of the result you want at a lower cost. Done properly, each test answers one question and the platform splits the audience so the versions are compared fairly.
The pages that rank for this topic, such as AppsFlyer’s and Jon Loomer’s, explain why creative matters and how one testing feature works. This guide sets the platforms’ testing tools side by side with the rules from their own help pages, puts a budget on a test before it starts, and covers what changes when you test creative for a client.
Why creative testing needs a method
Ad platforms now automate much of the targeting and bidding, so creative is one of the things an advertiser still decides directly. That makes testing creative tempting to do casually: launch several ads, let the platform spend on the ones it likes, and call the favourite the winner.
That is not a test. When the platform chooses where to spend, the ads get different budgets, different audiences and different times of day, so the comparison is not fair. Meta’s own A/B testing page says it does not recommend testing informally, such as turning ad sets or campaigns on and off by hand, because it can lead to inefficient delivery and unreliable results.
A method fixes three things before any money is spent: the question, the budget and the way the result will be read. The rest of this guide covers each.
The testing tools in Meta and Google Ads
Both platforms have built-in tools that split delivery fairly between versions. They differ in what they compare and in the rules that come with them.
| Tool | What it compares | Rules from the platform’s help pages |
|---|---|---|
| Meta creative test | 2 to 7 new copies of an ad, inside an existing campaign | Highest volume bid strategy only; Meta suggests no more than 20% of the budget; no confidence level in the results; no automatic changes after the test |
| Meta A/B test | Two versions of a campaign, ad set or ad, with one variable changed | Nobody sees both; at least 7 days for reliable results, up to 30; avoid audiences used by other campaigns at the same time |
| Google Ads ad variations | Changed headlines, descriptions or URLs across Search ads | You set the share of budget for the variation; users are split by cookie, so each tends to see one version |
| Google Ads Demand Gen A/B test | Campaigns that differ in one variable, including creative | Traffic split, with 50% recommended; results shown at a confidence level you choose |
Meta’s creative test
Meta’s creative test is set up inside an existing campaign. You choose an ad, make 2 to 7 copies of it, change the creative in each copy, and choose how much of the campaign or ad set budget to spend across the test ads; Meta suggests no more than 20% to limit the effect on the rest of the campaign. It only works with the Highest volume bid strategy.
Two details matter for agencies. Meta says the results identify the top-performing test ads against your chosen metric, but that a confidence level is not included. And the test makes no automatic changes: the test ads keep running alongside the others on the original budget until you turn them off. Jon Loomer’s walkthrough adds that the ad you start the test from is not itself part of the test, only its copies.
Meta’s A/B test
Meta’s A/B test compares two versions of a campaign, ad set or ad with one variable changed, such as the image, the text, the audience or the placement, and shows each version to a different part of the audience so that nobody sees both. Its best practices say to keep the versions identical except for the variable being tested, to run tests for at least 7 days and up to 30, and to avoid using the same audience for any other campaign at the same time, because overlapping audiences can contaminate the results.
Google Ads ad variations
For Search campaigns, Google’s ad variations let you change headlines, descriptions or final URLs and send a chosen share of the campaign’s budget to the variation. Google splits users by cookie, so each person tends to see only one version however many times they search.
Google Ads experiments for Demand Gen
For Demand Gen campaigns, Google’s A/B experiments compare campaigns that differ in one variable, including creative. Google recommends a 50% traffic split, and the results show whether one arm is better at the confidence level you choose. Its experiments FAQ warns that even with an even traffic split, one arm may spend more than the other, because spend follows how well each arm wins auctions.
Which tool to use
The choice depends on the platform, the campaign type and whether the campaign is already running well.
Yes
Yes
No
No
Yes
No
Whichever tool you use, check its help page before each test. These features change often, and the limits in this guide are the ones the platforms published when it was written.
What a test costs, before you start
Most creative tests that end without an answer were never given the budget to produce one. Each ad needs enough results for a difference to be more than chance, and results cost money.
Enter the account’s own numbers. The defaults are an example, not a benchmark, and the results per ad is your decision, not a platform rule.
With the example figures, which are placeholders, three ads with 30 results each at $25 a result is a $2,250 test. At 20% of a $500 daily budget, $100 a day goes to the test, so it needs 23 days, or 28 in whole weeks. If that is too long, the options are fewer ads, a cheaper result to measure, such as a click or an add to cart instead of a purchase, or a larger share of the budget.
How many results per ad?
Neither Meta nor Google publishes a single minimum. Meta’s best practices say only that a test needs a budget that produces enough results to determine a winner with confidence. Meta’s A/B test and Google’s experiments report a confidence level; Meta’s creative test does not. The number in the calculator is your own decision, and it is worth agreeing with the client before the test, so a result is not argued over afterwards.
The same logic applies to tests on websites, where the traffic maths is set out in the guide to running a CRO testing programme, and the A/B test significance calculator checks a result once a test has ended.
Make ads that differ enough to matter
Small changes produce small differences, and small differences need large budgets to detect. A test of two headlines that differ by one word may never produce a clear answer at an agency client’s budget.
- Format. A video against a static image, or a carousel against a single image.
- Angle. The problem the product solves against the result the customer gets.
- Proof. A customer quote against a number against a demonstration.
- Audience. The same product shown to different kinds of customer, with the creative written for each.
- Offer. A discount against free delivery against a guarantee the client actually offers.
Jon Loomer makes the same point about Meta’s creative test: make the differences between the ads significant rather than subtle, and consider different formats, creative themes, customer personas and messaging angles. Once a broad direction wins, smaller tests within it make more sense.
Write a brief for every test
A brief is one page, agreed before the ads are made. It stops a test being reinterpreted after the results are in.
- The question. What the test will answer, such as whether customer proof beats a product demonstration for this audience.
- The ads. What each one is, and what is the same across all of them.
- The tool. Which of the platform’s testing tools, and why.
- The metric. The one number that decides the test, usually cost per result for the client’s real goal.
- The budget and dates. From the calculator, with the end date fixed.
- The decision. What happens to the winner, and what happens if there is no clear winner.
Choosing that number is part of choosing which marketing metrics tie to revenue. Clicks and click-through rate are useful for diagnosing why an ad won or lost. They should not decide the test when the client is paying for leads or sales.
Name every ad so the results can be read later
A test log is only useful if each result can be traced back to what was tested. That depends on naming: if an ad is called Copy of Copy of Ad 3, no one will know in six months what it was.
A naming convention puts the test, the variable and the version into every ad’s name, for example the client, the test number, the variable tested and the version, in a fixed order. The same convention should run through the campaign, ad set and ad names and through the UTM parameters, so the ad platform and analytics describe the test the same way. The campaign naming convention generator builds one, and the UTM builder keeps the links consistent.
Run the test without touching it
Once a test starts, the rules are simple and hard to keep.
- No edits. Changing an ad, the budget or the audience during a test changes what is being compared. Meta notes that changing the schedule of an A/B test affects the reliability of the results collected so far.
- No new ads in the test. Adding a version mid-test gives it less time and a different budget.
- No overlapping campaigns. Meta advises against using the test’s audience in any other campaign at the same time.
- No early winners. A test that looks decided on day three can look different on day ten. Read it at the end.
One practical point: in Meta’s creative test, ads that pass review at different times start spending at different times, as Jon Loomer observed. Publish all the test ads together and check they are all active before treating the test as started.
Read the result once, at the end
When the test ends, compare the ads on the metric in the brief. Meta’s A/B test and Google’s experiments report whether a difference is significant at a confidence level; Meta’s creative test identifies top performers without one. For a creative test, treat a small difference as no difference, and look at the result alongside the number of results each ad produced.
- A clear winner. Move budget to it on purpose. Meta’s creative test makes no automatic changes, so the test ads keep running on the original budget until someone acts.
- A clear loser. Turn it off and record why it lost; that is evidence about the audience.
- No clear difference. Keep the cheaper or simpler ad, and make the next test bolder.
Then watch the winner. Creative can wear out as the same people see it again, so a winner today is a starting point for the next test, not a permanent answer.
The creative testing cycle for a client
For an agency, creative testing is a programme, not a one-off. Each test should feed the next.
- 1LearnPast results, comments, reviews and sales calls: what the audience responds to.
- 2BriefOne question per test, the ads, the metric, the budget and the end date.
- 3MakeAds that differ in a way big enough to matter, named so results can be read later.
- 4RunThe right tool, the planned budget and dates, no changes mid-test.
- 5ReadOnce, at the end, against the metric in the brief.
- 6ReportWhat won, what lost, what it says about the audience, and the next test.
The learning step is where most of the value is. Comments on past ads, the reviews customers leave, the questions sales teams hear and the results of earlier tests all suggest what to test next. A test that cannot point to any of these is a guess.
Reporting creative tests to clients
Clients see the ads, so they have opinions about them. A report that shows the test and its result, rather than just the winning ad, turns opinions into evidence.
- The question and the answer. One sentence each.
- The ads. Small images of each version, so the client sees what was compared.
- The result. The deciding metric for each ad, the number of results behind it, and whether the platform reported a confidence level.
- What changes. Budget moved to the winner, losers switched off, and the next test.
- The running log. Every test so far and what it found, so the client sees a body of learning rather than a series of favourites.
Cost figures in the report follow the same rules as any paid media report. For Google Ads, that includes reporting cost exactly as Google charged it, with the agency’s fee shown separately; the PPC reporting guide covers those rules, and client reporting for agencies covers the wider process.
Producing enough creative to test
A testing programme needs a steady supply of ads that differ in meaningful ways. For many agencies, that is the real constraint, not the media buying.
- Plan in themes, not single ads. Each test needs a set of ads that share everything except the variable; plan them together.
- Brief designers with the question. A designer who knows the test is about proof against demonstration will make a better pair of ads than one asked for three variations.
- Keep the client’s brand rules. Bold tests still need to look like the client; brand guidelines set the limits.
- Reuse what won. A winning angle can be remade in new formats and sizes for the next round.
Agencies without in-house design can buy campaign creative under their own brand; that is what white-label graphic design covers, and the media buying itself is on the white-label paid social page.
Common creative testing mistakes
Most creative tests that teach nothing fail in one of a few ways.
- Too many ads, too little budget. Seven ads on a small budget give each too few results to compare.
- Differences too small. One word or one colour changed, so any difference is noise.
- Testing by switching ads on and off. Meta advises against it, because delivery is not split fairly.
- Editing mid-test. The comparison changes the moment the test does.
- Calling it early. A strong first few days decides nothing.
- Deciding on clicks. The ad with the best click-through rate is not always the one with the cheapest sale.
- Leaving winners unscaled. Meta’s creative test does not move budget for you.
- No names, no log. Six months later, nobody knows what was tested.
The ad creative testing checklist
The checklist follows the test from brief to report.
If you would rather have the testing run for you, my paid social management and PPC management services include creative testing as part of monthly management.
Before a creative test, write one question, make ads that differ enough to matter, and work out the budget: ads x results per ad x cost per result. Use Meta’s creative test inside a working campaign or its A/B test for a clean comparison (7 to 30 days), Google’s ad variations for Search and experiments for Demand Gen. Change nothing mid-test, read it once, and report losers as well as winners.
The test I see fail most often is the one with seven ads and a week of budget. Three bold ads and a month usually teach more.
I keep a running log of every test for each client. After a year, it is worth more than any single winning ad.
Frequently asked
What is ad creative testing?
How do I test ad creative on Meta?
How much budget does a creative test need?
How long should a creative test run?
How many ads should I test at once?
Does Meta’s creative test show statistical significance?
How do I test ads in Google Ads?
I have 3 client slots open for Q4 2026. If this raised a question about your own situation, or you want help with it, send it over.
Say hello