Never Odd Or Even: PPC Ad Text Testing Made Easy

By ,


Everyone in the PPC world understands the importance of ad text testing. We care, we think it’s important, we wanna do it…and you know what? For reals, it’s complicated. The headline lied. Most of us don’t have one campaign with three ad groups to deal with, we have three hundred ad groups, and a lot of us aren’t really willing to sit around and let tests run for the sake of running to completion when we just know that x or y or z optimization is going to make things work better…so we slack. We run a kinda-valid test. We sort of compare two things that should be…similar.  We, and many of our compatriots, have been writing for years about the right way to do PPC ad text testing– and it can be less than straightforward. But it isn’t impossible, and a little bit of planning and some structure can go a long way in keeping things manageable. Some of the basic concepts we need to keep in mind are as follows:

To really know how an ad is performing, you have to compare it to an ad that is being served to the same audience. Different ad groups have different keywords, and their users have different search intent. You know that has to impact CTR and conversion rate, people, but lots of us try and test across ad groups anyway. If you’re going to, mitigate the damage by having ALL of the same ads running in the same ad groups so that when you compile statistics each ad text will have an equal chance at traffic, and ensure they at least all go to the same landing page. But yeah, a test is really only valid within its ad group. I’m not saying we haven’t made successful broader generalizations from cross-ad-group testing, however.

Your ads, they have to be on rotate, not optimize, or they won’t receive the same proportion of traffic. This seems obvious, but hey.

You can’t make decisions without statistically valid data. I’ve seen the recommendation that you should choose how many ads to test in your ad groups according to the formula (ad groups # clicks per month)/100, and that seems reasonable to me. You don’t want your test to take a year to run, and it will take that long to get valid data if you run too many ads for your traffic. Use a tool like this statistical validity test to determine if your ads have enough data in comparison with one another to choose a winner. This is especially important in the case of an ad which has been running longer than another and therefore will have substantially more data available.

To make a valid test, you need to try to keep keywords and landing pages as consistent as possible throughout the ad text test. This is why I said sometimes our desire to optimize gets in the way of our desire to test- we want to add modified broad match keywords! We want to change the headline on the landing page!  We wanna use Website Optimizer! But we’re going to screw our ad text test up, and that’s something we’re going to have to keep in mind.

After we remember all of those important little details, to further confuse and enlighten us I’ve attached an excel file which can be used for developing and tracking ad text testing. In the first tab, you can list the various headlines, description lines, display URLs, and landing pages you plan to test. In the second tab, I’ve given examples of the number of ads that would be required to test various combinations of elements. In the third and fourth tabs, there are record-keeping charts in which various combinations can be recorded with or without data for easier analysis later, as tracking ad text changes within the PPC interfaces can be…tricky. You may find this to be the most horrible ad text testing planner ever, but it’s helped me out and I figured I’d share!

PPC Ad Text Testing Template

Twitter Facebook LinkedIn Google+ Email Print More

4 thoughts on “Never Odd Or Even: PPC Ad Text Testing Made Easy

  1. Pingback: SearchCap: The Day In Search, August 10, 2010

  2. Chad Summerhill

    Hey Jessica, love the post! Ad testing is a pain, but well worth the trouble. I just created a free excel download for validating sample sizes. Maybe you would like to use it in your PPC Ad Text Testing Template. It calculates significance for CTR, ConvR, and Imp2Conv. It also has a Minimum Test Size Calculator you can use to determine how long it might take to do a test. There is a video that explains how I built it and how to use it.

    I hope you can put it to good use.

  3. John

    The only minimally useful piece of information is the (ad groups # clicks per month)/100, but I prefer to use the much simpler rule that each ad must get at least 100 clicks per day to be able to decide. So, if you have an ad group with 300 clicks per day, you can have up to three different ads in that ad group and the information will be statistically significant after a day, but of course the longer the better, and if the difference is within 10%, it’s not significant anyway. Plus, to be honest I didn’t really understand this rule… why should the number of ad groups matter?
    Also, in my opinion landing page has nothing to do with ad text in this context. The objective of testing different ad texts is primarily to improve CTR (and CR). Sure, different ad text might influence the CR of the same landing page, and different ad text will match different landing pages better, but different landing pages do not influence CTR, which is the primary metric to define success of an ad.


Leave a Reply

Your email address will not be published. Required fields are marked *