Everyone in the PPC world understands the importance of ad text testing. We care, we think it’s important, we wanna do it…and you know what? For reals, it’s complicated. The headline lied. Most of us don’t have one campaign with three ad groups to deal with, we have three hundred ad groups, and a lot of us aren’t really willing to sit around and let tests run for the sake of running to completion when we just know that x or y or z optimization is going to make things work better…so we slack. We run a kinda-valid test. We sort of compare two things that should be…similar. We, and many of our compatriots, have been writing for years about the right way to do PPC ad text testing– and it can be less than straightforward. But it isn’t impossible, and a little bit of planning and some structure can go a long way in keeping things manageable. Some of the basic concepts we need to keep in mind are as follows:
To really know how an ad is performing, you have to compare it to an ad that is being served to the same audience. Different ad groups have different keywords, and their users have different search intent. You know that has to impact CTR and conversion rate, people, but lots of us try and test across ad groups anyway. If you’re going to, mitigate the damage by having ALL of the same ads running in the same ad groups so that when you compile statistics each ad text will have an equal chance at traffic, and ensure they at least all go to the same landing page. But yeah, a test is really only valid within its ad group. I’m not saying we haven’t made successful broader generalizations from cross-ad-group testing, however.
Your ads, they have to be on rotate, not optimize, or they won’t receive the same proportion of traffic. This seems obvious, but hey.
You can’t make decisions without statistically valid data. I’ve seen the recommendation that you should choose how many ads to test in your ad groups according to the formula (ad groups # clicks per month)/100, and that seems reasonable to me. You don’t want your test to take a year to run, and it will take that long to get valid data if you run too many ads for your traffic. Use a tool like this statistical validity test to determine if your ads have enough data in comparison with one another to choose a winner. This is especially important in the case of an ad which has been running longer than another and therefore will have substantially more data available.
To make a valid test, you need to try to keep keywords and landing pages as consistent as possible throughout the ad text test. This is why I said sometimes our desire to optimize gets in the way of our desire to test- we want to add modified broad match keywords! We want to change the headline on the landing page! We wanna use Website Optimizer! But we’re going to screw our ad text test up, and that’s something we’re going to have to keep in mind.
After we remember all of those important little details, to further confuse and enlighten us I’ve attached an excel file which can be used for developing and tracking ad text testing. In the first tab, you can list the various headlines, description lines, display URLs, and landing pages you plan to test. In the second tab, I’ve given examples of the number of ads that would be required to test various combinations of elements. In the third and fourth tabs, there are record-keeping charts in which various combinations can be recorded with or without data for easier analysis later, as tracking ad text changes within the PPC interfaces can be…tricky. You may find this to be the most horrible ad text testing planner ever, but it’s helped me out and I figured I’d share!
