Ad Testing Plan for the Professional Ad-(wo)man

By , President at Hanapin Marketing


I recently co-authored an eBook with Elisa over at Wordstream called “Tricks to Get the Click: 10 Ad Text Best Practices.” While writing it the one thing that stood out the most was the need for a systematic ad testing process. I’m talking beyond just a system for picking winners. I am talking about a system for planning out what to test, when, for how long and how to use what you learning to influence other channels.

Part of the reason for having an ad testing plan is that it help insure that you learn from winning ads and incorporate the winning messaging into future ad testing. Sure sometimes you will want to write completely new ads but often times you can combine winning messages together to create super ads!

The following process is how I’ve been structuring my ad tests since writing that eBook. Of course we name everything over here at Hanapin, so this is called the Ad Testing Matrix (which is different than the Ad Matrix).


1. Plan out the duration of your tests

The click volume of the account/campaign/ ad group you are working in will determine the length of your test. I like a few hundred clicks before picking a winning ad so I typically think through how long it should take me to get to 500 clicks across a few ads. If it would take 2 weeks to reach that point then I would plan for my tests to last 2 weeks and so on.


2. Identify the best message you currently have

For “best message” I don’t mean the entire ad but the best call-to-action, headline, or description line. To do this you can do a Sumif formula off an ad report and determine which of your messaging is the strongest. (Eric wrote a great post on this and other useful Excel PPC tools.


3. Pick new messaging to test

Using the ad testing matrix, pull out the best features, benefits, and call-to-actions of your product or service. Then write a headline, or description line that best illustrates this. Now you have a new message that you can test in conjunction with the old messaging. Yes, in conjunction. You are going to coming the two and see if you can create a new, better ad with the old messaging plus new messaging.


4. Write additional new messages that you will test down the road

Here’s the proactive, planning part of the ad test plan. At the same time that you pick the next ad you will write, create some additional new messaging that you will test. In an excel doc you can map out these tests by adding in the new messaging at the testing durations you schedule at step one of this process. (Below there is a link to a sample matrix that will give you a better idea of what is meant by all this.)


5. Always compare the control with the new messaging

When you begin a new test you will compare the control (the most recent winning ad) to the new messaging that you are testing. If the old one wins then you’ll back up to the control and test the next new message in your testing matrix. If the new version wins then you’ll pause the control and repeat. In the perfect world you’ll win about half the time so don’t get frustrated if a few of your new messages take a step back. Keep at it and you’ll find the CTR/CR gains you are looking for!


6. Record results

Seems obvious, but I’ve seen it all too often when PPC account managers conclude ad tests but do not record results. They simply pause the old ad, upload the new and then move on with their life. It’s a shame, for a lot of reasons. Partly because you do not use that information to influence future tests and partly because it makes it harder to report your results at your next quarterly PPC review. If you’ve mapped it out it’s easy to add a graph with CTR (hopefully moving up and to the right) along with some arrows pointing to dates on the graph where new ad messaging was rotated in. It’s data that can inform the business on other channels and it makes you look great for being so organized and scientific in your testing!


That’s more or less it! Now you can sit back, wait for results to come in, and upload the new tests with your predetermined messaging at your predetermine durations.


Not sure what I mean by all this? Below is a sample of the ad testing matrix that hopefully will shed some additional light on this process:

Sample Ad Testing Matrix


In this sample I continue carrying on the winning messaging throughout. So you can see that the “Over 500,000 Happy Customers” shows up a few times but then is replaced by “Free Shipping” after that messaging won in my fake test!


Oh, and before you slam me in the comments section, a couple caveats to this are that this matrix works for a generic messaging test. So if you have multiple products/services you probably need to add a campaign field to the testing matrix to insure you are testing relevant messaging that is as specific to the produce as possible. You’ll also want to make your message generic enough that you can try to squeeze your keyword(s) into it so you can keep ad relevance high!

Twitter Facebook LinkedIn Google+ Email Print More

8 thoughts on “Ad Testing Plan for the Professional Ad-(wo)man

  1. Justin Sous

    Hi Jeff,
    Nice post and I liked the E-book you co-wrote. I also got a kick out of the disclaimer in the last paragraph lol. A few questions arose while reading the E-book:

    1. You mentioned that the use of the display url increased your conversion rate for that ad significantly. I found that interesting because I’ve never really seen that correlation. I’ve definitely seen an increase in CTR as you said too, but not necessarily in conversion. Do you think it’s because your visitors were more qualified? What do you think lead to the increase in conversion?
    2. You said you should always declare an ad winner and end a split test. Interesting case made; however, I find that even if you feel you found a winning ad, you should still be split testing. This could be a simple change like a the display url for example, or even the landing page (if possible). I’ve seen two ads which were almost identical perform differently and have varying average positions. Sometimes I think google just decides they don’t like an ad sometimes and punishes its position :p . So even if we have a winning Headline, D1 and D2… I’d still test something small to keep some sort of test going.
    3. Does hyphenating keywords in the display URL have a different effect in terms of relevance (and how google reads the ad) than not hyphenating?

    Thank you, and well done!

    1. Jeff AllenJeffreyAllen

      1. I have seen increase in CR with the display URL. I think it has more to do with the mindset. If I say “CompareShoes” in the display URL it is likely to put people into the mindset of well…comparing. But if I put “BuyShoes” I am obviously signaling that buying shoes is what the site is about. This is not always the case, and sometimes it has a negative effect.

      2. You are right that best case scenario you are always actively testing something. I’ve found that this isn’t always the case, though. Sometimes you have enough data to call a test but are working on other areas of the account so do not have another test in the works. Those are the cases where I say it’s best to just let the winner run until you have something new to test.

      3. It appears that hyphenating the display url has the same effect that not hyphenating it has. I am basing this off the empirical evidence that the keywords are both bolded in the ad with and without the hyphen (I did a quick search on Nursing Schools to verify).

      1. Justin Sous

        Thanks Jeff, and that was my thought with #3. I suppose a good test to run is difference in CTR when hyphenating the display url as opposed to no hyphen.


Leave a Reply

Your email address will not be published. Required fields are marked *