You’ve probably noticed that every single PPC agency out there is “data-driven”.
So why is it that most agencies we meet are struggling with data silos that make cross-channel optimization difficult and agency-wide insights near impossible?
We call this the data-driven contradiction: most agencies aspire to be data-driven, yet the technically complex reality of implementing this means most struggle to be truly driven by data.
Every innovative agency has its own unique strategies and rules for things like budget management and ad optimization, and to work efficiently, you need to automate these across multiple platforms.
But building the necessary custom optimization tools that use your unique PPC strategies to increase your clients’ campaign performance is expensive and time consuming.
Or is it?
It used to be prohibitively expensive for smaller agencies to build custom tools. But the underlying technology has changed! I’m here to tell you that the size of your agency no longer matters.
We’re a team of specialists who build custom PPC automation solutions and purpose-built advertising data infrastructure solutions for agencies – and we found initial success as a business working with smaller agencies that have 12 to 60 employees. That gives our developers and cloud engineers a unique perspective within the PPC industry.
We had to find more efficient ways of working, and today I will show you an approach that literally any size agency can implement: it’s flexible, infinitely scalable, and extremely cost-effective. In fact, most agencies already work with the required tools.
Stop managing your clients in silos
For PPC data infrastructure experts like us, cross-channel reporting and automating the process of making optimization changes across platforms are actually two sides of the same coin.
Think holistically about your data!
At the most basic level, you need a PPC data pipeline that pulls data from various data sources like the ad networks, cleans and normalizes it, and creates an automatic data flow to your agency’s own data warehouse. In a second step, you need to push optimization changes back to the ad networks.
We will go through the steps in turn.
Step 1: Which data warehouse?
Your clients have hired you to optimize spend and manage budgets efficiently. If you don’t have all your data in one place, how will you analyze all your client data at once across multiple channels to get insights about the account management strategies that make your agency successful and optimize it? How will you easily compare the performance of all the different accounts you manage as an agency?
Your data warehouse is where you unify and store all of your agency’s data. There are huge benefits to this and it’s always worth doing. For one thing, it’s easy to run advanced analytics on your data for optimization purposes.
For example, we recently helped an agency rethink the keyword strategy for an entire vertical by using n-gram analysis comparing Google ads and Microsoft ads side by side. This would not have been possible without having the data stored in a central data warehouse. And that’s precisely the reason why you should steer clear of online cross-channel reporting platforms at all costs: you’re sacrificing control for convenience.
Of course ease of use is important and that’s why we honestly wouldn’t recommend anything but Google BigQuery. You get enterprise-level performance at a low cost and your agency most likely already uses other Google tools like Sheets and Data Studio that integrate seamlessly with it. It also has direct connections with SA360 and Google Analytics, so you don’t even need to be very technical to pull that data.
Most importantly, BigQuery ticks all the boxes in terms of performance, flexibility, and scalability. Plus, your data is automatically backed up for disaster recovery purposes. And it’s very, very cheap!
It’s also worth noting that BigQuery integrates with all major data visualization platforms, but Data Studio is usually more than enough if you know how to use it (here’s a guide).
Step 2: Which data pipeline?
Since few agencies we work with have specialist cloud engineers and devs in-house, we actually built a managed advertising data warehouse that is purpose-built for digital marketing agencies and pulls in terabytes of PPC data from various sources every day.
Here’s what we learned:
First, you don’t need 500+ connectors that the likes of Supermetrics, Funnel, Improvado, Adverity or similar platforms advertise. Most likely Google Ads, Facebook ads, Microsoft ads, Instagram ads, Linkedin ads and Twitter ads, and a few more will be in focus for your agency.
Focus instead on minimizing manual steps and maintenance. We ended up building our managed service around an innovative new advertising data infrastructure solution called Shape ADI because it has a powerful two-way API!
Not to get too technical, but the powerful thing is that it lets us make calls using one API to pull data from the most popular ad networks and returns normalized data in BigQuery.
Data normalization can be a huge hidden cost because it can involve additional steps so you need to use solutions that automate it. When we pull an agency’s historic client data for Google ads, Facebook ads, Youtube ads, Microsoft ads, Instagram ads, Linkedin ads and Twitter into BigQuery it’s all readily comparable. We use pre-configured tables and views for BigQuery (massive time saver!).
Maintenance of APIs is another thing to consider. Ad networks (Facebook and Google especially) constantly update their APIs. We used to have to maintain seven or more separate APIs – now we can rely on one API knowing it’s always updated (don’t underestimate peace of mind!).
Finally, coverage of the ad network’s APIs can vary wildly between data pipeline companies so you might be left missing key pieces of information that you can’t include in your analysis. Focus on maximum coverage of APIs that are relevant for your PPC use cases rather than the total number of connectors.
Bottom line, with this set-up it takes us one afternoon to set up cross-channel reporting for a mid-sized agency and it requires minimal maintenance.
Step 3: Write back optimization changes to ad networks
To recap, our goal is not just to pull data, but also to automate writing back optimization changes to the ad networks for purposes like budget management. For this we need a two-way API – as the name suggests it allows your PPC data to flow both ways.
Conventional data pipelines like Supermetrics, Funnel, Improvado, Adverity and the like don’t give you this option. That is extremely limiting. We build custom automation solutions for agencies, and Shape’s two-way API was developed to help agencies create PPC management platforms and other scalable, state-of-the-art advertising technology.
But it is not necessary for you to always be building a whole platform for a two-way API to be necessary. People underestimate the power of a good ol’ Google Sheet! Here are two common use cases we encounter that you can solve with a Google Sheet in combination with the kind of data infrastructure I’ve described above.
Example use case 1: Budget management
In this scenario, agencies typically ask us to help them pause all overspending campaigns across channels or increase budgets for all campaigns for a specific client account. We all remember the pain of Google increasing the daily overspend range and the importance of keeping costs in check.
It can be annoying to log into different platforms and it’s better to have central control. With the data infrastructure we discussed above you can manage budgets centrally from a Google Sheet without having to maintain separate APIs.
It also means you can stop setting budgets in silos – instead of deciding on a Search budget and a Social budget independently, for example, you can easily create models that let you predict the performance for an entire client when splitting the overall budget in different ways.
Example use case 2: Healthchecks and alerts
Every agency has their own rules for what constitutes “normal” behaviour (how many keywords per ad group, how many ads, how much testing, naming conventions etc.). But it’s a pain for account managers to look at different platforms individually for alerts, not to mention a time sink. Why wouldn’t you want to build a central report that consolidates alerts?
These two examples show that building highly effective automation tools is definitely in reach for any size agency and you don’t need an army of developers to get there. The important thing is that you have the right advertising data infrastructure set-up in place that allows you to start and to keep your options open.
