Facebook has some severe limitations when it comes to split-testing. This issue tends to get glossed over (because people don’t know what to do), but it’s important to figure it out.
“What’s the problem exactly?”
Facebook will serve the most traffic to the best performing ad in an Ad Set. Notice I said “ad” (singular) not “ads” (plural).
If you put more than one ad in an Ad Set, whichever ad gets the early clicks is the one that will get “whooshed” all the traffic. Unfortunately, this doesn’t mean it’s the best converting ad, it just means that it’s the ad that gets the early clicks, so it got sent the majority of the available impressions.
A new client came to me and wanted to know why all of his ads weren’t being shown. His account was a perfect illustration of what happens when you have more than one ad in an Ad Set. I did what any normal human being would do and asked if I could take a screenshot.
This is a exactly what happens when you have more than one ad in an Ad Set:
One ad was served more than twice the impressions of the next, and several of the ads were never even shown. There is literally no way to know if the ad with the most impressions is the best performing ad unless all of the ads are actually shown.
“How do you get around this to run more accurate split-tests?”
Here’s what I do:
Place one ad per Ad Set, and then test it against other Ad Sets. (This used to be even more cumbersome before we had Ad Sets and had to do it at the campaign level.)
Once you’ve done this step, there are a couple of things you can do to circumvent the lack of ability to split-test.
1. First, do serialized testing.
Facebook ads are absolutely not “set and forget”. You have to use a bunch of creatives.
I got tired of wasting creatives because I followed the recommended procedure and placed 3-4 ads per campaign, only to have one ad get the majority of the traffic and some of the ads got none.
So, I starting doing serialized testing. Instead of placing all of my ads in the same Ad Set, I place them in different Ad Sets and run them one after another, rather than at the same time.
This strategy doesn’t take into account ‘day of the week’ and other variables, but… it’s better than nothing.
“What is Ad Pulsing?”
As part of my Facebook Ads strategy, I use a process that I call ‘ad pulsing’.
As part of Facebook’s algorithm, they send new ads an initial burst of traffic. We noticed time after time that we got a lot more traction on ads in the first couple of days. We’d see graphs like this most of the time, with an initial spike of traffic, then a drop off:
So… we thought, “What if we only ran ads for 1 or 2 days?”
We started testing this, and our ROI went thru the roof. Serialized testing gives me answers that are reasonable “enough” to make decisions about winners & losers. With Ad Pulsing I’m able to re-use ads after 3-4 weeks, so when I build ad creatives, I’m actually building an asset that I can use over and over.
2. Second, do head-to-head testing.
This is the closest we’ve been able to get to true split testing. I run two Ad Sets in a head-to-head test to the same target group for the same amount of time. It drives the cost up because I’m bidding against myself, but I have a much clearer idea about which ad truly converts the best.
The problem with this approach is that if you run two campaigns at the same time, you have potentially doubled the impressions to the group which biases the results you get from *both* ads.
But… odds are that the winning ad is still most likely the ‘actual’ winner.
Now, you definitely don’t want to do this all the time, because it will drive your costs up.
I usually do this if I’m testing out a new ad type, or an entirely new concept, and usually just for a day or so. I don’t do head-to-head testing for minor changes.
Facebook is a young platform. They seriously make changes almost daily. I work hard to keep up.
When I asked a Facebook spokesperson if split-testing capabilities were a feature they were planning to add at some point, he stated that while Facebook knows there is interest, “there are no immediate plans for that type of testing at this time, [but] that could certainly change in the future.”
In the meantime, these strategies for serialized & head-to-head testing will have to meet our split-testing needs.
If you’d like to talk with us about managing your Facebook Advertising, schedule a time here to meet with me for a 20 min. call. We’ll have a nice chat & talk about how we can help make Facebook work for you.