Split Testing Basics
The main goal of tracking our campaigns is:
a) To know when they are in profit, or not.
b) To be able to make informed decisions about the elements in the creative funnel.
You will hear it said in our industry, “Testing. Always be testing.”
Good advice, I’d say.
You need to be constantly tinkering with your creative funnel to find the best combinations of offers, landing pages and ads.
This is the prioritized ‘Order of Testing’ that I use with my campaigns:
- Test different offers
- The angle
- The type of landing page (e.g. Rules LP vs. Flog)
- The components of the landing page
- The banners / ads
The most important element to test is the offer itself.
- John’s Weight Loss Product A vs. Jane’s Weight Loss Product B
- Match vs. eHarmony
- Battery Saver vs. Battery Booster
If you promote the wrong offer, you can write off the rest of the campaign.
Success is tough for any affiliate, but damn near impossible if you concede the best converting offer.
In the initial offer testing, I will run a straightforward angle to get some benchmark figures .
My primary focus is to collect enough data to decide which offer is the winner.
We can use an A/B split test, or we can do multivariate testing .
A/B Split Testing
This is where you test A vs. B.
There are only two variables. It is a straight fight to determine a winner.
Does eHarmony convert better than Match.com? Y/N
Multivariate Testing
In this case, we might test A vs. B vs. C
W vs. X vs. Y vs. Z.
Every additional element we test increases the required ad spend to draw any meaningful insights. Multivariate testing is therefore more expensive than A/B split testing.
In practice, I suggest you restrict multivariate testing to the very start of the campaign when you are testing lots of offers with a general angle, or lots of angles on a single offer, or for the testing of banner/text ads.
Most affiliates prefer to run A/B tests on offers and landing pages, but they will run multivariate tests on the angles and ads used.
So in practice we might have two landing pages being tested, and 20 different banners – or two offers, and five different angles.
In a perfect world we would only ever test one element at a time. But the reality of affiliate marketing is that if you take a whole day to decide whether Banner A beats Banner B, you won’t be moving fast enough to catch up with the gravy train.
Back to our Order of Testing…
Once I am convinced that I am running the best offer, I will test all versions of it that I can find from affiliate networks that I trust.
So, let’s say Match.com is beating eHarmony on Affiliate Network X. My next step will be to test:
Match on Network X vs. Match on Network Y vs. Match on Network Z
Why do we do this?
Because the same offer can convert much better with one network than it does on the others.
There are many reasons for this: technology, broker leakage, and sometimes just poor tracking at one network.
If you find a great offer, you can add 20/30% to the ROI by simply shopping around at various networks until you find the version that converts the best .
So, that’s what we do. We shop around .
Once we have the best offer, on the best network, the next step is to test the angle itself.
It’s worth noting that we shouldn’t expect to be in profit at this point.
Profit is tied to the angles you run – unless you have a Shit-Your-Pants Hot Offer, or a dirt-cheap traffic source (in which case ‘general angles’ can do the business).
Angle split-testing is, thus, essential.
I suggest coming up with at least 5 angles to test with your best offer.
Note: In some cases, a great angle can merit a technically worse offer. Maybe SingleParentsDate.com has a poorer conversion rate on general dating traffic, but explodes through the roof when tied to your single parent angle. This is one exception where angle testing can dictate which offers you actually test. Some affiliates prefer to choose their angles first and their offers later – or they may choose offers where the angle is already self-evident.
Once we have our 5 angles, we need to make sure that the rest of the creative funnel is adapted so that it is consistent.
Let’s say we choose these dating angles:
- “Over 40s Only! These Women Love Older Guys”
- “Would You Date a Single Parent?”
- “Meet Christian Singles Near You”
It doesn’t really make sense to use a generic landing page with unrelated copy, does it?
So, we’d cater each landing page and banner for the specific angle .
The copy must be rewritten so that the angle is cohesive: no matter where the user enters your funnel, the message remains the same.
We don’t need to go bat-shit crazy with our landing page variations, or by rolling out dozens of banners.
At this stage, we are simply trying to establish which angle is the most effective. We are testing funnels.
- Angle 1: Banners A B, C – Landing Page 1 – Offer A
- Angle 2: Banners D, E, F – Landing Page 2 – Offer A
- Angle 3: Banners G, H, I – Landing Page 3 – Offer A
- Angle 4: Banners J, K, L – Landing Page 4 – Offer A
- Angle 5: Banners M, N O – Landing Page 5 – Offer A
An angle is a complete funnel, and should be tested as such.
Many affiliates don’t bother to customize their landing pages for each angle .
They will use a system like this:
- Angle 1: Banners A, B, C – Landing Page 1 – Offer A
- Angle 2: Banners D, E, F – Landing Page 1 – Offer A
- Angle 3: Banners G, H, I – Landing Page 1 – Offer A
- Angle 4: Banners J, K, L – Landing Page 1 – Offer A
- Angle 5: Banners M, N, O – Landing Page 1 – Offer A
Note the landing page that stays the same regardless of the different angles.
My view is that you haven’t tested angles effectively until you have customized your landing pages for them.
Once we have established which angle is the best converter, we should be someway closer to profitability .
Indeed, this step is often the ‘breakthrough’.
When you hear affiliates talking about how their campaigns ‘just clicked’, or they woke up and finally ‘got’ affiliate marketing… the resulting force is rarely magic.
It’s probably because they found an angle that worked on an offer that converted well.
(Or because they changed the type of landing page they were using. Another game changing optimization.)
A beautifully effective angle, like all good advertising, slices through the market like a knife through butter. It can transform your ROI from a string of losses to massive, double-your-money-every-day success.
The best affiliates know that finding the best angle for the best offer is about as close as you can come to a ‘Get Rich Quick’ scheme that actually works.
As strategies go, it is golden.
Once you get this close to success, or perhaps if you are already enjoying it with a flute of Champers, you will want to crack on with what I call the End Game Optimisations.
Armed with the best angle, our next step is to work on the creatives that drive this angle to market.
That includes:
- The types of landing pages we are using
- The components of the landing page itself
- The banners and text ads
You can make a strong argument for testing landing page types immediately after the angle.
For example, which works best?
- A flog
- A ‘rules’ landing page
- A short-sell landing page
- An advertorial
- And so on
It is after this stage that we will test things like:
- Changes to the main ad copy
- Changes to the images we use in our ads
- Different calls-to-action
- Different colour schemes
- Tweaked headlines
- Image captions
- Matching our adverts to the style of the site we are advertising on.
- Matching our landing page to the style of the offer we are promoting.
From my experience, it is very rare to go from failure to success by optimizing landing page components alone, or by tinkering with banner elements.
If this is the decisive factor, your margins will be razor thin .
For a major breakthrough, you will almost certainly be relying on an angle that beats out what you had before, or a new offer that has transformative effects on your metrics, or a different category of landing page that works so much better.
Nonetheless, these optimizations are vital for boosting your ROI in the face of banner blindness and the arrival of competition.
Banner blindness is what happens when your ads have been running for several days on the same small traffic source.
Users will naturally get used to them.
They will stop clicking.
This reduction in clicks means your margins are at risk. You will need to have sufficient profitability built in to the campaign for it to survive long-term.
We offset this by working hard to test new banners and landing pages regularly.
We do this not only to find potentially lucrative combinations, but also to keep the funnel fresh for our audience who will quickly grow tired of the same message.
The second need for End Game Optimizations arises from direct competition.
Other affiliates are unwilling to watch your carefully chosen angles prosper without having a crack at them too.
You can’t deny this luxury to your competition, but you can make their life harder by ensuring that whatever they see is one or two steps behind your latest iteration.
At what point can we make decisions based on our data?
If you are going to split test effectively, it helps to have a sense of which figures are meaningful and which are merely coincidental.
Here is a useful calculator for assessing statistical significance.
Incremental Improvements vs. Wholesale Change
One of the toughest skills we must master is the ability to ‘read’ when a campaign is destined to fail vs. when it can be rescued with small incremental optimizations.
I have read hundreds of affiliate ‘Follow Along’ posts where the affiliate has been roundly praised for getting his campaign from -75% ROI down to -50% ROI, in the course of two weeks.
That is:
- Before – $200 spend, $50 revenue.
- After – $200 spend, $100 revenue.
“Good progress,” he is told.
“You are definitely on the right track,” he hears.
Call me a cynical bastard, but I am not quite so accepting of a campaign that has me working my arse off for two weeks and is still losing half of my money.
This begs the question:
At what point should we throw in the towel on a campaign that keeps losing money?
In my view, the testing of angles is a critical juncture in any campaign.
If I am running the best known offer in a vertical, and my 10 best angles are incapable of reducing the deficit to better than a 50% loss, then I see little point in testing the colour of a banner, or the wording of a call-to-action.
It would be inconsequential to the numbers that matter.
Like building a fucking sandcastle as the tsunami rushes to shore.
There will no doubt be ‘success stories’ of guys who broke even, or prospered, after finally uncovering some hidden combination that actually worked.
Good for them.
Seriously. Good for them.
Personally, I have a low tolerance for abysmal results from my best efforts – especially if those efforts are sustained for several days or weeks.
While I believe affiliates should embrace the pursuit of constant incremental improvements in their campaigns, this has to be matched to a realistic assessment of when to cut your losses.
If you are throwing everything at the wall and celebrating a minor fall in an otherwise catastrophic loss, my advice is to take what you’ve learnt and pivot fast.
Angles are the best indication of whether your campaign is worth sticking with in its current form.
The wider the fluctuation in performance by angle, the more confidence you should have that success hinges on a creative breakthrough.
If all angles are clustered around a similar loss, then there is something fundamentally wrong with the offer, the traffic source, or your mathematics.
Don’t try and change this by thinking up fancier headlines.
Get out.
Find a better offer that doesn’t need one.
Metrics To Live By
What are some of the most important metrics we should be tracking?
Meaning can be inferred for just about any metric if you ask the right question, but these are some of our more common interests:
EPC (Earnings Per Click)
This is the offer payout divided by the number of clicks it takes to produce a conversion. It is often used to gauge the performance of an offer relative to its competitors. If Match has an EPC of $0.45, but eHarmony has $0.40, we might consider Match the better offer to promote – assuming our costs are the same .
An offer’s EPC is always more important than its payout.
An offer that pays $100 may seem more attractive than an offer that pays $5.
But if the $100 payout is linked to a $0.10 EPC, and the lower payout boasts a $0.20 EPC, we will make more money from the $5 payout.
Don’t be seduced by merchants that use high payouts to get your traffic.
Always ask for the EPC.
CPC (Cost Per Click)
How much does it cost to generate a single click from the traffic source? This can be either a direct cost (you are charged by the click), or an inferred cost (if we are charged $2 for 1000 views and we get 5 clicks, we can calculate the CPC is $0.40).
Traffic sources that charge a fixed CPC are, generally, easier to track. They are less vulnerable to banner blindness as you are only getting charged for a click. But they are less flexible too.
CPM (Cost Per 1000 Impressions)
How much does it cost for our ad to receive 1000 impressions? This rate will vary massively depending on your traffic source.
We can test different bidding strategies – lowering our CPM to reduce the CPC, or even increasing the CPM with the hope of getting ‘better’ impressions through a higher bid, and therefore getting more clicks, or clicks with greater intent.
EPV (Earnings Per View)
This is similar to EPC, except it also weighs in the users who reached your landing page but didn’t click through to the offer. If our EPC is $0.30, but only 33% of our users are clicking from landing page-to-offer, then our effective EPV is $0.10. We must ensure that our CPC at the traffic source remains below this to stay in profit.
CPV (Cost Per View)
If you are bidding on pop or PPV traffic, comparing the CPV to your EPV will reveal how close (or far) you are to profit.
For example, on a PPV source, we might bid 0.02 per view of our ad.
If our offer delivers an EPV of 0.03, that adds up to a 50% positive ROI.
CTR (Clickthrough Rate)
Your CTR is relevant for both the ads you run, and the landing pages you drive the traffic through.
Clickthrough rates are also the source of a gajillion forum threads from newbie affiliates wondering: “Is my CTR normal?”
There’s no such thing as a ‘normal’ clickthrough rate.
The clicks you draw with your ads can only ever be compared to a similar funnel if your intention is to gauge their effectiveness.
What is the point in comparing the CTR of a banner designed to attract ‘accidental clicks’ vs. one that hard sells an offer?
Answer: there is none.
Likewise, you can’t expect a long sales letter to deliver a clickthrough rate as high as the page spoofing the user with fake virus alerts.
Our efforts to improve CTR have to be matched by an equal impetus to increase the overall conversion rate, as well as the quality of our leads.
There are many angles you can run that lead to tremendous CTRs on the back of a headline that confuses the consumer. This might appear beneficial for the sake of attracting clicks, but not all clicks are equal.
The best affiliates focus on increasing their CTR without sacrificing user intent. A very handy skill to learn.
Conversion Rate (CVR or CR)
Your conversion rate doesn’t have to be the best in the business.
It simply has to work within the parameters defined by that your clickthrough rate and lead quality.
For example:
- Low conversion rate + high clickthrough rate can work on CPM sources.
- Low conversion rate + excellent lead quality can work on expensive traffic if the merchant raises your payout as reward for sending sales.
This is yet another example of the complexity in affiliate marketing.
You always want to increase your conversion rate.
But the ability to bid high or low, on great quality traffic or average traffic, means there is always more than one way of striking profit.
Wal-Mart and WholeFoods are examples of companies at opposite ends of the quality spectrum. They are both profitable, even if their metrics are aligned to different goals.
It would be useless to compare their conversion rates without understanding the unique costs of their models.
Leads to Sales Conversion Rate
I’ve written entire blog posts on the importance of this metric.
For CPA affiliates, it is the most valuable metric of them all. This is what defines your worth to the merchant as an affiliate, and thereby the ease at which you will be able to run your business.
There are three paths ahead of you:
- Low lead quality – You produce less paying customers than the merchant expects. Your payout will be cut, or you will be kicked from the offer (in severe cases).
- Average lead quality – You produce about as many paying customers as the merchant expects. Your payout will stay the same unless you can deliver massive volume and provide value through economies of scale.
- High lead quality – You produce more paying customers than the merchant expects. Your payout will be raised, and you will be badgered to send more traffic. You have instant leverage and can use the advantages of higher payouts to buy more quality traffic.
Most advice you encounter will paint the path of high lead quality as the one you should follow.
I am an advocate of this approach.
I think the most viable way to provide value, as an affiliate, is to consistently deliver more of what the merchant wants: paying customers.
Outperforming your competition in this regard will win you respect, further opportunities, access to better offers, and – of course – higher payouts.
After all, sales are the only transactions of true commercial significance.
A lead is a theoretical transaction.
A lead is a ‘possible sale in the future’.
To know which of your marketing results in sales is the closest you will come to enlightenment.
But this is just one direction you can take.
Many affiliates see equal opportunity in the path of average lead quality backed by massive volume.
That is, if they can provide the ‘expected lead quality’ backed by unexpectedly high volume, the merchant is not going to want to lose that traffic to a rival. So payout bartering (and entry to the merchant’s golden circle) can begin.
Then you have affiliates who make a fortune on the back of low lead quality.
They do this by working other metrics in their favour.
Like Wal-Mart.
Who cares if your lead payout gets slashed from $5.00 to $2.00 if you’re only spending $0.50 to acquire the lead?
We might conclude that it is simply not worth judging the attractiveness of an offer by its lead payout.
There are ways to promote any offer – for great profit – both under and over the market rate.
I hope you are beginning to see why condensing this industry in to a ‘newbie’ guide is such a thankless task.
There are a thousand possible models we can analyse. One affiliate’s fixed beliefs are about as substantial as a piss in the wind.
It’s not for me to tell you what can or cannot work. If the correct metrics are manipulated in the right way, just about anything can be made to work.
Avoid Costly Errors: Get Accurate Data!
If there’s one thing we should agree on, it is that a metric is only useful if it is backed by legitimate data.
One BIG mistake is to assume a CPC or CPV based on what your tracking software tells you.
When costs are incurred, you must always refer to the traffic source for an accurate report of what you are being charged.
Never assume based on traffic received.
If the traffic source is reporting 50% more clicks than you see in your tracker, and you are paying by CPC, then your own calculations of profitability are going to be way off the mark.
Click loss is commonly reported in the affiliate industry.
There are many reasons for it – too many to list.
Your true costs are not defined by the data in your tracker (which, remember, is a third party tool).
This goes for the revenue too.
- What your affiliate network tracks… is what you will be paid.
- What your traffic source tracks… is what you will pay.
It’s in your best interest to ensure that any tracking discrepancies are dealt with early, and/or factored in to your profit and loss calculations.
Anything less will end in tears.
The Art of Placement Optimisation
Most of the tracking concerns we’ve looked at so far involve parts of the funnel that are under our control.
- Landing Page A vs. Landing Page B
- Offer X vs. Offer Y
- Banner 1 vs. Banner 2 vs. Banner 3
If affiliate marketing were as simple as ensuring our own house is kept in order, a lot more affiliates would be successful at it.
Unfortunately, we have to factor in where our ads will be running.
You can have a perfect funnel for capturing traffic, but if it’s placed under a horse’s arsehole, you’re going to end up with your hands full of shit.
Well, on that note, optimizing for the best placements is a bit like taking your ads from arsehole-to-arsehole, website-to-website, hoping – praying – maybe, one of them will shoot out some gold .
Understanding Valuable Placements
The best placements are those where users enter our funnel with maximum intent.
You might call a text ad on the side of Google, particularly one that appears for a valuable term, a great placement.
There’s plenty of intent on the Google sidebar, right?
Users know what they are looking for, and that’s exactly what you will offer them.
Now imagine you are cruising on XHamster… when a popup springs on to your filthy desktop. Your video is just about to start and the last thing you want to do is buy some viagra from a Flog, so you close the popup.
This is the type of audience a popup marketer has to conquer.
An audience of distinctly low commercial intent.
While we know that a Google Ad is much higher quality than an XHamster popup, in terms of intent shown, we can’t say for sure that it will be more profitable – because we haven’t factored in the costs relative to the placement.
This is the case throughout affiliate marketing.
Most of us know where the best advertising can be purchased, but we aren’t so sure that it can be turned profitable at the price it will cost.
Likewise, we know where we can buy advertising dirt cheap, but we don’t know if we’ll be able to turn those weaker hits in to conversions.
Essentially, nobody knows anything in affiliate marketing.
That is, until the campaign is launched and gathering data.
It doesn’t matter whether you advertise on Google, Facebook, or the world’s most obscure display network, you will still have to assess individual placements on a case-by-case basis.
It’s at this point we should introduce two very distinct bands of marketers.
- Affiliates who focus on individual placements.
- Affiliates who focus on RON (run of network) traffic.
If I have a weight loss product, and I decide to run ads on LoseWeightFast.com, this would be an example of an individual placement. I have chosen exactly where I want my ads to appear. I can launch the campaign and discover, probably quite quickly, whether LoseWeightFast.com is capable of delivering leads at a profit. RON traffic is a different beast altogether. In this case, we are launching a campaign across hundreds, potentially even thousands, of websites.
The affiliates who take this path will spend a lot of money to collect a lot of data, before deciding which of those placements show the most potential. The big display traffic sources offer both forms of advertising.
You can launch a RON 300×250 banner campaign on to every website that has a 300×250 placement available – and in doing so, you will spunk a lot of money in rapid time.
Maybe you’ll find some winning combinations, but you will almost certainly lose money first.
Or you can choose specific websites to run your ads on.
You’re unlikely to bankrupt yourself with this method, but it can make for tedious progress and your ability to scale large campaigns is shot dead on Day One.
With popunder and redirect sources, it is extremely rare to find lists of websites where you can choose where to place your ads.
We are forced to advertise without knowing exactly which sites our ads will appear on.
This is called blind advertising.
Individual placements will be credited with an ID (e.g. 1069938), but you’ll never be able to match it to a domain.
For these networks, we have to rely on an optimization philosophy that cuts through the data and brings our campaign to profit through some other means.
Unsurprisingly, there are two conflicting philosophies.
Whitelisting vs. Blacklisting
If you want to run RON campaigns, like many affiliates, you will need to become familiar with both whitelisting and blacklisting.
- Whitelisting: These are campaigns where your ads will only appear on placements that you have allowed in advance (= whitelisted).
- Blacklisting: These are campaigns where your ads will appear on all placements except those that you have removed (= blacklisted).
Now, the paradox here is that you won’t be able to start whitelisting or blacklisting unless you launch the campaign; completely naked; and accept that you are likely to lose money – regardless of whatever genius creative funnel you have designed.
Once your campaign starts to receive traffic, you will use your tracking software to determine:
- Profitable placements
- Dud placements
- Inconclusive placements
Whitelisters and blacklisters share the same philosophy on profitable placements (keep them), and dud placements (get rid of them).
Where they differ is in their attitude towards inconclusive placements: those placements where it is impossible to judge long-term profitability without buying more data.
Whitelist advertisers believe it is better to isolate the profitable placements immediately. They will round up every site ID that is making money and launch a separate campaign that targets only these placements. The RON campaign is seen as a means to this end: a way to pluck out profitable placements and isolate them fast.
In theory, they should be profitable straight away with their new whitelist campaign.
Blacklist advertisers believe it is better to focus on removing dud placements, and let the rest acquire enough data to be judged fairly. They will round up every site ID that is losing money and blacklist these placements from receiving any further advertising.
We can make a strong argument for each philosophy, and there’s no strict ‘best practice’ that you should tie your campaigns to.
Here’s my assessment:
When to Whitelist
- If your budget is small and you are happy to sacrifice scalability and long-term sustainability for an immediately profitable campaign.
- When the majority of RON traffic is skewed towards a small handful of placements.
Sometimes you will find one website delivering 90% of the traffic in your RON campaign. In these cases, you are better to whitelist that single placement and design a funnel that maximizes its gains.
When to Blacklist
- If you want to build a large campaign, to last for several months, you will need the fresh inflow of new sources – so blacklisting is a much better option.
- If you have hundreds (or thousands) of placements that are spending less than 2x the offer payout.
Let’s say my offer pays out $5.00 and I have 300 placements spending $0.30 per day each. A blacklist campaign will keep these placements active and in-play, whereas a whitelist campaign will cull them on an extremely weak basis without testing their true profitability.
My golden rule is to look for ways to get profitable through blacklisting before I consider whitelisting.
This means improving the fundamental metrics of my campaign with a stronger angle, a better offer, or a more persuasive landing page.
If you can raise the performance across every placement, this strategy has a much greater upside. It is more progressive.
There is, as usual, an exception to the Whitelist vs. Blacklist battle .
The Hybrid Whitelist/Blacklist Approach
Many affiliates enjoy the improved margins of a whitelist campaign, but don’t want to sacrifice the exploratory nature of a campaign that is constantly testing new placements.
So, they will have both.
You can get the best of both worlds by having a two-pronged attack on RON traffic.
- One whitelist campaign dedicated to the best placements.
- One blacklist ‘feeder’ campaign where you focus on removing the worst placements, and also the best placements.
In effect, once a placement proves itself as profitable, you remove it from the feeder campaign and switch it over to your whitelist.
Meanwhile, you are actively removing the placements that are proven duds so that your exploratory costs do not spiral out of control on the blacklist campaign.
When should you remove a dud placement?
I use two criteria:
- If a placement receives 2x the offer payout in ad spend without producing a conversion. E.g. When the offer payout is $5, and the placement receives $10 ad spend without converting, it’s dead to me.
- If a placement receives 1x the offer payout in ad spend without producing a single click to my landing page. E.g. When the offer payout is $5, and the placement receives $5 in ad spend without producing a single click to my landing page, this placement is also dead to me.
The vast majority of placements will produce a tiny trickle of traffic that is impossible to evaluate without further data.
I try to keep these placements in-play for as long as possible and work on improving the creative side of my campaign.
A whitelist marketer might argue that any placement that produces a tiny trickle of traffic is not worth optimizing for, anyway.
This is true, but it neglects the accumulation of ‘freak’ conversions that many RON campaigns are known for.
- 1 click, 1 sale
- 1 click, 1 sale
- 2 clicks, 1 sale
- And so on…
These ‘freak’ conversions are not possible with a whitelisting philosophy. You would have paused them before you found out they were profitable.
Optimisation: The Balancing Act
Okay, so we’ve seen both sides of the optimisation process.
We’ve looked at split testing individual elements of our creative funnel, and we’ve addressed the challenges of pinning down the best placements where our ads should be shown.
What we are left with… is an almighty balancing act.
A newbie affiliate can just about get his mind around the need to test one landing page versus another, or to search for the best placements.
The problem is:
How do we combine all of these different tests, one after the other, without obliterating the legitimacy of our data?
If we’re optimising placements in one tab, and uploading new landing pages in the other, how can we ever trust the causation behind a success or failure?
It’s a good question.
When you launch a campaign, the data you obtain on Day 1 is likely to suggest several best courses of action:
- Remove LP X
- Remove Offer Y
- Scrap Banners A, B and F
- Blacklist Placements 23, 43 and 75.
- Etc., etc.
But what if LP X suddenly performs better once we cull the bad placements that were responsible for its traffic? Or once we focus on a different offer?
By optimizing several variables, we can easily spiral out of control, drowning in our own data and completely misrepresenting its context.
The alternative, to optimize variables through painstaking scientific means, is expensive, slow and often pointless (on a low budget).
My advice before blacklisting banners, landing pages, placements and offers… is to start from an imaginary ‘Worst Offenders’ list.
Let’s take this campaign scenario:
- Offer A: $100 spend, $140 revenue (+$40)
- Offer B: $100 spend, $75 revenue (-$25)
Angle 1: $100 spend, $95 revenue (-$5)
Landing Page 1a: $50 spend, $40 revenue (-$10)
Landing Page 1b: $50 spend, $45 revenue (-$5)
Banner 1a: $33.33 spend, $15 revenue (-$18.33)
Banner 1b: $33.33 spend, $20 revenue (-$13.33)
Banner 1c: $33.33 spend, $60 revenue (+$26.67)
Angle 2: $100 spend, $120 revenue (+$20)
Landing Page 2a: $50 spend, $70 revenue (+$20)
Landing Page 2b: $50 spend, $50 revenue (+$0)
Banner 2a: $33.33 spend, $25 revenue (-$8.33)
Banner 2b: $33.33 spend, $40 revenue (+$6.67)
Banner 2c: $33.33 spend, $55 revenue (+$21.67)
Now, the temptation for an over-eager affiliate might be to optimize as follows:
Offer A: $100 spend, $140 revenue (+$40)
Offer B: $100 spend, $75 revenue (-$25)
Angle 1: $100 spend, $95 revenue (-$5)
Landing Page 1a: $50 spend, $40 revenue (-$10)
Landing Page 1b: $50 spend, $45 revenue (-$5)
Banner 1a: $33.33 spend, $15 revenue (-$18.33)
Banner 1b: $33.33 spend, $20 revenue (-$13.33)
Banner 1c: $33.33 spend, $60 revenue (+$26.67)
Angle 2: $100 spend, $120 revenue (+$20)
Landing Page 2a: $50 spend, $70 revenue (+$20)
Landing Page 2b: $50 spend, $50 revenue (+$0)
Banner 2a: $33.33 spend, $25 revenue (-$8.33)
Banner 2b: $33.33 spend, $40 revenue (+$13.33)
Banner 2c: $33.33 spend, $55 revenue (+$16.67)
Here, the affiliate is following a blacklist philosophy of culling everything that doesn’t produce profit, without factoring in the correlation between those various elements.
But what if Landing Page 1b is actually the strongest LP?
Maybe it received a greater amount of traffic from Banners 1a and 1b (which were awful) to merit the weaker performance than those in Angle 2.
By optimizing so many variables, so quickly, we are likely to draw false conclusions. Premature conclusions.
Here’s what I would do at the end of Day 1 with the same campaign:
Offer A: $100 spend, $140 revenue (+$40)
Offer B: $100 spend, $75 revenue (-$25)
Angle 1: $100 spend, $95 revenue (-$5)
Landing Page 1a: $50 spend, $40 revenue (-$10)
Landing Page 1b: $50 spend, $45 revenue (-$5)
Banner 1a: $33.33 spend, $15 revenue (-$18.33)
Banner 1b: $33.33 spend, $20 revenue (-$13.33)
Banner 1c: $33.33 spend, $60 revenue (+$26.67)
Angle 2: $100 spend, $120 revenue (+$20)
Landing Page 2a: $50 spend, $70 revenue (+$20)
Landing Page 2b: $50 spend, $50 revenue (+$0)
Banner 2a: $33.33 spend, $25 revenue (-$8.33)
Banner 2b: $33.33 spend, $40 revenue (+$13.33)
Banner 2c: $33.33 spend, $55 revenue (+$16.67)
The only optimization I would make is to remove Offer B, which is lagging significantly behind Offer A.
For every other variable, I want more data.
Why target the offer first?
Experience tells me that offers are a decisive factor in the success of a campaign. Given a clear signal that I have one offer better than the other, it makes sense to target this dud first.
I don’t want it to corrupt the rest of my data.
Many affiliates operate in reverse.
They will target banners first because, in theory, a profitable banner can be multiplied for a significant improvement in the campaign’s overall ROI.
Imagine an affiliate who optimizes like this:
This affiliate is calculating that by running only Banner 1c with its close-to100% ROI ($33.33 spend, $60 revenue), he will suddenly start doubling his money across the entire campaign.
Sorry… but this never happens.
Your early optimizations should target the most decisive elements of the campaign:
- Find the best offer
- Find the best angle
- Find the best type of landing page
- Blacklist dud placements
I don’t see any point in optimizing banners or specific landing page elements until all of the above criteria have been met to some level of satisfaction.
Note: The lower quality the traffic source you work with, the more likely that ‘blacklisting dud placements’ will rise up the list and overtake ‘find best offer’ as #1 in the order of priorities. The reason? You can’t monetise a handful of horseshit – no matter which offer you send it to.
Now imagine we allow our campaign data to accumulate for a couple more days.
It now looks like this:
At this point we can say with reasonable certainty that Angle 1 is our best hope of success.
So we optimize like this :
We want to focus on our best performing angle, since we know finding this elusive element is a decisive factor in profitable campaigns.
While we’re optimizing, we will remove Banner 1a since it is clearly underperforming relative to Banners 1b and 1c.
It looks like Landing Page 1a beats 1b, but the results aren’t conclusive enough for my liking. And seeing how they are both profitable anyway, there’s no harm in letting each landing page gather more data.
What happens next? Well, we have a profitable campaign. That’s the good news.
But we only have two banners.
This campaign is guaranteed to decline in performance if we don’t roll out fresh new banners to prevent users from growing blind to them.
We know what angle is working, and we can see the banners producing the highest return, so the next step is obvious:
Create more banners, based on what we know works.
Once your campaign enters the maintenance phase, this is when we must get busy testing different banners, tweaking landing pages, and perhaps even testing different versions of the same offer (Network X vs. Network Y).
It is important to do this in a practical order.
Your offer is the most critical factor, followed by your angle, and then the types of landing pages.
If you are advertising on more than one website, placements are vital too.
The Challenge of Mobile Optimization
Affiliates running mobile campaigns are unlucky when it comes to optimization.
Here we have a whole new raft of variables to consider:
- Operating Systems
- Models
- Brands
- Screen sizes
- WiFi vs. Carrier connection
- Phone capabilities
Unlike web campaigns, these ‘minor’ variables can be decisive in shaping our success.
Your choice of angle is unlikely to improve campaign performance if the biggest fault lies with iOS massively outperforming Android – and your campaign receiving 80% Android traffic – or vice versa.
This is what I mean when I refer to the ambiguity of mobile campaigns.
It is difficult to predict a successful mobile campaign because doing so would take an unfathomably sound knowledge of handset, carrier and operating system combinations.
How do we know that Samsung will convert like a demon, and HTC will flop like a wet flannel?
We don’t.
Mobile advertising requires balls of steel in the testing process.
You will lose money, through no fault of your own.
The upside is that, if you are good at working through rows and columns of data, there is tremendous money to be made in discovering the winning combinations.
My advice when dealing with mobile campaigns (and dozens of minor variables) is to focus strictly on cautious blacklisting.
Don’t block a handset, or operating system, or model of phone… until it is clearly losing you money on a terminal basis.
Angles have a gigantic say in your success with mobile campaigns, and they should remain a top priority (along with fishing for a good offer).
Technical optimizations are essential too, but they should happen continuously in the background as you weed out the duds.
Note: You can alleviate some of this pressure by communicating with your affiliate manager to work out which phones, carriers and operating systems are the most suitable for a given offer. Any merchant worth his salt will have stacks of data to shape your efforts.
One final word of advice on mobile optimizations:
Blacklist as specifically as you can.
I have seen many campaigns where Android has been leaking money at first glance, but if I dig through my data I will find that it is actually a single handset at fault – not Android.
(一直往下钻，直到找到明显的亏钱的要素为止)
There is no point in blacklisting all Android devices if you can determine that removing Samsung handsets alone would put you back in profit.
Make use of your tracking software’s drilldown reporting to seek out the true cause of lower conversion rates.
You’ll be pleased to hear that this decision making process gets much easier with time, experience and practice!
Next up, we are going to look at a template campaign launch; a start-to-finish checklist you can follow to launch your own campaigns.
发表评论