Preventing Bad Behavior in Header Bidding: Supply Path Optimization

As publishers implement header bidding in increasing numbers, we are beginning to see new techniques that allow an aggressive SSP to significantly alter outcomes for both the publisher and the buyer. This results in short-term gain for the SSP but long-term pain for the advertiser, publisher, and the programmatic ecosystem as a whole.

Fortunately, there is a solution. But first, let's look at the problem in more detail.

Header Bidding in Action

To see how header bidding works, try Headerbid Expert on Chrome. When you visit a website, you'll see who's bidding on the inventory. In this example, you see 10 different bidders competing to buy the same impression:

What you can't see is what happens next: each SSP sends these requests to all of the RTB bidders that they work with. This means that each impression is being bid upon not just by these 10 header bidders, but by over 100 RTB bidders. I pulled the backend data and the AppNexus DSP saw this particular impression 6 times through different SSPs.

The following examples are hypothetical and no Daily Mail inventory or impressions were harmed in creating them!

If you set up a campaign that targets all platform-audited inventory, it will bid on every one of these requests:

SSPBid PriceOutcomePrice Paid
SSP 1$5.00Loss
SSP 2$5.00Loss
SSP 3$5.00Loss
SSP 4$5.00Win$5.00
SSP 5$5.00Loss

If every SSP is passing the same bid amount into DFP, then the winner should be random. However, if we inspect this auction process, we find that SSP 4 always wins1. Why?

Each SSP has different logic for how it submits bids into DFP. Some SSPs submit a second-price bid based on the next-best price that they see. However, SSP 4 submits a first-price bid - which always wins.

<2nd price>
SSPBid PriceSubmitted PriceAuction LogicOutcomePrice Paid
SSP 1$5.00$2.352nd priceLoss
SSP 2$5.00$3.65Partial 1st priceLoss
SSP 3$5.00$1.48Loss
SSP 4$5.00$5.001st priceWin$5.00
SSP 5$5.00$2.522nd priceLoss

This is great news for SSP 4, and seems optimal for the publisher. They're getting the most possible money for this impression. On the other hand, let's look at the world from the buyer's perspective:

PublisherCPMConversion rateCPA
Auto Trader$2.250.02%$11.20
Ebay$2.650.025%$10.60
Daily Mail$5.000.04%$12.50

Even though Daily Mail has the best-performing inventory, the effective CPA is much higher. The buyer (either a human or an algorithm) will shift spend away from Daily Mail.

Impact of an aggressive SSP

As we see here, the header bidding setup rewards the bidder that has the most aggressive auction dynamics... in the short term. However, in the long term, this behavior damages the publisher and causes overall spend to shift away. The publisher might say "SSP 4 is the best partner I have, and the others aren't doing anything". However, if the publisher turned off SSP 4, what would happen? SSP 2 would win, causing the CPM of Daily Mail to drop to $3.65, resulting in a CPA of $9.13. This is the best CPA of these publishers, even though the price is the highest, and a smart buyer will shift spend to Daily Mail based on these results.

How do buyers react to this? It depends:

  • A human trader runs a report by domain every few days and turns off underperforming domains. This will remove all spend on Daily Mail, effectively penalizing both the publisher and the rest of the SSPs.
  • An algorithm adjusts bid prices based on historical ROI of the domain, reducing bids on Daily Mail by 20% based on its poor price/performance
  • No reaction. This does happen, often with retargeting or audience-targeted campaigns run via naive bidders.

Let's say that buyers are evenly distributed across these three strategies. A third of buyers would stop buying Daily Mail and another third would cut their bids. Overall, 40% of all demand on Daily Mail would disappear - all because of one SSP. And Daily Mail wouldn't even know. In fact, they would say "Wow, our programmatic revenue is down... but thank goodness for SSP 4 who is crushing it!"

What we are seeing is the reason that programmatic hasn't succeeded as a channel. The incentives favor aggressive middlemen who maximize their own revenue and profit at the cost of optimal outcomes for both the marketer and the publisher. SSP 4 is probably bragging to their investors about their amazing results due to header bidding, and the other SSPs are struggling to keep their business alive.

So what can we do about it?

No-bidding Aggressive SSPs

The easiest way to fix this problem is actually rather simple. The buyer just needs to stop bidding on SSP 4 on Daily Mail. The price drops to $3.65; Daily Mail now outperforms other publishers; the buyer ramps up spend, and everybody wins (except SSP 4).

SSPBid PriceNext Best PricesSubmitted PriceOutcomePrice Paid
SSP 1$5.00$2.35, $2.25$2.35Loss
SSP 2$5.00$2.30, $2.25$3.65Win$3.65
SSP 3$5.00$1.48, $1.47$1.48Loss
SSP 4No bid$2.30, $2.25$2.55
SSP 5$5.00$2.52, $2.25$2.52Loss

We see that SSP 2 wins - but they're not running a pure second price auction either. The price is better, but still not optimal for the buyer. Let's turn off SSP 2 as well.

SSPBid PriceNext Best PricesSubmitted PriceOutcomePrice Paid
SSP 1$5,00$2.35, $2.25$2.25Loss
SSP 2No bid$2.30, $2.25$2.27
SSP 3$5.00$1.48, $1.47$1.48Loss
SSP 4No bid$2.30, $2.25$2.25
SSP 5$5.00$2.52, $2.25$2.52Win$2.52

This strategy proves to be optimal. We win the auction for a reasonable price, and we pay the 2nd price across all of the SSPs as we should (note that this doesn't always happen - but that's for another article).

How does this look for our buyer?

PublisherCPMConversion rateCPA
Auto Trader$2.250.02%$11.20
Ebay$2.650.025%$10.60
Daily Mail$2.520.04%$6.30

Now Daily Mail has the best-performing inventory and it passes through to the buyer with the lowest CPA. Any human buyer or algorithm will see this and shift spend to Daily Mail, increasing CPMs and fill rate over time. CPMs and fill rates will grow as buyer see excellent results.

Supply Path Optimization

Over the past couple of months, the Advertiser Technology Group (ATG) team at AppNexus has experimented with penalizing SSPs that operate non-second-price auctions, which we call "Supply Path Optimization". Note that AppNexus has a separate Publisher Technology Group, responsible for our SSP and full-stack ad serving solutions, and we apply the same logic to our own SSP.

The recipe:
1. Choose a publisher that has a large number of SSPs reselling its inventory via header bidding
2. Look at traffic and win patterns across different formats and geos, looking for situations where a particular SSP is using aggressive auction tactics
3. Stop buying on the open exchange for the sub-optimal SSPs (deals will still work)

Here are the results for one of the first publishers we tried this on. We implemented SPO on August 2.

We see very little impact on overall spend or CPMs day over day. Looking more deeply at the data, the single most important driver of CPMs and spend is allocation of spend by major buyers. In other words, a major buyer deciding to remove a site from a particular campaign for underperformance far outweighs the impact of removing SSPs. This is why you see spend increasing a few days after the change: buyers are seeing better performance and shifting spend toward this particular publisher.

How does this impact individual SSPs? In the chart below, each pair of bars represents an SSP. The blue bar is before SPO; the orange bar is after SPO. As you can see, spend shifted away from certain SSPs toward others. This particular publisher has different "best" SSPs in different markets and formats, so this isn't quite as clean as the Strategy 3 example above, but the outcome is as we expected.
As a side note, SSPs have definitely noticed the changes we've made. One CEO came by my office and said "Wow! You guys have doubled your spend in the past couple of weeks!" Another CEO emailed me and said "What gives? Your spend is down 30% in the past month." I told both of them the same thing: implement fair, second-price auction dynamics and we will route spend your way.

Next up on our roadmap is to automate this technology and roll it out across the wider marketplace.

Recommendations and Insights

I see many correlations between the initial years of online travel agents (OTAs) like Orbitz, Travelocity, and Expedia and the initial years of SSPs. In the early days, they operated as supply aggregators, and often arbitraged the airlines to their consumers. With the launch of Kayak in 20042, the ecosystem flipped. Instead of benefiting the OTA that could charge the highest prices and spend the most on marketing, now consumers were routed to the OTA that charged the lowest prices. This allowed airlines to change their yield management strategies to maximize revenue, and put intense pressure on the OTA business model, resulting in consolidation (Expedia now owns both Orbitz and Travelocity).

Supply path optimization is effectively implementing Kayak for programmatic, giving buyers the ability to route their spend in the most cost-effective way. Just as in the airline business, where the same seat on the same flight used to cost a different amount on each OTA, we live today in a world where the same exact impression costs a different amount through different SSPs. That is going to change, and it is incredibly good for the marketplace3. Publishers and advertisers will benefit from greater efficiency and transparency, allowing them to increase their investment in the programmatic ecosystem.

Some recommendations and insights for each marketplace participant:

Publishers

  • Just because an SSP spends the most money doesn't mean they're doing the best job representing your interests.
  • Buyer allocation is the most important driver of spend, and it may take weeks for buyers to reallocate spend. An aggressive SSP could look great short-term but be hurting buyer performance significantly, causing buyer allocations to shift away.
  • Keep your header bidding wrapper separate from the header bidders. Use an open source solution like prebid.js or the Index Wrapper that doesn't play games with auction dynamics.

Buyers

  • Be aggressive about finding the best path to supply. This is challenging, and likely requires using a DSP with built-in supply path optimization (shameless plug: like AppNexus!)
  • Treat SSPs as pipes, not as media companies. If an SSP sends salespeople to your office, be very concerned. And definitely, do not give insertion orders to SSPs unless you plan to turn them off via your DSP - this creates another route for self-competition that supply path optimization can't address.
  • Don't use your DSP as an SSP. DSPs have significant operational disadvantages in working with publishers since it's not their core business, and runs the risk of suboptimal integration with publishers as we see increased throttling of header bidders based on fill rate.

SSPs

  • An athlete that uses performance-enhancing drugs will win some races - but damages the sport and her reputation. Don't mess with auction dynamics.
  • Look for ways to add real value4 to buyers and publishers.
Acknowledgements

Thank you to many people for reviewing and providing feedback on this including Andrew Casale, Joe Kowan, Walter Knapp, Karl Bunch, Tom Shields, Catherine Williams, Ben Kneen, and Gabriel Weintraub.

Footnotes
  1. The idea of an aggressive SSP isn't hypothetical. Every time I tried this example on Daily Mail the same SSP did win, even though our bidding pattern is identical.

  2. Kayak was co-founded by Steve Hafner, who helped found Orbitz. I see parallels given my role launching the first SSP and now building a meta-SSP optimizer...

  3. Incentivizing publishers to drive better ROI for buyers is a great way to make the marketplace work better. I think this is the dynamic we see on Uber and AirBNB, where drivers are penalized for having low ratings by customers. The allocation process does penalize bad sellers in the ad marketplace, but it's opaque to the seller. I'm going to think about how to create a "buyer happiness index" so publishers can see how they look to buyers.

  4. I have serious concerns that the proliferation of SSPs is short-lived once buyers start implementing intelligent supply path optimization strategies. It's easy to say "add value" but I don't know that there are a lot of options other than to layer proprietary data onto the transaction, operate in unique formats, or create some kind of vertical strategy that aggregates niche inventory.