DrPeering –
What is Remote Peering?
Josey Wales, Missouri Bushwacker
Josey - thanks for the question as it allows me to discuss an emerging trend in the form of a Great Remote Peering Debate.
The Great Remote Peering Debate
As Internet Transit prices bob around $1 per megabit per second, ISPs and content providers continue to ask the question:
“Does peering makes sense anymore?”
To examine this we typically look at the peering versus transit graph shown below. Since Internet Peering has a fixed monthly cost, the unit cost of peered traffic (MonthlyCostOfPeering/PeeredTrafficVolume) decreases proportionate to the amount of traffic peered. Competing against peering is the metered Internet Transit service, which is a tiered service that has been dropping in price between 10 and 40% per year. This tradeoff is summarized in the Peering vs. Transit graph.
The point at which the unit cost of peering is the same as the unit price of transit is called the peering break even point where an network operator is financially indifferent between peering and transit. However, the underlying assumptions of the peering model are changing.
The Traditional Internet Peering Model
Traditionally, peering involves selecting an Internet Exchange Point and extending a network by leasing transport (e.g. circuits) into the IXP, and colocating a router in a colocation facility with a popular peering fabric. These monthly costs are shown in the figure below.
The Traditional Peering Model Challenge
One challenge that has always faced the traditional peering regime is that the cost of peering is relatively static compared to the perpetually falling price for Internet Transit. For example, colocation fees don’t tend to drop at the same rate as the price drops of transit. If colocation fees did drop at the same rate as Internet Transit, then powered racks would be priced today at about $1.20 per month. (The price of powered racks at network dense colocation centers remains between $1000-$2000/rack).
Routers and transport have dropped in price but not consistently, and not at the rate of transit price drops.
Peering port prices are under the control of exchange point operators and have historically dropped at a similar rate to transit price drops. These port price decreases have been largely driven by the European IXPs operating as not-for-profit associations both internal and external pressures to not make too much profit. This European pricing pressure extends overseas where the other IXPs are pressured to reduce their prices in kind. How often have we heard questions like “Why is my European xxxG port costing me $2500/month and my U.S. xxxG port is $4000/month? These peering port drops have lead to a delayed but inevitable drop in U.S. peering port prices, but with a slight time lag behind European IXP price drops.
In summary, while the cost of peering components under the control of the IXPs have dropped, the total monthly peering costs have not kept pace with the price drops of transit. This has been a continuous struggle over time, and the end result is that peering looks less financially attractive unless one can exchange a large number of bits across the peering infrastructure.
As the price of Internet Transit drops, the peering breakeven point shifts to the right, showing that one needs to be able to peer away more traffic for free for peering to be financially prudent.
For large volume traffic exchange, peering easily still makes financial sense, but the rest of the industry always faces these dynamics and the question “Does peering make sense anymore?”.
How Can Peering Compete with Transit?
From a purely financial perspective, peering makes sense as the unit cost of peering drops at a rate faster than the drop in the unit price of transit.
From a performance perspective, peering may provide sufficient benefits (control over routing, lower latency, lower packet loss, greater visibility, etc.) to make it acceptable for peering to cost more than transit.
Some segments of the market seem to be adopting a hybrid approach of blending Internet Transit and “Remote Peering.”
The Remote Peering Model
One way to reduce the monthly cost of peering is an increasingly popular method of peering called “Remote Peering.”
Definition: Remote Peering is peering without a physical presence at the peering point.
A remote peer connects to the peering fabric without colocating a router at the IXP, and therefore, without the expense of equipment or rack space. There is no deployment cost, no “remote hands” costs to deploy additional peering routers or to turn up circuits. Companies like IX Reach, Atrato, and others are providing access to multiple IXPs using a single router interface at a single metered rate (with commits). The remote peering model is shown architecturally below.
Remote Peering Today
Adoption of Remote Peering. According to Job Witteman (CEO, AMS-IX), of the 500+ customers at the AMS-IX, 70 new remote peers have been added in the last year. Remote Peering is no longer a fringe market.
Peering Control without Peering Expense. Zaid Ali (LinkedIn) is a big proponent of remote peering solutions:
“Our engineering team initially expressed concerns about the reliability of a Layer 2 Virtual Private Network (L2VPN) service, but we have found that remote peering solutions like the IX Reach MPLS network, are very stable. Remote peering allows us to be in control of our own destiny and get closer to our eyeballs; with transit we don’t have control of the routes.”
Implementation of the Remote Peering Model
From the remote peering customer perspective, the router’s network interface has several sub interfaces, each essentially a VLAN extension to the remote peering fabric. This is how LinkedIn gets its content at least one AS hop closer to the access networks that are peering at IXPs across Europe.
IX Reach also streamlined the administrative process of joining multiple IXPs, handling the paperwork and speeding the deployment far faster than if LinkedIn had to deploy routers, circuits, and review, adjust, and execute contracts for colocation and IXP various services with all of these deployment facilities. Turning up peering was a lot faster with the remote peering model.
The Business Case for Remote Peering
Most of the remote peering cost reduction comes from removing the colocation, circuits, and equipment costs from the peering equation. Even if the MPLS metered cost is greater than the cost of transit, the network deployment may still make sense for those that want to peer to control routing to improve the end-user experience. Consider two scenarios: remote peering from an expensive peering ecosystem and remote peering from an inexpensive peering ecosystem.
Case 1 : Remote Peering from an Expensive Internet Peering Ecosystem
Here are some price points collected from a content provider that compared traditional peering and remote peering from a European city where the transport and transit services were relatively expensive into well-populated exchange points where the price of transit and circuits was inexpensive (Frankfurt, London, Amsterdam, etc.).
The fact that the circuits were so expensive in this scenario makes remote peering seem a much cheaper option than the traditional peering deployment model. The bulk of these traditional peering model costs were removed from the peering equation and replaced with a metered rate to access four IXPs from one interface.
One will observe that in this context, $6000/month divided by 100Mbps is $60/Mbps, a price higher for peering that is much higher than the market price for transit across Europe today. This content provider remains convinced that the hybrid approach (buying cheap commodity transit while remote peering away strategic traffic sets) is a sound performance-enhancing strategy. As with every content provider I have worked with,
“the #1 priority is to improve the end-user experience. “
With their blend of relatively inexpensive transit along with selective remote peering to reach key players, they say that they are very satisfied today with their price/performance ratio.
Case 2 : Remote Peering from an Inexpensive Internet Peering Ecosystem
Price points for remote peering are much lower in more mature Internet Peering Ecosystems such as Frankfurt, Amsterdam and London. In these peering hot spots you can get a 1Gbps remote peering service and reach the major exchange points across Europe for a price closer to $1250/month. At the same time, note that the circuit costs that remote peering would allow you to avoid is much lower (only a few thousand dollars per month as opposed to ten’s of thousand’s per month.)
In both cases though, the logic is the same; remote peering removes the colocation, equipment and circuit cost from the equation and replaces those costs with a metered transport alternative.
The Great Remote Peering Debate
It couldn’t call this a debate without ferreting out some other viewpoints on remote peering, and as usual, it turns out there are strongly held, who are we kidding, almost religiously held, beliefs in the peering community about peering methodologies. Over a few beers and dinners at recent Internet operations meetings I teased out a few of these contrary views of remote peering. Here are some of the criticisms expressed about remote peering:
The counter argument seems to be that poor routing can be obtained equally at layer 2 and layer 3. It is an network engineering decision as to how to implement its interconnect strategy.
So there you have it - whether you embrace remote peering or not, you need to understand that it exists and seems to be getting more popular. I expect that there will be much religious debate about this trend. If there is any light generated (rather than the traditional heat) from these discussions, then I will extend this article with additional data. Until then, I hope that this helps.
