Herding Packets

Reader Feedback!

Recently, I posted about some lab verification I did during a customer setup where I had to strip private BGP AS numbers (64512-65535) before sending BGP-learned prefixes upstream to an ISP.

While I used the purpose-built “remove-private-as” neighbor command, several readers commented to me via the blog and via Twitter with some ideas for alternate solutions. They even got me thinking about an idea of my own.

This was the original topology, in case you need a little refresher:

First, a response from Twitter with a couple ideas:

@BobMcCouch 2 other solution could be , AS-Path prepend last AS , and as override I think.

— Orhan ERGUN (@OrhanErgunCCIE) August 19, 2013

Next, Jerome Tissieres (another networking Tweep I know) commented on my original post:

Did you consider the possibility to use confederations for that?
The result would be the Seller_Corp AS as confederation main AS and the Buyer_Corp AS a confed private AS, so “invisible” to the external peers.

I’m always a fan of coming up with multiple ways to do something. It stretches your mind a bit, and it can be good to have plans B, C, and sometimes D in case Plan A doesn’t work out. As hard as it was, this one something I really enjoyed when preparing for the CCIE lab — coming up with all those twisted ways to pull something off.

Even though I had some opinions on which of these might or might not work, I decided to run each of these test cases through my lab. This is one of the beautiful things about virtual labs (like the forthcoming Cisco VIRL), you can save a complete topology — connections, configs, and all — and load it back up in the future for more testing.

The Lab

Again, as described in my previous post, a simple BGP setup mimicking a portion of the real-life topology. Here’s the actual workspace I used in GNS3:

We need to get ISP to learn the 4.4.4.0/24 prefix, originated by AS65001, but the ISP is unaware of AS65001 so we must hide that ASN and make the prefix appear to come from SELLER’s ASN, 64512. In our base configuration, ISP is learning the prefix, but it is learning is with BUYER’s AS 65001 in the path which would not fly in real life as the ISP should be filtering any other BGP ASNs from advertisements received from their customer SELLER:

R2’s BGP config:

R2#show run | s router bgp
router bgp 64512
 bgp log-neighbor-changes
 network 2.2.2.0 mask 255.255.255.0
 neighbor 12.12.12.1 remote-as 2001
 neighbor 23.23.23.3 remote-as 64512
 neighbor 23.23.23.3 next-hop-self

AS64512’s view of the world:

R2#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       12.12.12.1               0             0 2001 i
*> 2.2.2.0/24       0.0.0.0                  0         32768 i
*>i3.3.3.0/24       23.23.23.3               0    100      0 i
*>i4.4.4.0/24       23.23.23.3               0    100      0 65001 i

ISP’s view:

R1#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       0.0.0.0                  0         32768 i
*> 2.2.2.0/24       12.12.12.2               0             0 64512 i
*> 3.3.3.0/24       12.12.12.2               0             0 64512 i
*> 4.4.4.0/24       12.12.12.2                             0 64512 65001 i

Again, this works from a connectivity standpoint in the lab but in real life ISP’s BGP policy would prevent this from working.

Test 1: Prepend Last-AS

Personally, I was a bit skeptical of this idea, but I thought it was worth a try. Prepends are applied through the “set” action of a route map, like so:

route-map BGP-OUT permit 10
 set as-path prepend last-as 4
!
router bgp 64512
 neighbor 12.12.12.1 route-map BGP-OUT out

R2#show route-map
route-map BGP-OUT, permit, sequence 10
  Match clauses:
  Set clauses:
    as-path prepend last-as 4
  Policy routing matches: 0 packets, 0 bytes

So we wait for BGP to refresh and check R2:

R2#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       12.12.12.1               0             0 2001 i
*> 2.2.2.0/24       0.0.0.0                  0         32768 i
*>i3.3.3.0/24       23.23.23.3               0    100      0 i
*>i4.4.4.0/24       23.23.23.3               0    100      0 65001 i

And R1, the ISP:

R1#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       0.0.0.0                  0         32768 i
*> 2.2.2.0/24       12.12.12.2               0             0 64512 i
*> 3.3.3.0/24       12.12.12.2                             0 64512 i
*> 4.4.4.0/24       12.12.12.2                             0 64512 65001 65001 65001 65001 65001 i

Unfortunately this one is a no-go. It simply prepends the AS_PATH using the ASN of the most-recently-appended (or next-hop) AS on the prefix, rather than some arbitrary list.

Test 2: AS-Override

The AS-Override feature sounds promising. But when we try to configure it…

R2(config)#router bgp 64512
R2(config-router)#neigh 12.12.12.1 as-o
R2(config-router)#neigh 12.12.12.1 as-o?
% Unrecognized command

You may even notice that the command reference entry for this command is under the MPLS section and not the BGP section of the Doc-CD. That’s because this is an MPLS VPN-specific feature. It’s only applicable under the IPv4 VRF BGP address family. It’s intended for PE-CE routing in the case where all customer sites in the VPN have the same ASN assigned by the provider. In fact, as I recall, it only works by replacing any occurrences the eBGP neighbor’s ASN in the AS_PATH with the local ASN during an advertisement so VRF requirements aside this would probably only work if the ISP was using the same ASN as BUYER.

Test 3: BGP Confederations

Now we’re on to something! BGP confederations can absolutely do exactly this… they provide a means for several “sub-AS” networks to work together and pretend to external networks that they are really one. Here’s the setup for this. I recommend reading up on BGP confederations if you’re not able to figure out roughly what the commands are doing. Any CCIE candidate should be well-familiar with confeds.

R2(config)#router bgp 64512
R2(config-router)#bgp confederation id 64512

R3(config)#router bgp 64512
R3(config-router)#bgp confederation id 64512
R3(config-router)#bgp confederation peer 65001
R3(config-router)#neigh 34.34.34.4 next-hop-self

R4(config)#router bgp 65001
R4(config-router)#bgp confederation identifier 64512
R4(config-router)#bgp confederation peers 64512
R4(config-router)#^Z
R4#show run | s router bgp
router bgp 65001
 bgp log-neighbor-changes
 bgp confederation identifier 64512
 bgp confederation peers 64512
 network 4.4.4.0 mask 255.255.255.0
 neighbor 34.34.34.3 remote-as 64512

After applying all these changes, we can take a look at the state of the network. We see that AS65001 is recognizing the confederation learned routes (notated with the parentheses around the ASN in the AS_PATH):

R4#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       34.34.34.3               0    100      0 (64512) 2001 i
*> 2.2.2.0/24       34.34.34.3               0    100      0 (64512) i
*> 3.3.3.0/24       34.34.34.3               0    100      0 (64512) i
*> 4.4.4.0/24       0.0.0.0                  0         32768 i

R2 is also recognizing that AS65001 is now part of its confederation:

R2#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       12.12.12.1               0             0 2001 i
*> 2.2.2.0/24       0.0.0.0                  0         32768 i
*>i3.3.3.0/24       23.23.23.3               0    100      0 i
*>i4.4.4.0/24       23.23.23.3               0    100      0 (65001) i

Finally, we see that R2 has “fooled” R1 into thinking that all the prefixes it’s advertising are coming from a single, unified AS, 64512:

R1#show ip bgp | b Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       0.0.0.0                  0         32768 i
*> 2.2.2.0/24       12.12.12.2               0             0 64512 i
*> 3.3.3.0/24       12.12.12.2                             0 64512 i
*> 4.4.4.0/24       12.12.12.2                             0 64512 i

So this is a viable option. I still wouldn’t have used it. Why? Mainly because for my use case it’s like using a wrench to hammer in a nail. It will work, but it’s awkward. The two networks I’m working on here are not really one. They are separate businesses, intending to operate separate networks, under separate administrative control and policy. They just have to cooperate for a few months. Using a confederation would require basically telling the BGP routers from both networks that they’re really part of the same club. That changes some of their BGP behaviors, and it’s just not reflective of how the networks really operate. So big kudos to Jerome for coming up with a working alternative, totally fit for a CCIE scenario! But I don’t see the value gained in exchange for adding this complexity/weirdness for my specific scenario.

Closing Thoughts

Any time you’re evaluating design options for production networks, take a few minutes to consider some alternatives. Sometimes the other ideas you come up with just flat out won’t work. Sometimes, they may be valid but may not be the best solution. And sometimes, you’ll realize there is a better option than your original idea. If you consider these options up front (and take the time to lab validate them, if needed), you’ll be less likely to wonder if you should have taken a different course of action and you’ll also have some ground to stand on if your chosen design is challenged by someone since you can show that you evaluated several options and ultimately chose the one you implemented. You also get the benefit of exploring features or use cases that may not be a help this time around, but could serve you well in the future.