Tuesday, November 13, 2007

Cisco IOS: PIM Designated Forwarder (DF)


BI-DIR PIM Designated Forwarder (DF) equal-metric tie-breaker on Cisco IOS


After reading the RFC for BI-DIR PIM (RFC 5015), I didn't find a section describing what would happen if two routers had equal metrics, in regards to the DF election. Obviously the AD and routing metric are used, but what if they're equal - the same? Then what? Well, since I couldn't find it in the RFC, I decided to see how Cisco handled this in their implementation. Below is what I found…


To start with, I created a test network:


BI-DIR PIM Equal-Metric DF Election Example


The relevant portions of the configurations are shown below:



R1:



ip multicast-routing


!


interface FastEthernet1/0


ip address 192.168.1.11 255.255.255.0


ip pim sparse-dense-mode


duplex auto


speed auto


!


interface Serial2/0


ip address 10.0.0.4 255.255.255.254


ip pim sparse-dense-mode


serial restart-delay 0


!


router ospf 1


router-id 1.1.1.1


log-adjacency-changes


network 10.0.0.0 0.0.0.255 area 0


network 192.168.1.0 0.0.0.255 area 0


network 192.168.11.0 0.0.0.255 area 0


!


ip pim bidir-enable






R2:



ip multicast-routing


!


interface FastEthernet1/0


! NOTE THAT BY DEFAULT, THIS PORT IS A MEMBER OF VLAN1 (which is why it's not specified)


interface FastEthernet1/1


! NOTE THAT BY DEFAULT, THIS PORT IS A MEMBER OF VLAN1 (which is why it's not specified)


interface Serial2/0


ip address 10.0.0.1 255.255.255.252


ip pim sparse-dense-mode


serial restart-delay 0


!


interface Vlan1


ip address 192.168.1.1 255.255.255.0


ip pim sparse-dense-mode


!


router ospf 1


router-id 2.2.2.2


log-adjacency-changes


network 10.0.0.0 0.0.0.255 area 0


network 192.168.1.0 0.0.0.255 area 0


network 192.168.2.0 0.0.0.255 area 0


!


ip pim bidir-enable






R3:



ip multicast-routing


!


interface Loopback0


ip address 172.16.32.3 255.255.255.255


ip pim sparse-dense-mode


!


interface FastEthernet1/0


ip address 192.168.10.1 255.255.255.0


ip pim sparse-dense-mode


duplex auto


speed auto


!


interface Serial2/0


ip address 10.0.0.2 255.255.255.252


ip pim sparse-dense-mode


serial restart-delay 0


clock rate 128000


!


interface Serial2/1


ip address 10.0.0.5 255.255.255.254


ip pim sparse-dense-mode


serial restart-delay 0


clock rate 128000


!


router ospf 1


router-id 3.3.3.3


log-adjacency-changes


network 10.0.0.0 0.0.0.255 area 0


network 172.16.32.0 0.0.0.255 area 0


network 192.168.10.0 0.0.0.255 area 0


!


ip pim bidir-enable


ip pim send-rp-announce Loopback0 scope 16 bidir


ip pim send-rp-discovery Loopback0 scope 16


!






R4:



ip multicast-routing


!


interface FastEthernet1/0


ip address 192.168.1.4 255.255.255.0


ip pim sparse-dense-mode


ip igmp join-group 239.1.1.5


duplex auto


speed auto


!


router ospf 1


router-id 4.4.4.4


log-adjacency-changes


network 192.168.1.0 0.0.0.255 area 0


!


ip pim bidir-enable






R5:



ip multicast-routing


!


interface FastEthernet1/0


ip address 192.168.10.5 255.255.255.0


ip pim sparse-dense-mode


ip igmp join-group 239.1.1.5


duplex auto


speed auto


!


router ospf 1


router-id 5.5.5.5


log-adjacency-changes


network 192.168.10.0 0.0.0.255 area 0


!


ip pim bidir-enable



To set the stage, we have two equal-cost paths going back to R3 from R1 and R2 (these are not shared, so each acts as its own network segment, therefore each will have a DF assigned). Both R1 and R2 are connected to the same ethernet segment via their FastEthernet ports. Note that R2 is acting as the switch for this network segment, so the IP address is assigned to VLAN1 so that both R1 and R4 have connectivity to R2. The network segment connecting R1, R2 and R4 together is where the DF election process will show what acts as a tie-breaker. Let's continue…


Looking at the IP routing table on R2:




r2#sho ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

O 192.168.10.0/24 [110/65] via 10.0.0.2, 00:14:05, Serial2/0
172.16.0.0/32 is subnetted, 1 subnets
O 172.16.32.3 [110/65] via 10.0.0.2, 00:14:05, Serial2/0
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 10.0.0.0/30 is directly connected, Serial2/0
O 10.0.0.4/31 [110/65] via 192.168.1.11, 00:14:05, Vlan1
C 192.168.1.0/24 is directly connected, Vlan1
r2#



Look at the IP routing table on R1:




r1#sho ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

O 192.168.10.0/24 [110/65] via 10.0.0.5, 00:15:01, Serial2/0
172.16.0.0/32 is subnetted, 1 subnets
O 172.16.32.3 [110/65] via 10.0.0.5, 00:15:01, Serial2/0
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O 10.0.0.0/30 [110/65] via 192.168.1.1, 00:15:01, FastEthernet1/0
C 10.0.0.4/31 is directly connected, Serial2/0
C 192.168.1.0/24 is directly connected, FastEthernet1/0
r1#



Lastly, look at R4's IP routing table:




r4#sho ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

O 192.168.10.0/24 [110/66] via 192.168.1.11, 00:17:22, FastEthernet1/0
[110/66] via 192.168.1.1, 00:17:22, FastEthernet1/0
172.16.0.0/32 is subnetted, 1 subnets
O 172.16.32.3 [110/66] via 192.168.1.11, 00:17:22, FastEthernet1/0
[110/66] via 192.168.1.1, 00:17:22, FastEthernet1/0
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O 10.0.0.0/30 [110/65] via 192.168.1.1, 00:17:22, FastEthernet1/0
O 10.0.0.4/31 [110/65] via 192.168.1.11, 00:17:22, FastEthernet1/0
C 192.168.1.0/24 is directly connected, FastEthernet1/0
r4#



Looking at the above IP routing tables, R4 will never have a chance in the DF election process on this ethernet segment - the metrics are higher than R1 and R2. R1 and R2 are different though - their routing tables are nearly identical, except for the point-to-point serial links. Looking at network 192.168.10.0/24, both have the same AD and both share the same metric. Equal-cost paths - what now?


Take a look at the output from R1 and R2:




r1#show ip pim int df
* implies this system is the DF
Interface RP DF Winner Metric Uptime
FastEthernet1/0 172.16.32.3 *192.168.1.11 65 00:04:36
Serial2/0 172.16.32.3 10.0.0.5 0 00:21:30
r1#


r2#sho ip pim int df
* implies this system is the DF
Interface RP DF Winner Metric Uptime
Serial2/0 172.16.32.3 10.0.0.2 0 00:21:09
Vlan1 172.16.32.3 192.168.1.11 65 00:04:42
r2#



It appears that R1 is the DF for the ethernet segment. Let's look at the output from the debug ip pim df command while we change R2's Vlan1 interface IP address to 192.168.1.100/24:




r2#debug ip pim df
PIM RP DF debugging is on
r2#conf t


r2(config)#int vlan1


r2(config-if)#ip address 192.168.1.100 255.255.255.0



Now, the debug output (there was a lot, so I've eliminated some of the redundant messages and have removed the routing protocol messages as they had to renegotiate as the interface IP address changed):




! BELOW, I'd changed the IP from 192.168.1.100 back to 192.168.1.1, so I believe it was still negotiating the DF from this change.


! I left this part as it shows the output when another router (.11) is the DF.


*Mar 1 00:27:17.651: PIM(0): Send v2 Offer on Vlan1 (Non-DF) for RP 172.16.32.3
*Mar 1 00:27:17.655: PIM(0): Sender 192.168.1.1, pref 110, metric 65
*Mar 1 00:27:18.067: PIM(0): Send v2 Winner on Vlan1 (DF) for RP 172.16.32.3
*Mar 1 00:27:18.067: PIM(0): Sender 192.168.1.1, pref 110, metric 65
*Mar 1 00:27:18.087: PIM(0): Receive DF Winner message from 192.168.1.11 on Vlan1 (DF)
*Mar 1 00:27:18.091: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:18.095: PIM(0): Metric is better


! Before the address change back to .100 on R2 takes place, R1 is elected the DF, and the metric is better. Why? Obviously the because of the higher IP address (although the debug output never states this)


*Mar 1 00:27:18.159: PIM(0): Receive DF Offer message from 192.168.1.4 on Vlan1 (Non-DF)
*Mar 1 00:27:18.163: PIM(0): RP 172.16.32.3, pref 2147483647, metric 2147483647
*Mar 1 00:27:18.163: PIM(0): Metric is equal or worse


! R4, as discussed above, will always have higher metrics, so will never become the DF for the Ethernet segment


! What have we below? It appears that the IP address change has taken place. R2 flushes the DF for Vlan1 and begins the election process. Here's where we get to see the debug output from the winner's side!


*Mar 1 00:27:18.223: PIM(0): Flush DF for Vlan1, RP 0.0.0.0
*Mar 1 00:27:18.251: PIM(0): Elect DF for Vlan1, new RP 172.16.32.3
*Mar 1 00:27:18.251: PIM(0): Send v2 Offer on Vlan1 (Non-DF) for RP 172.16.32.3
*Mar 1 00:27:18.255: PIM(0): Sender 192.168.1.100, pref 110, metric 65
*Mar 1 00:27:18.311: PIM(0): Receive DF Winner message from 192.168.1.11 on Vlan1 (Non-DF)
*Mar 1 00:27:18.315: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:18.315: PIM(0): Metric is equal or worse


! R1 is currently the winner (the elected DF), so is sending out winner messages. Well, let's see what happens next...


*Mar 1 00:27:18.343: PIM(0): Receive DF Offer message from 192.168.1.4 on Vlan1 (Non-DF)
*Mar 1 00:27:18.343: PIM(0): RP 172.16.32.3, pref 2147483647, metric 2147483647
*Mar 1 00:27:18.347: PIM(0): Metric is equal or worse


! R4 is attempting to contribute, but due to the higher metric, will never be able to... still, it's sending Offer messages (like it should be)


*Mar 1 00:27:18.347: PIM(0): Receive DF Backoff message from 192.168.1.11 on Vlan1 (Non-DF)
*Mar 1 00:27:18.351: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:18.355: PIM(0): Offer Addr 192.168.1.100, pref 110, metric 65, interval 1
*Mar 1 00:27:18.355: PIM(0): Metric is equal or worse, to us


! Whoa, what's this? We've received a backoff message from the current DF. So far, things are looking good!


*Mar 1 00:27:18.359: PIM(0): Receive DF Offer message from 192.168.1.4 on Vlan1 (Non-DF)
*Mar 1 00:27:18.359: PIM(0): RP 172.16.32.3, pref 2147483647, metric 2147483647
*Mar 1 00:27:18.363: PIM(0): Metric is equal or worse


! Again, R4 putting its two cents' worth in...


*Mar 1 00:27:18.431: PIM(0): Receive DF Pass message from 192.168.1.11 on Vlan1
*Mar 1 00:27:18.435: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:18.435: PIM(0): Winner 192.168.1.11, pref 110, metric 65
*Mar 1 00:27:18.439: PIM(0): Metric is equal or worse, not to us


! Okay, looks like R1 is relinquishing ownership of the DF role...


*Mar 1 00:27:18.443: PIM(0): Receive DF Offer message from 192.168.1.4 on Vlan1 (Non-DF)
*Mar 1 00:27:18.443: PIM(0): RP 172.16.32.3, pref 2147483647, metric 2147483647
*Mar 1 00:27:18.447: PIM(0): Metric is equal or worse
*Mar 1 00:27:18.467: PIM(0): Receive DF Winner message from 192.168.1.11 on Vlan1 (Non-DF)
*Mar 1 00:27:18.471: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:18.471: PIM(0): Metric is equal or worse


! Above, R1 has issued a winner message, but below, we see that the DR has changed.


*Mar 1 00:27:19.307: %PIM-5-DRCHG: DR change from neighbor 192.168.1.11 to 192.168.1.100 on interface Vlan1
*Mar 1 00:27:19.583: PIM(0): Send v2 Offer on Vlan1 (Non-DF) for RP 172.16.32.3
*Mar 1 00:27:19.587: PIM(0): Sender 192.168.1.100, pref 110, metric 65
*Mar 1 00:27:19.911: PIM(0): Receive DF Backoff message from 192.168.1.11 on Vlan1 (Non-DF)
*Mar 1 00:27:19.911: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:19.915: PIM(0): Offer Addr 192.168.1.100, pref 110, metric 65, interval 1
*Mar 1 00:27:19.915: PIM(0): Metric is equal or worse, to us


! Okay, backoff time is over, now declare ourselves the winner:


*Mar 1 00:27:27.715: PIM(0): Send v2 Winner on Vlan1 (DF) for RP 172.16.32.3
*Mar 1 00:27:27.715: PIM(0): Sender 192.168.1.100, pref 110, metric 65


! Now, R1 and R4 will continue to send offer messages, but will not be elected the DF as their metric (R4) is higher or they have a lower IP (R1)


*Mar 1 00:27:30.623: PIM(0): Receive DF Offer message from 192.168.1.11 on Vlan1 (DF)
*Mar 1 00:27:30.627: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:30.631: PIM(0): Metric is equal or worse
*Mar 1 00:27:30.631: PIM(0): Send v2 Winner on Vlan1 (DF) for RP 172.16.32.3
*Mar 1 00:27:30.635: PIM(0): Sender 192.168.1.100, pref 110, metric 65
*Mar 1 00:27:30.747: PIM(0): Send v2 Offer on Vlan1 (Non-DF) for RP 172.16.32.3
*Mar 1 00:27:30.747: PIM(0): Sender 192.168.1.100, pref 110, metric 65
*Mar 1 00:27:34.947: PIM(0): Receive DF Offer message from 192.168.1.11 on Vlan1 (DF)
*Mar 1 00:27:34.951: PIM(0): RP 172.16.32.3, pref 110, metric 65
*Mar 1 00:27:34.955: PIM(0): Metric is equal or worse
*Mar 1 00:27:34.955: PIM(0): Send v2 Winner on Vlan1 (DF) for RP 172.16.32.3
*Mar 1 00:27:34.959: PIM(0): Sender 192.168.1.100, pref 110, metric 65



Looking at the above debug output and at RFC 5015 which defines BI-DIR PIM, it appears that things are congruent and working as intended.


In conclusion, the part that I couldn't find spelled out in the BI-DIR PIM RFC (5015) in terms of what determines the DF winner when a tie occurs with the routing metrics, Cisco has chosen to use the highest IP address. I didn't try using a secondary IP address to see if that would act as a tie-breaker, but I doubt it as most routing protocols don't use or allow the secondary IP address to be taken into account in terms of the routing protocol.


Let me know if you have any feedback, comments or suggestions on this! I did search RFC 5015 and didn't find a specification for a tie-breaker - if it's there, let me know!