Route Table Profiling - 6200networks.com
Route Table Profiling
Posted in November 7th, 2007, by Joe Harris in Routing/Switching, General
How many of you knew there was a command that actually monitored route table fluctuations’? Well there is, it’s called Route Profiling and the feature was developed to assist network engineers in diagnosing routing table fluctuations, which may be the result of route flapping, network failure, or network service restoration. Originally introduced as a hidden IOS command in 11.1CC, the feature is now visible, however documentation on the command is somewhat sparse to say the least. The Route Table Profiling feature is enabled globally.The command is ‘ip route profile’ in global configuration mode. This feature can be disabled with the command ‘no ip route profile’ in global configuration mode. Routing table change statistics can be viewed with the ‘show ip route profile’ command in exec mode.
The Route Table Profiling feature is a statistics collection mechanism to record routing table churn or routing instabilities.The collection facilities are intended to identify route flapping over brief time intervals. The time interval for each sample, or sampling interval, is a fixed value and is set at 5 seconds.
There are two sets of statistics collected. The per-interval statistics are collected over a sampling interval, while the routing table change statistics are the result of aggregating the per-interval statistics. The per-interval statistics are collected as a single set of counters, with one counter tracking one event. All counters are initialized at the beginning of each sampling interval; counters are incremented as corresponding events occur anywhere in the routing table. Change per-prefix or per-protocol is not tracked.
At the end of a sampling interval, the per-interval statistics for that sampling interval are integrated with the routing table change statistics collected from the previous sampling intervals. The counters holding the per-interval statistics are reset and the process repeats itself. The routing table change statistics are stored with less granularity than the per-interval statistics. The routing table change statistics consist of a table of event types and a set of buckets for various frequencies of occurrence for each event. The frequency is the number of occurrences of an event over a sampling interval. This feature is intended to provide general trending information on routing table fluctuations.
To avoid having a bucket for every frequency, a non-linear algorithm is used to define the frequency values. A step size is defined in addition to a multiplier. The frequency value starts at 0 and increases progressively by an increment. After the number of increments reaches the step size, the multiplier is applied to the increment. This process continues until the maximum frequency is reached. The increment begins at 1, with a step-size of 5, a multiplier of 5, and a maximum frequency of 10000. These values are all fixed. Using the parameters above, the following example will demonstrate how the per-interval statistics are integrated with the routing table change statistics. The frequency buckets are defined based on the non-linear algorithm described earlier. During the sampling interval, 22 new prefixes are added to the routing table, the Prefix Add counter for the sampling interval would be set to 22. When integrating this with the routing table change statistics, the frequency buckets available are either 20 or 25. The frequency bucket of 20 for the Prefix Add event would be incremented by 1. The counter for the event is rounded down to the nearest frequency. A counter of 24 would still increment the frequency 20 and not 25.
Routing table statistics are collected for the following events:
Fwd-Path Change – This is the number of changes in the forwarding path. It is the accumulation of Prefix-Add, Next-Hop Change, and Pathcount Change.
Prefix-Add – A new prefix was added to the routing table.
Next-Hop Change – A prefix is not added or removed, but the next-hop changes. This is only seen with recursive routes that are installed in the routing table.
Pathcount Change – The number of paths in the routing table has changed. This is the result of an increase in the number of paths for an IGP prefix in the routing table.
Prefix Refresh – Standard routing table maintenance, forwarding behavior was not changed.
The following is an example output from “show ip route profile”:
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 221 222 225 222 220
1 0 1 0 1 0
2 1 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 1
10 0 0 0 0 0
15 2 2 0 2 3
20 2 2 0 0 0
25 0 0 0 0 0
30 1 0 0 0 0
55 0 0 0 0 0
R1#
The Change/Interval column represents the frequency buckets. A Change/Interval of 20 represents the bucket that is incremented when a particular event occurs 20 times in a sampling interval. In the previous section, the Prefix Add event occurred 22 times in one interval and 24 times in another interval. The above output represents this with a Fwd-path Change of 2 and a Prefix Add of 2. It is very common to see high counters for the Change/Interval bucket for 0. This counter represents the number of sampling intervals in which there were no changes to the routing table. Route removals are not counted in the statistics, only route additions.
So let’s look at an example. The topology for the example consists of two Cisco 3725 routers with dual FastEthernet connections in a back to back setup. EIGRP and BGP are used to demonstrate the various forms in which routing table information can be changed. Routing information is sourced from R2. Route table profiling is performed on R1. The initial configuration will involve an IBGP session between R1 and R2. The IBGP peering is performed between loopback addresses, with EIGRP providing the internal reachability. A group of loopback addresses (loop1 – loop16) will be redistributed into BGP on R2.
R2#show running-config | begin eigrp
router eigrp 50
network 172.16.2.0 0.0.0.255
network 192.168.1.0
network 192.168.2.0
no auto-summary
eigrp event-logging
!
router bgp 50
no synchronization
bgp log-neighbor-changes
neighbor 172.16.1.1 remote-as 50
neighbor 172.16.1.1 update-source Loopback0
no auto-summary
!
R2#sh ip int brief | begin Loopback1
Loopback1 10.2.1.2 YES NVRAM up up
Loopback2 10.2.2.2 YES NVRAM up up
Loopback3 10.2.3.2 YES NVRAM up up
Loopback4 10.2.4.2 YES NVRAM up up
Loopback5 10.2.5.2 YES NVRAM up up
Loopback6 10.2.6.2 YES NVRAM up up
Loopback7 10.2.7.2 YES NVRAM up up
Loopback8 10.2.8.2 YES NVRAM up up
Loopback9 10.2.9.2 YES NVRAM up up
Loopback10 10.2.10.2 YES NVRAM up up
Loopback11 10.2.11.2 YES NVRAM up up
Loopback12 10.2.12.2 YES NVRAM up up
Loopback13 10.2.13.2 YES NVRAM up up
Loopback14 10.2.14.2 YES NVRAM up up
Loopback15 10.2.15.2 YES NVRAM up up
Loopback16 10.2.16.2 YES NVRAM up up
R2#
!
R1#show running-config | begin eigrp
router eigrp 50
network 172.16.1.0 0.0.0.255
network 192.168.1.0
network 192.168.2.0
no auto-summary
eigrp event-logging
!
router bgp 50
no synchronization
bgp log-neighbor-changes
neighbor 172.16.2.2 remote-as 50
neighbor 172.16.2.2 update-source Loopback0
no auto-summary
!
ip route profile
!
The initial state of the route table profiling information reflects no changes in routing information have occurred since the profiling was enabled.
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 33 33 33 33 33
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
10 0 0 0 0 0
15 0 0 0 0 0
20 0 0 0 0 0
25 0 0 0 0 0
30 0 0 0 0 0
55 0 0 0 0 0
R1#
The BGP session between R2 and R1 is already up and established however at this point we now redistribute the 15 loopback interfaces into the BGP process on R2 and those routes will be received by R1. Lets see if this changes the route profiling on R1. I am going to perform a redistribute connected with a route-map specified that only includes the ip addresses associated with Loopback1 - 16.
R2(config)#access-list 10 permit 10.2.0.0 0.0.255.255
R2(config)#route-map Red-Loop permit 10
R2(config-route-map)#match address 10
R2(config-route-map)#end
R2#
R2#config t
Enter configuration commands, one per line. End with CNTL/Z.
R2(config)#router bgp 50
R2(config-router)#redistribute connected route-map Red-Loop
R2(config-router)#end
R2#
R2#sh ip bgp
BGP table version is 17, local router ID is 172.16.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete
Network NextHop Metric LocPrf Weight Path
*> 10.2.1.0/24 0.0.0.0 0 32768 ?
*> 10.2.2.0/24 0.0.0.0 0 32768 ?
*> 10.2.3.0/24 0.0.0.0 0 32768 ?
*> 10.2.4.0/24 0.0.0.0 0 32768 ?
*> 10.2.5.0/24 0.0.0.0 0 32768 ?
*> 10.2.6.0/24 0.0.0.0 0 32768 ?
*> 10.2.7.0/24 0.0.0.0 0 32768 ?
*> 10.2.8.0/24 0.0.0.0 0 32768 ?
*> 10.2.9.0/24 0.0.0.0 0 32768 ?
*> 10.2.10.0/24 0.0.0.0 0 32768 ?
*> 10.2.11.0/24 0.0.0.0 0 32768 ?
*> 10.2.12.0/24 0.0.0.0 0 32768 ?
*> 10.2.13.0/24 0.0.0.0 0 32768 ?
*> 10.2.14.0/24 0.0.0.0 0 32768 ?
*> 10.2.15.0/24 0.0.0.0 0 32768 ?
*> 10.2.16.0/24 0.0.0.0 0 32768 ?
R2#
At this point R2 has the routes in its BGP table and advertises them to R1…R1 should now see an increment of 15 in the router profile.
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 94 94 95 95 95
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
10 0 0 0 0 0
15 1 1 0 0 0
20 0 0 0 0 0
25 0 0 0 0 0
30 0 0 0 0 0
55 0 0 0 0 0
R1#
The 15 prefixes are received and added to the table. The bucket corresponding to 15 prefixes is incremented by 1. During a single five-second period, those 15 prefixes were inserted into the routing table. This resulted in 15 Prefix Add and 15 Fwd-Path Change events. Now let’s clear the BGP session between the routers. This results in the prefixes being removed and re-added. The prefix removal was not tracked in the profiling, only the prefix additions. The result is that a flapping route is not double counted, but only counted on insertion into the routing table.
R1#clear ip bgp *
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 969 969 970 970 970
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
10 0 0 0 0 0
15 2 2 0 0 0
20 0 0 0 0 0
25 0 0 0 0 0
30 0 0 0 0 0
55 0 0 0 0 0
R1#
The routing table has the 15 BGP routes, 10.2.1.0/24 – 10.2.16.0/24. The next-hop attribute for all of these routes is 172.16.2.2, the IP address assigned to the loopback0 interface on R2. Changes to the next-hop attribute are tracked for BGP injected prefixes. Changes to next-hop for prefixes injected in the routing table are not tracked as next-hop changes, but as prefix additions.
In order to change the next-hop attribute for the BGP routes, a route-map will be applied outbound on R2 to set the next-hop to 172.16.2.222. The sessions will then be soft cleared on R1, to prevent it from performing a prefix removal and addition. R1 will change only the next-hop attribute, which will be reflected in the routing table and the route table profiling statistics.
R1#sh ip route
Gateway of last resort is not set172.16.0.0/24 is subnetted, 2 subnets
C 172.16.1.0 is directly connected, Loopback0
D 172.16.2.0 [90/409600] via 192.168.2.2, 01:32:05, Fast0/1
[90/409600] via 192.168.1.2, 01:32:05, Fast0/0
10.0.0.0/24 is subnetted, 16 subnets
B 10.2.8.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.9.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.10.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.11.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.12.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.13.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.14.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.15.0 [200/0] via 172.16.2.2, 00:09:18
B 10.2.1.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.2.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.3.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.4.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.5.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.6.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.7.0 [200/0] via 172.16.2.2, 00:09:19
B 10.2.16.0 [200/0] via 172.16.2.2, 00:09:18
C 192.168.1.0/24 is directly connected, Fast0/0
C 192.168.2.0/24 is directly connected, Fast0/1
R1#
R2(config)#route-map HOP permit 10
R2(config-route-map)#set ip next-hop 172.16.2.222
R2(config-route-map)#exit
R2(config)#router bgp 50
R2(config-router)#neighbor 172.16.1.1 route-map HOP out
R2(config-router)#end
R2#clear ip bgp * soft out
R2#
!
!
R1#sh ip route
Gateway of last resort is not set172.16.0.0/24 is subnetted, 2 subnets
C 172.16.1.0 is directly connected, Loopback0
D 172.16.2.0 [90/409600] via 192.168.2.2, 01:32:05, Fast0/1
[90/409600] via 192.168.1.2, 01:32:05, Fast0/0
10.0.0.0/24 is subnetted, 16 subnets
B 10.2.8.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.9.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.10.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.11.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.12.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.13.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.14.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.15.0 [200/0] via 172.16.2.222, 00:00:04
B 10.2.1.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.2.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.3.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.4.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.5.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.6.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.7.0 [200/0] via 172.16.2.222, 00:00:05
B 10.2.16.0 [200/0] via 172.16.2.222, 00:00:05
C 192.168.1.0/24 is directly connected, Fast0/0
C 192.168.2.0/24 is directly connected, Fast0/1
R1#
!
!
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 1493 1494 1495 1496 1495
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
10 0 0 0 0 0
15 3 2 1 0 1
20 0 0 0 0 0
25 0 0 0 0 0
30 0 0 0 0 0
55 0 0 0 0 0
R1#
The Pathcount Change is related to multiple paths for a prefix injected by an IGP. As the above profiling statistics indicates, there has been routing table churn through the addition and removal of prefixes and changes in the next-hop information. However, there have not been any Pathcount Changes. At this point, the static routes on R2 will not be injected into BGP anymore and will be advertised via EIGRP.
R2(config)#router eigrp 50
R2(config-router)#red
R2(config-router)#redistribute con
R2(config-router)#redistribute connected r
R2(config-router)#redistribute connected route-map Red-Loop
R2(config-router)#end
!
R1#sh ip eigrp nei
IP-EIGRP neighbors for process 50
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
1 192.168.1.2 Fa0/0 11 00:06:33 243 1458 0 11
0 192.168.2.2 Fa0/1 10 00:06:33 386 2316 0 10
The show ip eigrp neighbor output from R1 shows the two EIGRP adjacencies between the two routers, providing equal cost paths and enabling load sharing. The routing table after the loopbacks have been injected as External routes is seen next.
R1#show ip route
Gateway of last resort is not set
172.16.0.0/24 is subnetted, 2 subnets
C 172.16.1.0 is directly connected, Loopback0
D 172.16.2.0 [90/409600] via 192.168.2.2, 00:05:40, Fast0/1
[90/409600] via 192.168.1.2, 00:05:40, Fast0/0
10.0.0.0/24 is subnetted, 16 subnets
D EX 10.2.8.0 [170/409600] via 192.168.2.2, 00:00:07, Fast0/1
[170/409600] via 192.168.1.2, 00:00:07, Fast0/0
D EX 10.2.9.0 [170/409600] via 192.168.2.2, 00:00:07, Fast0/1
[170/409600] via 192.168.1.2, 00:00:07, Fast0/0
D EX 10.2.10.0 [170/409600] via 192.168.2.2, 00:00:07, Fast0/1
[170/409600] via 192.168.1.2, 00:00:07, Fast0/0
D EX 10.2.11.0 [170/409600] via 192.168.2.2, 00:00:07, Fast0/1
[170/409600] via 192.168.1.2, 00:00:07, Fast0/0
D EX 10.2.12.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.13.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.14.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.15.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.1.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.2.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.3.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.4.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.5.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.6.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.7.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
D EX 10.2.16.0 [170/409600] via 192.168.2.2, 00:00:09, Fast0/1
[170/409600] via 192.168.1.2, 00:00:09, Fast0/0
C 192.168.1.0/24 is directly connected, FastEthernet0/0
C 192.168.2.0/24 is directly connected, FastEthernet0/1
There is an interesting difference in the profiling statistics. The Fwd-path change hit 30, and the Pathcount change registered a hit of 20 during the sampling period. Each of the prefixes was inserted into the table. This incremented the Prefix Add and Fwd-path change. Then the second path was added, generating another Fwd-path change for each prefix, and a Pathcount change, as the number of paths for each of the prefixes increased from 1 to 2.
R1#sh ip route profile
IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
________________________________________________________
Change/ Fwd-path Prefix Nexthop Pathcount Prefix
interval change add change change refresh
________________________________________________________
0 1493 1494 1495 1496 1495
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
10 0 0 0 0 0
15 3 3 1 1 1
20 0 0 0 0 0
25 0 0 0 0 0
30 1 0 0 0 0
55 0 0 0 0 0
R1#
The final statistic is the Prefix refresh counter. This value is incremented when the routing table is touched in a manner that does not affect forwarding. This does not mean that no changes to forwarding happened. This means that the routing table was modified and the number of prefixes indicated did not have forwarding related changes. In the previous example, there were 2 prefixes during a single sampling period that were refreshed. It was actually a single prefix that was touched 2 times, 172.16.2.2. When changes to the EIGRP topology occur, the entire routing table is touched, or at least the entire routing table that originated from the EIGRP process, as the prefixes are all re-installed. If the prefix is the same, it is refreshed. When the 15 loopbacks were injected, this caused R1 to touch all EIGRP routes in the routing table. The prefix 172.16.2.2 was unmodified when each path was computed, resulting in the counter being incremented twice.
In summary, there has been no testing conducted that I have done or know of which assesses the performance impact of IP Route Profiling. The feature is not in the forwarding path and so performance is not affected by the amount of traffic being switched by the router. The lack of granularity results in a very minimal impact on memory resource utilization. Approximate memory utilization is under 1 kbyte. The processing impact consists of counter increments per prefix when changes are made to the routing table and the periodic statistics accumulation every 5 seconds.
