Network Working Group Internet Draft S.Hares Document: draft-hares-bgp-backoff-01.txt NextHop Technologies, Inc. Expires: August 2004 February 2004 BGP Peer Restart Backoff Mechanisms Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes mechanisms for methods for damping the oscillations of BGP-4 [1] peers by increasing the time between automatic start functions of the BGP peers. The increase of time between automatic start functions will be denoted as a backoff of the automatic start functions by this draft. This draft is an informational RFC that provides a description of mechanisms for back-off of the automatic start function in bgp-4 peer reconnections. This Informational RFC includes descriptions about the back-off mechanisms and how this information is recorded in the BGP-4 MIB version 2 [1]. The exponential back-off mechanism contained in this draft was first recorded in a draft of the bgp-4 specification. The author Hares Expires - August 2004 [Page 1] BGP Peer Restart Backoff Mechanisms February 2004 requests that anyone with information regarding the original authorsof that work contact the author. The lack of credit for those authors is based on the author's lack of knowledge the originators of the expotential back-off. Table of Contents 1. Overview.......................................................2 2. Conventions used in this document..............................3 3. Types of Back-off..............................................4 4. Exponential back-off...........................................4 5. Step-wise Back-off.............................................4 6. Recording the Back-off information in the BGP-4 MIB [].........5 7. Back-off Mechanisms in FSM.....................................6 7.1 Per BGP Peer Variables that are Optional Session attributes6 7.2 Additional Optional Session attributes for peer oscillation damping back-off...............................................7 7.3 Existing MIB objects used..................................8 7.4 IdleHold substate in Idle FSM description..................9 7.5 Action upon entering the Idle state.......................10 7.6 Substate processing.......................................10 Security Considerations..........................................14 References.......................................................14 Acknowledgments..................................................15 Author's Addresses...............................................15 1. Overview If a BGP speaker detects an error, it shuts down the connection and changes its connection state to Idle. Getting out of the Idle state requires generation of the Start event. If such an event is generated automatically, then persistent BGP errors may result in persistent flapping of the speaker. To avoid such a condition it is recommended that automatic start events should not be generated immediately for a peer that was previously transitioned to Idle due to an error. For a peer that was previously transitioned to Idle due to an error, the time between consecutive generation of start events, if such events are generated automatically, shall increase. This increase of time between automatic start events will be called in backoff of automatic BGP peer start events. The automatic start events listed in the BGP-4 specification are: - Event3: Automatic start - Event5: AutomaticStart_with_PassiveTcpEstablishment Hares Expires - August 2004 [Page 2] BGP Peer Restart Backoff Mechanisms February 2004 - Event6: AutomaticStart_with_DampPeerOscillations - Event 7: AutomaticStart_with_DampPeerOscillations_and_ PassiveTcpEstablishment Automatic stop followed by automatic start (events 3, event 5, event 6) can cause the peers to up and down. Events 5 and 6 indicate the optional session attribute DampPeerOscillations is set to true. BGP-4 implementations today have a variety of back-off mechanisms. This document is intended to provide information on these back-off mechanisms and how to record this information in a MIB. Two mechanisms are recorded in this Internet Draft: 1) Exponential back-off (section 2) and 2) Step-wise back-off (section 3). The author welcomes input on additional back-off mechanisms. Please send input to the author or the idr list (idr@ietf.org). A BGP peer oscillation backoff mechanism requires additional Processing in the bgp-4[1] Finite State Machine (FSM). The BGP-4 draft contains an indication of where the FSM would process the peer oscillation damping. The implementation of backoff mechanisms in a BGP-4 is optional, but wide spread in implementations. The exact mechanisms vary between implementations. Section 6.0 contains information on how the BGP-4 MIB encodes the time between establishing BGP-4 peer sessions. Section 7.0 contains the additional mechanisms needed in a BGP-4 FSM for back-off implementations. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. As an informational draft there are no mandatory functions. This draft hopes to recommend prudent mechanisms for back-off mechanisms in BGP peers. As in all recommendations for healthy life styles the user of BGP-4 is not required to do anything, except expire the BGP-4 connection according to the letter of the BGP-4 standard. Hares Expires - August 2004 [Page 3] BGP Peer Restart Backoff Mechanisms February 2004 3. Types of Back-off The author knows of three types of back-off at this time: - Exponential back-off, and - Step-wise back-off, - no back-off. The author welcomes any comments on additional back-offs and will hasten to include this information in the next revision of this draft. 4. Exponential back-off The original description occurred in RFC 1654 and was included in a version of bgp-4 [1] draft. If a BGP speaker detects an error, it shuts down the connection and changes its state to Idle. Getting out of the Idle state requires generation of the Start event. If such an event is generated automatically, then persistent BGP errors may result in persistent flapping of the speaker. To avoid such a condition it is recommended that automatic Start events should not be generated immediately for a peer that was previously transitioned to Idle due to an error. For a peer that was previously transitioned to Idle due to an error, the time between consecutive generation of Start events, if such events are generated automatically, shall exponentially increase. The IdleHoldTimer will be the timer is utilized for the count down of this feature. The value of the initial timer shall be 60 seconds. The time shall be doubled for each consecutive retry. The equation for the setting of the Idle time is as follows: Idle Hold timer = 2**(ConnectRetryCnt)*60 If the Idle Hold timer exceeds a local maximum value, the connection shall be held down until a manual start event occurs. The Idle Hold timer is initialized to zero. 5. Step-wise Back-off Hares Expires - August 2004 [Page 4] BGP Peer Restart Backoff Mechanisms February 2004 The Step-wise back-off uses the following back-off function. The step-wise back-off functions provide a step function based on the number of retries. An example of the Step-wise code, is as follows: - For 1-8 retries, the amount of time between retries is 8 seconds. - For 9-24 retries, the amount of time between retries, the amount time between retries is 64 seconds, - For larger than 25 retries, the amount of time between retries is 128 seconds, - Maximum number of retries is 35. 6. Recording the Back-off information in the BGP-4 MIB [3] The BGP-4 MIB version 2[3] will have two variables for the hold time Maximums: BGP-4 Maximum retry count value BGP-4 Maximum hold time value The BGP-4 MIB version 2 will have scalar flag indicating that the a backoff implementation is present in a BGP peer implementation The BGP-4 MIB version 2 will have the following table for BGP-4 Restart Peer Back-off: BGP-4 Peer Restart Back-off table retry cnt hold time value (seconds) -------- --------------- 0 value-1 1 Value-1 2 Value-2 3 Value-3 4 Value-4 5 Value-5 6 Value-6 à max-cnt max-cnt-value Hares Expires - August 2004 [Page 5] BGP Peer Restart Backoff Mechanisms February 2004 For our example of the step-wise back-off, the table would be filled in as Retry cnt hold time value --------- --------------- 1 8 seconds 2 8 seconds 3 8 seconds 4 8 seconds 5 8 seconds 6 8 seconds 7 8 seconds 8 8 seconds 9 64 seconds 10 64 seconds à 24 64 seconds 25 128 seconds 26 128 seconds .. 35 128 seconds BGP-max retry cnt =35 BGP-max hold time = 128 seconds 7. Back-off Mechanisms in FSM 7.1 Per BGP Peer Variables that are Optional Session attributes 1) IdleHoldTimer Definition: Timer that keeps track of exponential back- Off. This timer must expire before the BGP peer generates an automatic start event. Units: Seconds MIB Status: Needs to be added to BGP-4 MIB version 2[3] in running timers. [new category] 2) DampPeerOscillations Definition: Optional Session attribute [BGP4] that enables the damping of peer oscillations (up/down) mechanisms on automatic restarts. This mechanism prevents persistent bgp peer \ oscillation. Units: True/False MIB Status: Needs to be added to BGP-4 MIB Version 2 Hares Expires - August 2004 [Page 6] BGP Peer Restart Backoff Mechanisms February 2004 in configured options. 3) ConnectRetryCounter Definition: Count of times the bgp-4 peer automatically Attempts to restart the connection. Units: non-zero, positive integer Status: Needs to be added to BGP-4 MIB in running Running parameters. 7.2 Additional Optional Session attributes for peer oscillation damping back-off 1) LastIdleHoldTimer Definition: Time last set in the Idle Hold timer Units: seconds Status: Needs to be added to BGP-4 MIB version 2 [3] In running timers [new category] 2) IdleHoldTimeMaximum Definition: Maximum value for Idle Hold timer Units: Seconds Status: Needs to be added to BGP-4 MIB version 2 [3] 3) KeepIdle Definition: Optional session attribute set if the IdleHoldTimer exceeds the exceeds the IdleHoldTimerMaximum value. Units: true/false Status: Needs to be added to BGP-4 MIB version 2 in Running status. 4) MaximumRetryCount Definition: maximum number of times a bgp-4 peer will automatically try to restart the bgp-4 connection. Units: non-zero, positive integer Status: Needs to be added to BGP-4 MIB in configured Hares Expires - August 2004 [Page 7] BGP Peer Restart Backoff Mechanisms February 2004 Parameters 8) BGP-Backoff-RetryTimeTable Definition: Table that specifies the exact ConnectRetryInterval per each time the bgp-4 peer attempts to automatically retart. Units: Table-index = cnt Table-value = timer in seconds 9) IdleSubstate Definition: Idle Substate value (defined in section 7.3) Units: 0 = Idle hold Null 1 = Idle Hold wait 2 = Idle Hold down 3 = Idle Hold ticking 7.3 Existing MIB objects used The back-off functions are implemented as a processing within the Idle state in the BGP-4 FSM. The FSM processing for the BGP-4 also uses the following variables (found in BGP-4 MIB version 2). 1) bgpPeerState Definition: From the BGP-4 MIB version 2 [2] "The BGP Peer's FSM state". status: Exists in BGP-4 MIB version 2. 2) bgpPeerConnectRetryInterval Definition: Time interval in seconds for the ConnectRetry Timer. The suggested value for this timer is 120 Seconds if no back-off mechanism is engaged. If a Back-off mechanism is engaged, the bgpPeerBackoff Table entry is used for the value of ConnectRetryCnt. status: Exists in BGP-4 MIB version 2. Hares Expires - August 2004 [Page 8] BGP Peer Restart Backoff Mechanisms February 2004 7.4 IdleHold substate in Idle FSM description The Idle Hold has the following four substates: IdleHoldWait IdleHoldDown IdleHoldTicking IdleHoldNull The definitions of these sub-states are: sub-state1: IdleHold-Wait Definition: Idle Hold Down timer has expired. Local system is waiting for auto-start request on bgp peer session. Sub-state Status flags: Idle Hold Down timer is zero. Keep Idle flag is clear. Substate2: IdleHold-Down: Definition: Keep Idle flag is set, and Automatic start is disabled for peer session. Sub-state status flags: Keep Idle flag is set. Substate3: IdleHold-TimerTicking: Definition: Idle Hold Timer is ticking (running) and local system will not automatically restart the BGP session. Variable: Idle Hold Timer is non-zero. Substate4: IdleHold-Null Definition: No Idle Hold functions are being performed. Hares Expires - August 2004 [Page 9] BGP Peer Restart Backoff Mechanisms February 2004 Sub-state status flags: KeepIdleflag is clear, IdleHoldTimer is zero IdleHold is off How these sub-states are represented internally is an implementation matter. The important part is that these sub- states be available to the BGP MIB. 7.5 Action upon entering the Idle state Processing upon entering the Idle state: If bgp_stop_flap flag on, the Idle Hold sub state is entered: 1)Idle Hold time = connect-retry-table[connect-retry-count] 2)Idle Hold Timer = Idle Hold time If (Idle hold time > Idle Hold time maximum), then: 1) bgp peer sets the keep-idle-flag 2) set substate to Idle Hold down 3) clear Idle Hold timer else 1) increment ConnectRetryCnt, 2) start Idle Hold timer with value set above 3) set Idle-hold substate to "Idle Timer ticking" If bgp_stop_flag is off, this sub-state action should not occur. 7.6 Substate processing The events are defined from the BGP-4 draft in section 8 on BGP-4 FSM [2]. This processing occurs within the IDLE state if the Bgp peer oscillation back-off mechanism are used. The form of the table is state/sub-state/action. Hares Expires - August 2004 [Page 10] BGP Peer Restart Backoff Mechanisms February 2004 Idle State Idle State Idle State Idle state Substate 1 Substate2 Substate3 Substate4 # Event Idle Hold Idle Hold Idle Hold null Wait Down timer ticking ------------------------------------------------------------ 1 Manual Connect/ Connect / Connect/ Connect start null/ null/ null/ null/ A1 A2 A3 A1 ------------------------------------------------------------ 2 Manual Idle/ Idle/ Idle/ Idle/ stop null/ null/ null/ null/ Ignore A4 A5 Ignore ------------------------------------------------------------ 3 Auto Connect Connect/ Connect Connect start null/ null/ null/ null/ F1 F2 F3 F1 ------------------------------------------------------------- 4 Manual Active / Active / Active/ Active/ start null/ null/ null/ null/ with B1 B2 B3 B1 Passive TCPEstb. ------------------------------------------------------------- 5 Auto Active / Active / Active / Active start null/ null/ null/ null/ with B1 B2 B3 B1 Passive TCPEstab. ------------------------------------------------------------ 6 Automatic Connect/ Idle/ Idle/ Connect/ start_with null/ IdleHold IdleHold null/ DampPeer__ Down/ Timerticking/ Oscill. B1 ignore ignore B1 -------------------------------------------------------------- 7 Automatic Connect/ Idle/ Idle/ Connect/ start_with null/ IdleHold IdleHold null/ DampPeer__ down/ ticking/ Oscill. B1 ignore ignore B1 And_Passive TCP -------------------------------------------------------------- 7 Automatic Idle/ Idle/ Idle/ Idle/ Stop IdleHold IdleHold IdleHold Wait/ Down/ TimerTicking/ Ignore Ignore Ignore Hares Expires - August 2004 [Page 11] BGP Peer Restart Backoff Mechanisms February 2004 --------------------------------------------------------------- 8 Idle Hold Idle/ Idle/ Idle/ Idle/ timer IdleHold IdleHold IdleHold null/ expires wait/ Down/ Wait/ null/ V Ignore A6 V ---------------------------------------------------------------- Action A1: 1. Initialize all BGP resources 2. Set ConnectRetryCnt to zero 3. Start Connect retry timer with initial value 4. Initiate transport connection Request to the remote BGP peer 5. Listen for connection indication from remote BGP peer Action A2: 1. Stop and Clear IdleHoldTimer 2. Clear KeepIdle attribute 3. Initialize all BGP resources 4. ConnectRetryCounter set to zero 5. Start ConnectRetryDounter with initial value 6. Initiate transport connection request to remote BGP peer 7. Listen for connection indication from remote BGP peer Note: items 3-7 are the same actions as Action A of the BGP FSM. Action A3: 1. Stop and Clear IdleHoldTimer 2. Initialize all BGP resources 3. ConnectRetryCnt set to zero 4. Start ConnectRetryCounter with initial value 5. Initiate transport connection to BGP peer by send a TCP syn 6. Listen for connection (TCP syn, ack) from remote Peer Note: items 2-6 are the same actions as action A of the BGP FSM. Action A4: 1. clear KeepIdle attribute Action A5: 1. Clear Idle Hold Timer Hares Expires - August 2004 [Page 12] BGP Peer Restart Backoff Mechanisms February 2004 Action A6: 1. Idle Hold substate = Idle Hold Wait 2. Clear Idle Hold Timer Action B1: 1. Initialize all BGP resources 2. ConnectRetryCnt set to 0 3. Start Connect Retry timer with initial value 4. Listen for connection set-up by the remote BGP peer Note: B1 are the same actions as action B Action B2: 1. Clear Keep Idle Flag 2. Clear Idle Hold Timer 3. Initialize all BGP resources 4. ConnectRetryCounter set to 0 5. Start ConnectRetryTimer with initial value 6. Listen for connection set-up from the remote BGP peer [TCP Syn] Note: items 3-6 are the same actions as action B. Action B3: 1. Clear IdleHoldTimer 2. Initialize all BGP resources 3. ConnectRetryCounter set to 0 4. Start ConnectRetryTimer with initial value 5. Listen for transport connection indication from the remote BGP peer Note: items 2-5 are the same actions as action B. Action F1: 1. Restart ConnectRetryTimer (with initial value) 2. Initiates a transport connection request to the other bgp peer [if TCP send a TCP SYN] 3. Listen for remote transport connection indication may be initiated from the remote BGP peer (TCP Syn) Note: items 1-3 are the same actions as action F Hares Expires - August 2004 [Page 13] BGP Peer Restart Backoff Mechanisms February 2004 Action F2: 1. Clear IdleHoldTimer 2. Clear KeepIdle optional session attribute 3. Restart ConnectRetryTimer (with initial value) 4. Initiates a transport connection request to the remote bgp peer 5. Listen for a remote transport connection indication from the remote BGP peer Note: items 3-6 are the same as action F. Action F3 1. Clear IdleHoldTimer 2. Restart ConnectRetryTimer (with initial value) 3. Initiates a transport connection request to the remote bgp peer 4. Listen for remote transport connection that may be initiated by the remote BGP peer Note: items 2-4 are the same as action F. Action v: 1. Set FSM error in MIB reason code. Security Considerations Security concerns for BGP-4 are addressed in the BGP-4 specification, and accompanying specifications on TCP MD5 [4] and IP Security[5]. No additional considerations need to be made for the BGP-4 state machine description. References 1 "A Border Gateway Protocol 4 (BGP-4)" Y. Rekhter, T. Li Editors http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4- 23.txt 2 RFC 2119 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 3 "Definitions of Managed Objects for the Fourth Version of Border Gateway Protocol (BGP-4),Second Version", http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4-mibv2- 01.txt 4 "Protection of BGP Sessions via the TCP MD5 Signature Option" A. Heffernan, rfc2385.txt 5 Securing BGPv4 using IPSec , D. Ward, draft-ward-bgp-ipsec-00.txt Hares Expires - August 2004 [Page 14] BGP Peer Restart Backoff Mechanisms February 2004 Acknowledgments Author's Addresses Susan Hares NextHop Technologies, Incorporated 825 Victors Way, Suite 100 Ann Arbor, MI 48108-2738 Phone: 734.222.1600 Email: skh@nexthop.com Hares Expires - August 2004 [Page 15]