Internet Draft S. Hares Document: draft-hares-bgp-backoff-00.txt NextHop Technologies, Inc. Expires: December 2002 June 2002 BGP Peer Restart Backoff Mechanisms Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes mechanisms for methods for damping the oscillations of BGP-4 [1] peers by increasing the time between automatic start functions of the BGP peers. The increase of time between automatic start functions will be denoted as a backoff of the automatic start functions by this draft. This draft is an informational RFC which provides a description of mechanisms for back-off of the automatic start function in bgp-4 peer reconnections. This Informational RFC includes descriptions about the back-off mechanisms and how this information is recorded in the BGP-4 MIB version 2 [2]. The expontential back-off mechanism contained in this draft was first recorded in a draft of the bgp-4 specification. The author requests that anyone with information regarding the original authors of that work contact the author. The lack of credit for those authors is based on the author's lack of knowledge the originators of the expotential back-off. Hares Informational Expires December 2002 1 Internet Draft ietf-skh-bgp-backoff-00.txt June 20, 2002 Table of Contents Status of this Memo...........................................1 Abstract .................................................1 1.0 Overview ..............................................2 2.0 Conventions used in this document.........................3 3.0 Types of Back-off.........................................4 4.0 Exponential back-off......................................4 5.0 Step-wise Back-off .......................................5 6.0 Recording the Back-off information in the MIB ............5 7.0 Back-off Mechanisms in FSM ...............................6 7.1 Variables to support Back-off functions................6 7.2 Existing MIB objects used .............................8 7.3 Events only utilized by the back-off mechanisms .......8 7.4 IdleHold substate in Idle FSM description .............8 7.5 Action upon entering the Idle state ...................9 7.6 Substate processing...................................10 8.0 Security Considerations..................................13 9.0 References...............................................13 10. Author's Addresses.......................................13 1.0 Overview If a BGP speaker detects an error, it shuts down the connection and changes its connection state to Idle. Getting out of the Idle state requires generation of the Start event. If such an event is generated automatically, then persistent BGP errors may result in persistent flapping of the speaker. To avoid such a condition it is recommended that automatic start events should not be generated immediately for a peer that was previously transitioned to Idle due to an error. For a peer that was previously transitioned to Idle due to an error, the time between consecutive generation of start events, if such events are generated automatically, shall increase. This increase of time between automatic start events will be called in backoff of automatic BGP peer start events. The automatic start events listed in the BGP-4 specification are: - Event3: Automatic start - Event5: Automatic start with passive flag on - Event6: Automatic start with bgp_flap_stop flag on Event 3 is issued when an automatic stop occurs when no additional flags impact the starting of the bgp peer session. Event 5 is generated when the passive flag is set to allow the local peer to listen prior to starting a TCP session. Hares Informational - Expires December 2002 2 Internet Draft ietf-skh-bgp-backoff-00.txt June 20, 2002 Event 6 is issued when an automatic start occurs with the bgp peer oscillation damping features turned on. The "bgp_flap_stop" flag engages the damping of bgp peer oscillation. Event 7: Automatic stop is the only automatic stop event for the BGP finite state machine. Automatic stop followed by automatic start can also accelerate the BGP peer oscillation. BGP-4 implementations today have a variety of back-off mechanisms. This document is intended to provide information on these back-off mechanisms and how to record this information in a MIB. Two mechanisms are recorded in this Internet Draft: 1) Exponential back-off (section 2) and 2) Step-wise back-off (section 3). The author welcomes input on additional back-off mechanisms. Please send input to the author or the idr list (idr@merit.edu). A BGP peer oscillation backoff mechanism requires additional processing in the bgp-4[1] Finite State Machine (FSM). The BGP-4 draft contains an indication of where the FSM would process the peer oscillation damping. The implementation of backoff mechanisms in a BGP-4 is optional, but wide spread in implementations. The exact mechanisms vary between implementations. Section 6.0 contains information on how the BGP-4 MIB encodes the time between establishing BGP-4 peer sessions. Section 7.0 contains the additional mechanisms needed in a BGP-4 FSM for back-off implementations. 2.0 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. As an informational draft there are no mandatory functions. This draft hopes to recommend healthy life styles for back-off mechanisms in BGP peers. As in all recommendations for healthy life styles the user of BGP-4 is not required to do anything, except expire the BGP-4 connection according to the letter of the BGP-4 standard. Hares Informational - Expires December 2002 3 Internet Draft ietf-skh-bgp-backoff-00.txt June 20,2002 3.0 Types of Back-off The author knows of three types of back-off at this time: - Exponential back-off, and - Step-wise back-off, - no back-off. The author welcomes any comments on additional back-offs and will hasten to include this information in the next revision of this draft. 4.0 Exponential back-off The original description occurred in RFC 1654 and was included in a version of bgp-4 [1] draft. If a BGP speaker detects an error, it shuts down the connection and changes its state to Idle. Getting out of the Idle state requires generation of the Start event. If such an event is generated automatically, then persistent BGP errors may result in persistent flapping of the speaker. To avoid such a condition it is recommended that automatic Start events should not be generated immediately for a peer that was previously transitioned to Idle due to an error. For a peer that was previously transitioned to Idle due to an error, the time between consecutive generation of Start events, if such events are generated automatically, shall exponentially increase. The IdleHoldTimer will be the timer is utilized for the count down of this feature. The value of the initial timer shall be 60 seconds. The time shall be doubled for each consecutive retry. The equation for the setting of the Idle time is as follows: Idle Hold timer = 2**(ConnectRetryCnt)*60 If the Idle Hold timer exceeds a local maximum value, the connection shall be held down until a manual start event occurs. The Idle Hold timer is initialized to zero. Hares Informational - Expires December 2002 4 Internet Draft ietf-skh-bgp-backoff-00.txt June 20,2002 5.0 Step-wise Back-off The Step-wise back-off uses the following back-off function. The step-wise back-off functions provide a step function based on the number of retries. An example of the Step-wise code, is as follows: - For 1-8 retries, the amount of time between retries is 8 seconds. - For 9-24 retries, the amount of time between retries, the amount time between retries is 64 seconds, - For larger than 25 retries, the amount of time between retries is 128 seconds, - Maximum number of retries is 35. 6.0 Recording the Back-off information in the BGP-4 MIB [3] The BGP-4 MIB version 2[3] will have two variables for the hold time Maximums: BGP-4 Maximum retry count value BGP-4 Maximum hold time value The BGP-4 MIB version 2 will have scalar flag indicating that the a backoff implementation is present in a BGP peer implementaiton The BGP-4 MIB version 2 will have the following table for BGP-4 Restart Peer Back-off: BGP-4 Peer Restart Back-off table retry cnt hold time value (seconds) -------- --------------- 0 value-1 1 Value-1 2 Value-2 3 Value-3 4 Value-4 5 Value-5 6 Value-6 à max-cnt max-cnt-value For our example of the step-wise back-off, the table would be filled in as Retry cnt hold time value --------- --------------- 1 8 seconds 2 8 seconds 3 8 seconds 4 8 seconds 5 8 seconds Hares Informational - Expires December 2002 5 Internet Draft ietf-skh-bgp-backoff-00.txt June 20,2002 6 8 seconds 7 8 seconds 8 8 seconds 9 64 seconds 10 64 seconds à 24 64 seconds 25 128 seconds 26 128 seconds .. 35 128 seconds BGP-max retry cnt =35 BGP-max hold time = 128 seconds 7.0) Back-off Mechanisms in FSM 7.1) Per BGP Peer Variables to support Back-off functions 1) Idle Hold timer Definition: Timer that keeps track of exponential back- Off. This timer must expire before the bgp peer generates an automatic start event. Units: Seconds Status: Needs to be added to BGP-4 MIB version 2[3] In running timers. [new category] 2) bgp-4 Last Hold Timer value Definition: Time last set in the Idle Hold timer Units: seconds Status: Needs to be added to BGP-4 MIB version 2 [3] In running timers [new category] 3) Idle Hold Time Maximum Definition: Maximum value for Idle Hold timer Units: Seconds Status: Needs to be added to BGP-4 MIB version 2 [3] In configured timers. Hares Informational - Expires December 2002 6 Internet Draft ietf-skh-bgp-backoff-00.txt June 20,2002 4) Keep Idle Flag Definition: Flag set if the idle Hold timer value exceeds the Idle hold maximum value. Units: true/false Status: Needs to be added to BGP-4 MIB version 2 in Running status. 5) bgp_stop_flap flag Definition: Flag to enable the back off mechanisms on automatic restarts. These mechanism prevent persistent bgp peer oscillation. Units: True/False Status: Needs to be added to BGP-4 MIB Version 2 In configured options. 6) BGP-4 max retry count Definition: maximum number of times a bgp-4 peer will Automatically try to restart the bgp-4 connection. Units: non-zero, positive integer Status: Needs to be added to BGP-4 MIB in configured Parameters 7) BGP retry count Definition: Count of times the bgp-4 peer automatically Attempts to restart the connection. Units: non-zero, positive integer Status: Needs to be added to BGP-4 MIB in running Running parameters. 8) BGP-Backoff-RetryTimeTable Definition: Table that specifies the exact ConnectRetryInterval Per each time the bgp-4 peer attempts to automatically retart. Units: Table-index = cnt Table-value = timer in seconds 9) Idle Substate Definition: Idle Substate value (defined in section 7.3) Units: 0 = Idle hold Null 1 = Idle Hold wait 2 = Idle Hold down 3 = Idle Hold ticking Hares Informational - Expires August 2002 7 Internet Draft ietf-skh-bgp-backoff-00.txt February 28,2002 7.2 Existing MIB objects used The back-off functions are implemented as a processing within the Idle state in the BGP-4 FSM. The FSM processing for the BGP-4 also uses the following variables (found in BGP-4 MIB version 2). 1) bgpPeerState Definition: From the BGP-4 MIB version 2 [2] "The BGP Peer's FSM state". status: Exists in BGP-4 MIB version 2. 2) bgpPeerConnectRetryInterval Definition: Time interval in seconds for the ConnectRetry Timer. The suggested value for this timer is 120 Seconds if no back-off mechanism is engaged. If a Back-off mechanism is engaged, the bgpPeerBackoff Table entry is used for the value of ConnectRetryCnt. status: Exists in BGP-4 MIB version 2. 7.3 Events only utilized by the back-off mechanisms The following optional events are defined in section 8.1 of the BGP-4 Draft [1]. Event3: automatic start with bgp_stop_flap option Event6: Idle Hold timer expires 7.4 IdleHold substate in Idle FSM description The Idle Hold has the following four substates: IdleHoldWait IdleHoldDown IdleHoldTicking IdleHoldNull The definitions of these sub-states are: Hares Informational - Expires December 2002 8 Internet Draft ietf-skh-bgp-backoff-00.txt June 20, 2002 Substate1: Idle Hold Wait Definition: Idle Hold Down timer has expired. Local system is waiting for auto-start request on bgp peer session. Sub-state Status flags: Idle Hold Down timer is zero. Keep Idle flag is clear. Substate2: Idle Hold down: Definition: Keep Idle flag is set, and Automatic start is disabled for peer session. Sub-state status flags: Keep Idle flag is set. Substate3: Idle Hold timer ticking: Definition: Idle Hold Timer is ticking (running) and local system will not automatically restart the BGP session. Variable: Idle Hold Timer is non-zero. Substate4: Idle Hold Null Definition: No Idle Hold functions are being performed. Sub-state status flags: "Keep Idle flag" is clear, Idle Hold timer is zero Idle Hold off How these sub-states are represented internally is an implementation matter. The important part is that these sub-states be available to the BGP MIB. 7.5 Action upon entering the Idle state Processing upon entering the Idle state: If bgp_stop_flap flag on, the Idle Hold sub state is entered: 1)Idle Hold time = connect-retry-table[connect-retry-count] 2)Idle Hold Timer = Idle Hold time If (Idle hold time > Idle Hold time maximum), then: 1) bgp peer sets the keep-idle-flag 2) set substate to Idle Hold down 3) clear Idle Hold timer else 1) increment ConnectRetryCnt, 2) start Idle Hold timer with value set above 3) set Idle-hold substate to "Idle Timer ticking" If bgp_stop_flag is off, this sub-state action should not occur. Hares Informational - Expires November 2002 9 Internet Draft ietf-skh-bgp-backoff-00.txt June 20,2002 7.6 Substate processing The events are defined from the BGP-4 draft in section 8 on BGP-4 FSM [2]. This processing occurs within the IDLE state if the Bgp peer oscillation back-off mechanism are used. The form of the table is state/sub-state/action. Idle State Idle State Idle State Idle state Substate 1 Substate2 Substate3 Substate4 # Event Idle Hold Idle Hold Idle Hold null Wait Down timer ticking ------------------------------------------------------------ 1 Manual Connect/ Connect / Connect/ Connect start null/ null/ null/ null/ A1 A2 A3 A1 ------------------------------------------------------------ 2 Manual Idle/ Idle/ Idle/ Idle/ stop null/ null/ null/ null/ Ignore A4 A5 Ignore ------------------------------------------------------------ 3 Auto Connect Connect/ Connect Connect start null/ null/ null/ null/ F1 F2 F3 F1 ------------------------------------------------------------- 4 Manual Active / Active / Active/ Active/ start null/ null/ null/ null/ with B1 B2 B3 B1 passive ------------------------------------------------------------- 5 Auto Active / Active / Active / Active start null/ null/ null/ null/ with B1 B2 B3 B1 passive ------------------------------------------------------------ 6 Automatic Connect/ Idle/ Idle/ Connect/ start & null/ Idle hold Idle hold null/ bgp_stop_ down/ ticking/ _flap on B1 ignore ignore B1 -------------------------------------------------------------- 7 Automatic Idle/ Idle/ Idle/ Idle/ Stop Idle Hold Idle Hold Idle Hold Wait/ down/ tick/ Ignore Ignore Ignore --------------------------------------------------------------- 8 Idle Hold Idle/ Idle/ Idle/ Idle/ timer Idle Hold Idle Hold Idle Hold null/ expires wait/ down/ wait/ null/ V Ignore A6 V ---------------------------------------------------------------- Hares Informational - Expires August 2002 10 Internet Draft ietf-skh-bgp-backoff-00.txt February 20,2002 Action A1: 1. Initialize all BGP resources 2. Set ConnectRetryCnt to zero 3. Start Connect retry timer with initial value 4. Initiate transport connection to the BGP peer by sending a TCP SYN 5. Listen for connection set-up by remote BGP peer Action A2: 1. Stop and Clear Idle Hold timer 2. Clear Keep Idle flag 3. initialize all BGP resources 4. ConnectRetryCnt set to zero 5. Start Connect retry counter with initial value 6. Initiate transport connection to BGP peer by send a TCP syn 7. Listen for connection set-up by remote BGP peer Note: items 3-7 are the same actions as Action A of the BGP FSM. Action A3: 1. Stop and Clear Idle Hold Timer 2. initialize all BGP resources 3. ConnectRetryCnt set to zero 4. Start Connect retry counter with initial value 5. Initiate transport connection to BGP peer by send a TCP syn 6. Listen for connection (TCP syn, ack) from remote Peer Note: items 2-6 are the same actions as action A of the BGP FSM. Action A4: 1. clear Keep down flag Action A5: 1. Clear Idle Hold Timer Action A6: 1. Idle Hold substate = Idle Hold Wait 2. Clear Idle Hold Timer Hares Informational - Expires December 2002 11 Internet Draft ietf-skh-bgp-backoff-00.txt June 20, 2002 Action B1: 1. Initialize all BGP resources 2. ConnectRetryCnt set to 0 3. Start Connect Retry timer with initial value 4. Listen for connection set-up by the remote BGP peer Note: B1 is the same actions as action B Action B2: 1. Clear Keep Idle Flag 2. Clear Idle Hold Timer 3. Initialize all BGP resources 4. ConnectRetryCnt set to 0 5. Start Connect Retry timer with initial value 6. Listen for connection set-up by the remote BGP peer [TCP Syn] Note: items 3-6 are the same actions as action B. Action B3: 1. Clear Idle Hold Timer 2. Initialize all BGP resources 3. ConnectRetryCnt set to 0 4. Start Connect Retry timer with initial value 5. Listen for connection set-up by the remote BGP peer Note: items 2-5 are the same acttions as action B. Action F1: 1. Restart ConnectRetry timer (with initial value) 2. Initiates a transport connection to the other bgp peer [Send a TCP SYN] 3. Listen for remote transport connection that may be initiated by the remote BGP peer (TCP Syn) Note: items 1-3 are the same actions as action F Action F2: 1. Clear Idle Hold Timer 2. Clear Keep Idle Flag 3. Restart ConnectRetry timer (with initial value) 4. Initiates a transport connection to the other bgp peer [Send a TCP SYN] 5. Listen for a remote transport connection that may be initiated by the remote BGP peer [TCP SYN] Note: items 3-6 are the same as action F. Action F3 1. Clear Idle Hold Timer 2. Restart ConnectRetry timer (with initial value) 3. Initiates a transport connection to the other bgp peer [Send a TCP SYN] 4. Listen for remote transport connection that may be initiated by the remote BGP peer (TCP syn) Note: items 2-4 are the same as action F. Action v: 1. Set FSM error in MIB reason code. Hares Informational - Expires August 2002 12 Internet Draft ietf-skh-bgp-backoff-00.txt June 20, 2002 8.0 Security Considerations Security concerns for BGP-4 are addressed in the BGP-4 specification, and accompanying specifications on TCP MD5 [3] and IP Security[4]. No additional considerations need to be made for the BGP-4 state machine description. 9.0 References [1] RFC 2119 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [2] "A Border Gateway Protocol 4 (BGP-4)" Y. Rekhter, T. Li Editors http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4-17.txt [3] "Definitions of Managed Objects for the Fourth Version of Border Gateway Protocol (BGP-4),Second Version", http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4-mibv2- 01.txt [5] "Protection of BGP Sessions via the TCP MD5 Signature Option" A. Heffernan, rfc2385.txt [6] Securing BGPv4 using IPSec , D. Ward, draft-ward-bgp-ipsec-00.txt 10. Author's Addresses Susan Hares NextHop Technologies, Inc 825 Victors Way Phone: 1-734-222-1610 Ann Arbor, MI USA Email: skh@nexthop.com Hares Informational - Expires December 2002 13