Internet Draft T. Anderson Expiration: August 2002 Intel Labs File: draft-anderson-forces-arch-00.txt L. Yang Working Group: ForCES Intel Labs R. Dantu Netrake Corp. February 2002 ForCES Architectural Framework draft-anderson-forces-arch-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC-2119]. 1. Abstract This document defines the architectural framework for ForCES network elements (NE), associated entities and the interaction among them. The architecture defined herein is intended to satisfy the requirements specified in the ForCES requirements draft [FORCES- REQ]. 2. Definitions Internet Draft ForCES Framework February 2002 A set of terminology associated with the ForCES requirements is defined in [FORCES-REQ] and is not copied here. 3. Introduction An IP network element is composed of numerous logically separated entities that cooperate to provide a given functionality (such as routing or IP switching) and yet appear as a normal integrated network element to external entities. Two primary types of network element components exist: control element (CE) in control plane and forwarding element (FE) in forwarding plane (or data plane). Forwarding elements are ASIC, network-processor, or general-purpose processor-based devices that handle all data path operations. Control elements are typically based on general-purpose processors that provide control functionality including processing of routing or signaling protocols. ForCES aims to define a framework and associated mechanisms to standardize the exchange of information between the control plane and the forwarding plane. By defining a set of standard mechanisms for control and forwarding separation, ForCES will enable rapid innovation in both the control and forwarding planes while maintaining interoperability. A set of requirements for control and forwarding separation is identified in [FORCES-REQ]. This document satisfies the architectural requirements of that document and defines a framework for ForCES network elements and associated entities to facilitate protocol definition. 4. Architecture This section defines the ForCES architectural framework and associated logical components. This ForCES framework consists primarily of ForCES NEs but also includes several ancillary components. ForCES NEs appear to external entities as monolithic pieces of network equipment, e.g., routers, NATs, firewalls, or load balancers. Internally, however, ForCES NEs are composed of several logical components. These logical components can be connected in different kinds of topologies for flexible packet processing. By defining these logical components and specifying the interactions between them, the ForCES architecture allows these components to be physically separated. This physical separation accrues several benefits to the ForCES architecture. For example, separate components would allow vendors to specialize in one component without having to become experts in all components. Scalability is also provided by this architecture in that additional forwarding or control capacity can be added to existing network elements without the need for forklift upgrades. The logical components of the ForCES architecture and their relationships are shown in the following diagram. There are two kinds of components inside a ForCES network element (NE): control Anderson, et. al. Expires August 2002 [Page 2] Internet Draft ForCES Framework February 2002 element (CE) and forwarding element (FE). The framework allows multiple instances of CE and FE inside one NE (see sections 4.1 and 4.2 for more information). The diagram also shows two entities outside of the ForCES NE: CE Manager and FE Manager. These two entities provide configuration to the corresponding CE or FE in the pre-association phase (see Section 5.1). There is no defined role for FE Manager and CE Manager in post-association phase, thus these logical components are not considered part of the ForCES NE. For convenience, the interactions between these components are labeled by reference points Gp, Gc, Gf, Gr, Gl, and Gi. More detail is provided in Section 5 for each of these reference points. --------------------------------------- | ForCES Network Element | -------------- Gc | -------------- -------------- | | CE Manager |---------+-| CE 1 |------| CE 2 | | -------------- | | | Gr | | | | | -------------- -------------- | | Gl | | | Gp / | | | Gp| |----------| / | | | | |/ | | | | | | | | | Gp /|----| | | | | /--------/ | | -------------- Gf | -------------- -------------- | | FE Manager |---------+-| FE 1 | Gi | FE 2 | | -------------- | | |------| | | | -------------- -------------- | --------------------------------------- The interface between two ForCES NEs will be similar to interface between two conventional routers and these two NEs exchange the protocol packets through the in band interfaces. In fact, ForCES NEs can be connected to existing routers transparently. 4.1. Control Elements This architecture permits multiple CEs to be present in a network element. These CEs may be used for redundancy, load sharing, distributed control, or other purposes. Redundancy is the case where one or more CEs are prepared to take over should an active CE fail. Load sharing is the case where two or more CEs are concurrently active and where any request that can be serviced by one of the CEs can also be serviced by any of the other CEs. In both redundancy and load sharing, the CEs involved are equivalently capable. The only difference between these two cases is in terms of how many active CEs there are. Distributed control is the case where two or more CEs are concurrently active but where certain requests can only be serviced by certain CEs. When multiple CEs are employed in a ForCES NE, their internal organization is considered an implementation issue that is beyond Anderson, et. al. Expires August 2002 [Page 3] Internet Draft ForCES Framework February 2002 the scope of the ForCES effort. CEs are wholly responsible for coordinating amongst themselves to provide consistency, redundancy, load sharing, or distributed control, if desired. The Gr reference point defined here is used for this task and is included here for reference purposes only. In cases where an implementation uses multiple CEs, it is expected the invariant that the CEs and FEs together appear as a single NE will be maintained. This framework does not define the implementation or protocols used between CEs, nor does it define how to distribute functionality among CEs. The information provided above is included because it is recognized that these concepts are important in router design. 4.2. Forwarding Elements FEs are responsible for per-packet processing and handling as directed by CEs. FEs have no initiative of their own. Instead, FEs are slaves and only do as they are told. FEs may communicate with one or more CEs concurrently across reference point Gp (Section 6.6). However, FEs have no notion of CE redundancy, load sharing, or distributed control. Instead, FEs accept commands from any CE authorized to control them. For example, FEs can be configured to detect RSVP, BGP protocol related packets, and forward RSVP packets to one CE and BGP packets to another CE. Hence, FEs may need to do packet filtering for forwarding the protocol-specific packets to CEs. This architecture permits multiple FEs to be present in a NE. [FORCES-REQ] dictates that the ForCES protocol MUST be able to scale to at least hundreds of FEs (see [FORCES-REQ] Section 5, requirement #11). Each of these FEs may potentially have a different set of capabilities. FEs express their capabilities to CEs using the ForCES FE model (see [FORCES-REQ] Section 6). FEs are responsible for establishing and maintaining layer-2 connectivity with other FEs and with entities external to the NE. Thus, FEs are also responsible for any signaling required at layer-2. In a typical configuration, one CE may be connected to multiple FEs. In order to distribute the same capability to multiple FEs, CE can multicast the control information (e.g., FIB, Forwarding Information Base) to multiple FEs. A packet processing operation may need multiple FE capabilities. These operations may be pipelined or some times parallel. In order to accomplish full capabilities, one FE may feedback to another FE after partial processing. FEs are connected in different kinds of topologies and packet processing may spread across several FEs in the topology. Hence, logical packet flow may be different from physical FE topology. Anderson, et. al. Expires August 2002 [Page 4] Internet Draft ForCES Framework February 2002 4.3. CE Managers CE managers are responsible for determining which FEs a CE should control. It is legitimate for CE managers to be hard-coded with the knowledge of with which FEs its CEs should communicate. A CE manager may also be physically embedded into a CE and be implemented as a simple keypad or other direct configuration mechanism on the CE. Finally, CE managers may be physically and logically separate entities that configure the CE with FE information via such mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157]. 4.4. FE Managers FE managers are responsible for determining to which CE any particular FE should initially communicate. Like CE managers, no restrictions are placed on how a FE manager decides to which CEs its FEs should communicate, nor are restrictions placed on how FE managers are implemented. 5. Operational Phases Both FEs and CEs require some configuration in place before they can start information exchange and function as a coherent network element. Two operational phases are identified in this framework -- pre-association and post-association. 5.1.Pre-association Phase Pre-association phase is the period of time during which a FE Manager and a CE Manager are determining which FE and CE should be part of the same network element. A pre-association phase protocol might be used across Gl reference point (see Section 6.1) to facilitate the discovery of CEs and FEs. However, this pre- association phase protocol is entirely separate from the ForCES protocol and hence out of scope for ForCES. 5.2.Post-association Phase Post-association phase is the period of time during which a FE and CE have been configured with information necessary to contact each other and includes both communication establishment and steady-state communication. Once the association is established between the CE and the FE, the ForCES protocol is used by the CE and the FE to exchange information to facilitate packet processing (see Section 6.6). FEs and CEs may join and leave NEs dynamically (see [FORCES-REQ] Section 5, requirements #12 and #13). When a FE or CE leaves the NE, the association with the NE is broken. If the leaving party rejoins a NE later, to re-establish the association, it may or may not need to re-enter the pre-association phase. Loss of association can also happen unexpectedly due to loss of connection between the CE and the FE. Therefore, the framework allows the bi-directional transition Anderson, et. al. Expires August 2002 [Page 5] Internet Draft ForCES Framework February 2002 between these two phases, but the ForCES protocol is only applicable for the post-association phase. However, the protocol should provide mechanisms to support association re-establishment (see [FORCES-REQ] Section 5, requirement #7). 5.3.Association Re-establishment After the layer-2 connectivity is established, the following steps are executed at the ForCES entity to re-establish its association with a certain NE: - to identify the other ForCES communication entity (FE or CE) via either previous configuration still in its cache, or re- configuration, or re-entering the pre-association phase, etc.; - to exchange security information for authorization and authentication to join the NE; - (for FEs) to validate or re-synchronize its state information with CEs. Packet processing at FEs can begin after the above steps. Signaling required to accomplish all the above steps except the first one are part of the ForCES protocol. 6. Interaction between the Components 6.1. Gl Reference Point CE managers and FE managers may communicate across the Gl reference point in the pre-association phase in order to determine which CEs and FEs should communicate with each other. Communication across the Gl reference point is optional in this architecture. No requirements are placed on this reference point. CE managers and FE managers may be operated by different entities. The operator of the CE manager may not want to divulge, except to specified FE managers, any characteristics of the CEs it manages. Similarly, the operator of the FE manager may not want to divulge FE characteristics, except to authorized entities. As such, CE managers and FE managers may need to authenticate one another. Subsequent communication between CE managers and FE managers may require other security functions such as privacy, non-repudiation, freshness, and integrity. Once the necessary security functions have been performed, the CE and FE managers communicate to determine which CEs and FEs should communicate with each other. At the very minimum, the CE and FE managers need to learn of the existence of available FEs and CEs respectively. This discovery process may or may not entail one or both managers learning the capabilities of the discovered ForCES protocol elements. Note that ForCES protocol can be used over reference point Gp for CEs to discover the detailed capabilities of FEs, since it is Anderson, et. al. Expires August 2002 [Page 6] Internet Draft ForCES Framework February 2002 required that ForCES be able to express the FEs' capability information in the form of FE model (See [FORCES-REQ], Section 6). 6.2. Gf Reference Point The Gf reference point is used to inform forwarding elements of the association decisions made by FE managers in pre-association phase. Only authorized entities may instruct a FE with respect to which CE should control it. Therefore, privacy, integrity, freshness, and authentication are necessary between FE manager and FEs when the FE manager is remote to the FE. Once the appropriate security has been established, FE manager instructs FEs across this reference point to join a new NE or to disconnect from an existing NE. Note that when the FE manager function may be co-located with the FE (such as by manual keypad entry of the CE IP address), in which case this reference point is reduced to a built-in function. 6.3. Gc Reference Point The Gc reference point is used to inform control elements of the association decisions made by CE managers in pre-association phase. When the CE manager is remote, only authorized entities may instruct a CE to control certain FEs. Privacy, integrity, freshness and authentication are also required across this reference point in such a configuration. Once appropriate security has been established, the CE manager instructs CEs as to which FEs they should control and how they should control them. As with the FE manager and FEs, configurations are possible where the CE manager and CE are co-located and no protocol is used for this function. 6.4. Gi Reference Point ForCES requires that packets MUST be able to arrive at the NE by one FE and leave the NE via a different FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that enter the NE via one FE and leave the NE via a different FE are transferred between FEs across the Gi reference point. The Gi reference point is a separate protocol from the Gp reference point and is not currently defined by the ForCES architecture. 6.5. Gr Reference Point Varying degrees of synchronization are necessary to provide redundancy, load sharing or distributed control between multiple CEs. However, in all cases, consistency protocols between CEs take place across the Gr reference point and are out of scope for ForCES protocol. Likewise, detecting the inability to synchronize due to a loss of connectivity between CEs is also out of scope. Anderson, et. al. Expires August 2002 [Page 7] Internet Draft ForCES Framework February 2002 It is not necessary to define any protocols across the Gr reference point to enable simple control/forwarding separation (i.e., single CE and multiple FEs). 6.6. Gp Reference Point Based on the information acquired through CEs' control processing, CEs will frequently need to manipulate the packet-forwarding behaviors of their FE(s) by sending instructions to FEs. This is performed across the Gp ("p" meaning protocol) reference point. In this architecture, the ForCES protocol is exclusively used for all communication across the Gp reference point. Packets addressed to any of NE's interfaces are typically redirected by the receiving FE to its CE, and CE may originate packets and have its FE deliver them. Therefore, the communication across the Gp reference point includes not only the control messages from CEs to FEs and the status or statistics report from FEs to CEs, but also the data packets that are redirected between them. Moreover, one FE may be controlled by multiple CEs. In this configuration, the control protocols supported by the FORCES NEs may spread across multiple CEs. For example, one CE may support OSPF and another supports BGP. FEs are configured to recognize these protocol packets and forward them to the corresponding CE. CEs and FEs can be connected by a variety of interconnect technologies, including Ethernet connections, backplanes, and ATM (cell) fabrics, etc. ForCES should be able to support each of these interconnects (see [FORCES-REQ] Section 5, requirement #1). The proximity between CEs and FEs can range from very close (same box/one hop) to close (same room/small number of hops) (see [FORCES- APP]). Other scenarios will be considered but are not the focus of ForCES. When CEs and FEs are separated beyond a single hop, ForCES will make use of an existing RFC2914 compliant L4 protocol with adequate reliability, security and congestion control (e.g. TCP, SCTP) for transport purposes. In the case where a physical FE cannot implement (e.g., due to the lack of a general purpose CPU) the ForCES protocol directly, a proxy FE can be used in the middle of Gp reference point. This allows the CE communicate to the physical FE via the proxy by using ForCES, while the proxy manipulates the physical FE using some intermediary form of communication (e.g., a non-ForCES protocol or DMA). In such an implementation, the combination of the proxy and the physical FE becomes one logical FE entity. 7. Applicability to RFC1812 [FORCES-REQ] Section 5, requirement #9 dictates that "All proposed ForCES architecture MUST explain how that architecture may be applied to support all of a router's functions as defined in [RFC1812]." RFC1812 discusses many important requirements for IPv4 Anderson, et. al. Expires August 2002 [Page 8] Internet Draft ForCES Framework February 2002 routers from the link layer to the application layer. This section addresses the relevant requirements for implementing IPv4 routers based on ForCES architecture and how ForCES satisfies these requirements. IPv4 routers must implement IP to support its packet forwarding function, which is driven by its FIB (Forwarding Information Base). This Internet layer forwarding (see [RFC1812] Section 5) functionality naturally belongs to FEs in the ForCES architecture. A router may implement transport layer protocols (like TCP and UDP) that are required to support application layer protocols (see [RFC1812] Section 6). One important class of application protocols is routing protocols (see [RFC1812] Section 7). In ForCES architecture, routing protocols are naturally implemented by CEs. All modern routing protocols require routers communicate with each other. This communication between CEs in different routers is supported in ForCES by the FEs' ability to redirect data packets addressed to routers (i.e., NEs) and CEs' ability to originate packets and have them delivered by their FEs. This communication occurs across Gp reference point inside each router and between neighboring routers' interfaces. After routing protocol update its routing tables, ForCES is used to send the new routing table entries from CEs to FEs. It is also required that certain events like interface up/down must be reported back to upper layer application. ForCES provides the mechanism for FEs to report such events back to CE. RFC1812 also dictates "Routers MUST be manageable by SNMP." (see [RFC1157] Section 8) In general, for post-association phase, most external management tasks (including SNMP) SHOULD be done through interaction with the CE in order to support the appearance of a single functional device. Therefore, it is recommended that SNMP management agent be implemented by CEs and the SNMP messages sent to FEs be redirected to their CEs. In certain conditions (e.g. CE/FE disconnection), it may be useful to allow SNMP to be used to diagnose and repair problems. However, care should be taken when exercising such mechanisms and guidelines are provided in [FORCES- REQ], Section 5, requirement #4. 8. Security Considerations The security necessary across each reference point except Gp is discussed in Section 6. The security requirements for reference point Gp are discussed in detail in [FORCES-REQ] Section 8. 9. Intellectual Property Right The authors are not aware of any intellectual property right issues pertaining to this document. 10. Normative References Anderson, et. al. Expires August 2002 [Page 9] Internet Draft ForCES Framework February 2002 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914, September 2000. [RFC1157] J. Case, et. al., "A Simple Network Management Protocol (SNMP)", RFC1157, May 1990. [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning (COPS-PR)", RFC3084, March 2001. [RFC1812] F. Baker, "Requirements for IP Version 4 Routers", RFC1812, June 1995. 11. Informative References [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation of IP Control and Forwarding", work in progress, February 2002, . [FORCES-APP] A. Crouch, et. al., "ForCES Applicability Statement", work in progress, February 2002, . 12. Acknowledgments The authors would like to thank Alan Crouch, David Putzolu and Hormuzd M. Khosravi for their invaluable comments and suggestions. 13. Authors' Addresses Todd A. Anderson Intel Labs 2111 NE 25th Avenue Hillsboro, OR 97124 USA Phone: +1 503 712 1760 Email: todd.a.anderson@intel.com Lily L. Yang Intel Labs 2111 NE 25th Avenue Hillsboro, OR 97124 USA Phone: +1 503 264 8813 Email: lily.l.yang@intel.com Ram Dantu Netrake Corporation 3000 Technology Drive Plano, Texas 75074 Phone: +1 214 291 1111 Email: ramd@netrake.com Anderson, et. al. Expires August 2002 [Page 10] Internet Draft ForCES Framework February 2002 1. Abstract........................................................1 2. Definitions.....................................................1 3. Introduction....................................................2 4. Architecture....................................................2 4.1. Control Elements...........................................3 4.2. Forwarding Elements........................................4 4.3. CE Managers................................................5 4.4. FE Managers................................................5 5. Operational Phases..............................................5 5.1. Pre-association Phase......................................5 5.2. Post-association Phase.....................................5 5.3. Association Re-establishment...............................6 6. Interaction between the Components..............................6 6.1. Gl Reference Point.........................................6 6.2. Gf Reference Point.........................................7 6.3. Gc Reference Point.........................................7 6.4. Gi Reference Point.........................................7 6.5. Gr Reference Point.........................................7 6.6. Gp Reference Point.........................................8 7. Applicability to RFC1812........................................8 8. Security Considerations.........................................9 9. Intellectual Property Right.....................................9 10. Normative References...........................................9 11. Informative References........................................10 12. Acknowledgments...............................................10 13. Authors' Addresses............................................10 Anderson, et. al. Expires August 2002 [Page 11]