| Network Working Group | K. Ogawa |
| Internet-Draft | NTT Corporation |
| Updates: 5810 (if approved) | W. M. Wang |
| Intended status: Standards Track | Zhejiang Gongshang University |
| Expires: May 24, 2014 | E. Haleplidis |
| University of Patras | |
| J. Hadi Salim | |
| Mojatatu Networks | |
| November 20, 2013 |
ForCES Intra-NE High Availability
draft-ietf-forces-ceha-09
This document discusses Control Element High Availability within a ForCES Network Element. Additionally this document updates [RFC5810] by providing new normative text for the Cold-Standby High availability mechanism.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 24, 2014.
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
The following definitions are taken from [RFC3654], [RFC3746] and [RFC5810]. They are repeated here for convenience as needed, but the normative definitions are found in the referenced RFCs:
Figure 1 illustrates a ForCES NE controlled by a set of redundant CEs with CE1 being active and CE2 and CEN being a backup.
-----------------------------------------
| ForCES Network Element |
| +-----------+ |
| | CEn | |
| | (Backup) | |
-------------- Fc | +------------+ +------------+ | |
| CE Manager |--------+-| CE1 |------| CE2 |-+ |
-------------- | | (Active) | Fr | (Backup) | |
| | +-------+--+-+ +---+---+----+ |
| Fl | | | Fp / | |
| | | +---------+ / | |
| | Fp| |/ |Fp |
| | | | | |
| | | Fp /+--+ | |
| | | +-------+ | | |
| | | | | | |
-------------- Ff | --------+--+-- ----+---+----+ |
| FE Manager |--------+-| FE1 | Fi | FE2 | |
-------------- | | |------| | |
| -------------- -------------- |
| | | | | | | | | |
----+--+--+--+----------+--+--+--+-------
| | | | | | | |
| | | | | | | |
Fi/f Fi/f
Fp: CE-FE interface
Fi: FE-FE interface
Fr: CE-CE interface
Fc: Interface between the CE Manager and a CE
Ff: Interface between the FE Manager and an FE
Fl: Interface between the CE Manager and the FE Manager
Fi/f: FE external interface
Figure 1: ForCES Architecture
The ForCES architecture allows FEs to be aware of multiple CEs but enforces that only one CE be the master controller. This is known in the industry as 1+N redundancy. The master CE controls the FEs via the ForCES protocol operating on the Fp interface. If the master CE becomes faulty, i.e. crashes or loses connectivity, a backup CE takes over and NE operation continues. By definition, the current documented setup is known as cold-standby. The set of CEs controlling an FE is static and is passed to the FE by the FE Manager (FEM) via the Ff interface and to each CE by the CE Manager (CEM) in the Fc interface during the pre-association phase.
From an FE perspective, the knobs of control for a CE set are defined by the FEPO LFB in [RFC5810], Appendix B. In Section 3.1 of this document we discuss further details of these knobs.
It is assumed that the reader is aware of the ForCES architecture to make sense of the changes being described in this document. This document provides background information to set the context of the discussion in Section 4.
At the time this document is being written, the Fr interface is out of scope for the ForCES architecture. However, it is expected that organizations implementing a set of CEs will need to have the CEs communicate to each other via the Fr interface in order to achieve the synchronization necessary for controlling the FEs.
The problem scope addressed by this document falls into 2 areas:
The NE recovery and availability is dependent on several time-sensitive metrics:
The design intent of the current [RFC5810] as well as this document to meet the above goals are driven by desire for simplicity.
To quantify the above criteria with the current prescribed ForCES CE setup in [RFC5810]:
To achieve CE High Availability (HA), FEs and CEs MUST inter-operate per [RFC5810] definition which is repeated for contextual reasons in Section 3.1. It should be noted that in this default setup, which MUST be implemented by CEs and FEs requiring HA, the Fr plane is out of scope (and if available is proprietary to an implementation).
As mentioned earlier, although there can be multiple redundant CEs, only one CE actively controls FEs in a ForCES NE. In practice there may be only one backup CE. At any moment in time, only one master CE can control an FE. In addition, the FE connects and associates to only the master CE. The FE and the CE are aware of the primary and one or more secondary CEs. This information (primary, secondary CEs) is configured on the FE and the CE during pre-association by the FEM and the CEM respectively.
This section includes a new normative description that updates [RFC5810] for the Cold-Standby High Availability mechanism.
Figure 2 below illustrates the Forces message sequences that the FE uses to recover the connection in current defined cold-standby scheme.
FE CE Primary CE Secondary
| | |
| Association Establishment | |
| Capabilities Exchange | |
1 |<------------------------->| |
| | |
| State Update | |
2 |<------------------------->| |
| | |
| | |
| FAILURE |
| |
| Association Estbalishment,Capabilities Exchange |
3 |<----------------------------------------------->|
| |
| Event Report (primary CE down) |
4 |------------------------------------------------>|
| |
| State Update |
5 |<----------------------------------------------->|
Figure 2: CE Failover for Cold Standby
HA parameterization in an FE is driven by configuring the FE Protocol Object (FEPO) LFB.
The FEPO CEID component identifies the current master CE and the component table BackupCEs identifies the configured backup CEs. The FEPO FE Heartbeat Interval, CE Heartbeat Dead Interval, and CE Heartbeat policy help in detecting connectivity problems between an FE and CE. The CE Failover policy defines how the FE should react on a detected failure. The FEObject FEState component [RFC5812] defines the operational forwarding status and control. The CE can turn off the FE's forwarding operations by setting the FEState to AdminDisable and can turn it on by setting it to OperEnable. Note: [RFC5812] section 5.1 has an errata which describes the FEState as read-only when it should be read-write.
Figure 3 illustrates the defined state machine that facilitates the recovery of connection state.
The FE connects to the CE specified on FEPO CEID component. If it fails to connect to the defined CE, it moves it to the bottom of table BackupCEs and sets its CEID component to be the first CE retrieved from table BackupCEs. The FE then attempts to associate with the CE designated as the new primary CE. The FE continues through this procedure until it successfully connects to one of the CEs or until the CE Failover Timeout Interval (CEFTI) expires.
FE tries to associate
+-->-----+
| |
(CE changes master || | |
CE issues Teardown || +---+--------v----+
Lost association) && | Pre-Association |
CE failover policy = 0 | (Association |
+------------>-->-->| in +<----+
| | progress) | |
| | | |
| +--------+--------+ |
| CE Association | | CEFTI
| Response V | timer
| +------------------+ | expires
| |FE issue CEPrimaryDown ^
| V |
+-+-----------+ +------+-----+
| | (CE changes master || | Not |
| | CE issues Teardown || | Associated |
| | Lost association) && | +->---+
| Associated | CE Failover Policy = 1 |(May | FE |
| | | Continue | try v
| |-------->------->------>| Forwarding)| assn|
| | Start CEFTI timer | |-<---+
| | | |
+-------------+ +-------+-----+
^ |
| Successful V
| Association |
| Setup |
| (Cancel CEFTI Timer) |
+_________________________________________+
FE issue CEPrimaryDown event
Figure 3: FE State Machine considering HA
There are several events that trigger mastership changes: The master CE may issue a mastership change (by changing the CEID component), or teardown an existing association; and last, connectivity may be lost between the CE and FE.
When communication fails between the FE and CE (which can be caused by either the CE or link failure but not FE related), either the TML on the FE will trigger the FE PL regarding this failure or it will be detected using the heartbeat messages between FEs and CEs. The communication failure, regardless of how it is detected, MUST be considered as a loss of association between the CE and corresponding FE.
If the FE's FEPO CE Failover Policy is configured to mode 0 (the default), it will immediately transition to the pre-association phase. This means that if association is later re-established with a CE, all FE state will need to be re-created.
If the FE's FEPO CE Failover Policy is configured to mode 1, it indicates that the FE will run in HA restart recovery. In such a case, the FE transitions to the Not Associated state and the CEFTI timer [RFC5810] is started. The FE may continue to forward packets during this state depending upon the value of the CEFailoverPolicy component of the FEPO LFB. The FE recycles through any configured backup CEs in a round-robin fashion. It first adds its primary CE to the bottom of table BackupCEs and sets its CEID component to be the first secondary retrieved from table BackupCEs. The FE then attempts to associate with the CE designated as the new primary CE. If it fails to re-associate with any CE and the CEFTI expires, the FE then transitions to the pre-association state and FE will operationally bring down its forwarding path (and set the [RFC5812] FEObject FEState component to OperDisable).
If the FE, while in the not associated state, manages to reconnect to a new primary CE before CEFTI expires it transitions to the Associated state. Once re-associated, the CE may try to synchronize any state that the FE may have lost during disconnection. How the CE re-synchronizes such state is out of scope for the current ForCES architecture but would typically constitute the issuing of new configs and queries.
An explicit message (a Config message setting Primary CE component in ForCES Protocol object) from the primary CE, can also be used to change the Primary CE for an FE during normal protocol operation. In this case, the FE transitions to the Not Associated State and attempts to Associate with the new CE.
TML Level:
At this level, control of all lower layers, for example transport level (such as IP addresses, MAC addresses etc) and associated links going down are the role of the TML.
PL Level:
All other functionality, including configuring the HA behavior during setup, the Control Element IDs (CE IDs) used to identify primary and secondary CEs, protocol messages used to report CE failure (Event Report), Heartbeat messages used to detect association failure, messages to change the primary CE (Config), and other HA related operations described in Section 3.1, are the PL's responsibility.
To put the two together, if a path to a primary CE is down, the TML would help recover from a failure by switching over to a backup path, if one is available. If the CE is totally unreachable then the PL would be informed and it would take the appropriate actions described before.
In this section we describe small extensions to the existing scheme to enable hot standby HA. To achieve hot standby HA, we target to improve the specific goals defined in Section 2.2, namely:
As described in Section 3.1, in the pre-association phase the FEM configures the FE to make it aware of all the CEs in the NE. The FEM MUST configure the FE to make it aware which CE is the master and MAY specify any backup CE(s).
In order for the above to be achievable there is a need to make a few changes in the FEPO model. Appendix A contains the xml definition of the new version 1.1 of the FEPO LFB.
Changes from the version 1 of FEPO are:
Since no component from the FEPO v1 has been changed FEPO v1.1 retains backwards compatibility with CEs that know only version 1.0. These CEs however cannot make use of the HA options that the new FEPO provides.
The FE's FEPO LFB version 1.1 AllCEs table contains all the CE IDs that the FE may connect and associate with. The ordering of the CE IDs in this table defines the priority order in which an FE will connect to the CEs. This table is provisioned initially from the configuration plane (FEM). In the pre-association phase, the first CE (lowest table index) in the AllCEs table MUST be the first CE that the FE will attempt to connect and associate with. If the FE fails to connect and associate with the first listed CE, it will attempt to connect to the second CE and so forth, and cycles back to the beginning of the list until there is a successful association. The FE MUST associate with at least one CE. Upon a successful association, a component of the FEPO LFB, specifically the CEID component, identifies the current associated master CE.
While it would be much simpler to have the FE not respond to any messages from a CE other than the master, in practice it has been found to be useful to respond to queries and heartbeats from backup CEs. For this reason, we allow backup CEs to issues queries to the FE. Configuration messages (SET/DEL) from backup CEs MUST be dropped by the FE and logged as received errors.
Asynchronous events that the master CE has subscribed to, as well as heartbeats are sent to all associated-to CEs. Packet redirects continue to be sent only to the master CE. The Heartbeat Interval, the CE Heartbeat Policy (CEHB) and the FE Heartbeat Policy (FEHB) are global for all CEs(and changed only by the master CE).
Figure 4 illustrates the state machine that facilitates connection recovery with HA enabled.
FE tries to associate
+-->-----+
| |
(CE changes master || | |
CE issues Teardown || +---+--------v----+
Lost association) && | Pre-Association |
CE failover policy = 0 | (Association |
+------------>-->-->| in +<----+
| | progress) | |
| | | |
| +--------+--------+ |
| CE Association | | CEFTI
| Response V | timer
| +------------------+ | expires
| |FE issue CEPrimaryDown ^
| |FE issue PrimaryCEChanged ^
| V |
+-+-----------+ +------+-----+
| | (CE changes master || | Not |
| | CE issues Teardown || | Associated |
| | Lost association) && | +->----------+
| Associated | CE Failover Policy = 1 |(May | find first |
| | | Continue | associated v
| |-------->------->------>| Forwarding)| CE or retry|
| | Start CEFTI timer | | associating|
| | | |-<----------+
| | | |
+----+--------+ +-------+----+
| |
^ Found | associated CE
| or newly | associated CE
| V
| (Cancel CEFTI Timer) |
+_________________________________________+
FE issue CEPrimaryDown event
FE issue PrimaryCEChanged event
Figure 4: FE State Machine considering HA
Once the FE has associated with a master CE it moves to the post-association phase (Associated state). It is assumed that the master CE will communicate with other CEs within the NE for the purpose of synchronization via the CE-CE interface. The CE-CE interface is out of scope for this document. An election result amongst CEs may result in desire to change mastership to a different associated CE; at which point current assumed master CE will instruct the FE to use a different master CE.
FE CE#1 CE#2 ... CE#N
| | | |
| Association Establishment | | |
| Capabilities Exchange | | |
1 |<------------------------->| | |
| | | |
| State Update | | |
2 |<------------------------->| | |
| | | |
| Association Establishment | |
| Capabilities Exchange | |
3I|<-------------------------------------->| |
... ... ... ...
| Association Estbalishment,Capabilities Exchange |
3N|<----------------------------------------------->|
| | | |
4 |<------------------------->| | |
. . . .
4x|<------------------------->| | |
| FAILURE | |
| | | |
| Event Report (LastCEID changed) | |
5 |--------------------------------------->|------->|
| Event Report (CE#2 is new master) | |
6 |--------------------------------------->|------->|
| | |
7 |<-------------------------------------->| |
. . . .
7x|<-------------------------------------->| |
. . . .
Figure 5: CE Failover for Hot Standby
While in the post-association phase, if the CE Failover Policy is set to 1 and HAMode set to 2 (HotStandby) then the FE, after successfully associating with the master CE, MUST attempt to connect and associate with all the CEs that it is aware of. Figure 5 steps #1 and #2 illustrates the FE associating with CE#1 as the master and then proceeding to steps #3I to #3N the association with backup CEs CE#2 to CE#N. If the FE fails to connect or associate with some CEs, the FE MAY flag them as unreachable to avoid continuous attempts to connect. The FE MAY retry to reassociate with unreachable CEs when possible.
When the master CE for any reason is considered to be down, then the FE MUST try to find the first associated CE from the list of all CEs in a round-robin fashion.
If the FE is unable to find an associated FE in its list of CEs, then it MUST attempt to connect and associate with the first from the list of all CEs and continue in a round-robin fashion until it connects and associates with a CE or the CEFTI timer expires.
Once the FE selects an associated CE to use as the new master, the FE issues a PrimaryCEDown Event Notification to all associated CEs to notify them that the last primary CE went down (and what its identity was); a second event PrimaryCEChanged identifying the new master CE is sent as well to identify which CE the reporting FE considers to be the new master.
In most HA architectures there exists the possibility of split-brain. However, since in our setup the FE will never accept any configuration messages from any other than the master CE, we consider the FE as fenced against data corruption from the other CEs that consider themselves as the master. The split-brain issue becomes mostly a CE-CE communication problem which is considered to be out of scope.
By virtue of having multiple CE connections, the FE switchover to a new master CE will be relatively much faster. The overall effect is improving the NE recovery time in case of communication failure or faults of the master CE. This satisfies the requirement we set to achieve.
Following the policies outlined in "Guidelines for Writing an IANA Considerations Section in RFCs" [RFC5226], the Logical Functional Block (LFB) Class Names and Class Identifiers namespaces is updated.
A new column, LFB version, is added to the table after the LFB Class Name. The table now reads as follows:
| LFB Class Identifier | LFB Class Name | LFB Version | Description | Reference |
|---|---|---|---|---|
The same rules applies as defined in [RFC5812] with the addition that entries must provide the LFB version as a string.
Upon publication of this document, all current entries are assigned a value of 1.0.
New versions of already defined LFB, MUST NOT remove the previous version entries.
It would make sense to have LFB versions to appear in sequence in the registry. The table SHOULD be sorted, and the shorting should be done by Class ID first and then by version.
This document introduces the FE Protocol Object version 1.1 as follows:
| LFB Class Identifier | LFB Class Name | LFB Version | Description | Reference |
|---|---|---|---|---|
| 2 | FE Protocol Object | 1.1 | Defines parameters for the ForCES protocol operation | This document |
Security consideration as defined in section 9 of [RFC5810] applies securing each CE-FE communication. Multiple CEs associated with the same FE still require the same procedure to be followed on a per-association basis.
It should be noted that since the FE is initiating the association with a CE, a CE cannot initiate association with the FE and such message will be dropped. Thus the FE is secured from rogue or malfunctioning CEs.
While CE-CE plane is outside current scope of ForCES, we recognize that it may be subjected to attacks which may affect the CE-FE communication.
The following considerations should be made:
| [RFC5810] | Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, W., Dong, L., Gopal, R. and J. Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, March 2010. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. |
| [RFC5812] | Halpern, J. and J. Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, March 2010. |
| [RFC5226] | Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. |
| [RFC3654] | Khosravi, H. and T. Anderson, "Requirements for Separation of IP Control and Forwarding", RFC 3654, November 2003. |
| [RFC3746] | Yang, L., Dantu, R., Anderson, T. and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004. |
The xml has been validated against the schema defined in [RFC5812].
<LFBLibrary xmlns="urn:ietf:params:xml:ns:forces:lfbmodel:1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="lfb-schema.xsd" provides="FEPO">
<!-- XXX -->
<dataTypeDefs>
<dataTypeDef>
<name>CEHBPolicyValues</name>
<synopsis>
The possible values of CE heartbeat policy
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>CEHBPolicy0</name>
<synopsis>
The CE will send heartbeats to the FE
every CEHDI timeout if no other messages
have been sent since.
</synopsis>
</specialValue>
<specialValue value="1">
<name>CEHBPolicy1</name>
<synopsis>
The CE will not send heartbeats to the FE
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>FEHBPolicyValues</name>
<synopsis>
The possible values of FE heartbeat policy
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>FEHBPolicy0</name>
<synopsis>
The FE will not generate any heartbeats to the CE
</synopsis>
</specialValue>
<specialValue value="1">
<name>FEHBPolicy1</name>
<synopsis>
The FE generates heartbeats to the CE every FEHI
if no other messages have been sent to the CE.
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>FERestartPolicyValues</name>
<synopsis>
The possible values of FE restart policy
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>FERestartPolicy0</name>
<synopsis>
The FE restarts its state from scratch
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>HAModeValues</name>
<synopsis>
The possible values of HA modes
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>NoHA</name>
<synopsis>
The FE is not running in HA mode
</synopsis>
</specialValue>
<specialValue value="1">
<name>ColdStandby</name>
<synopsis>
The FE is running in HA mode cold Standby
</synopsis>
</specialValue>
<specialValue value="2">
<name>HotStandby</name>
<synopsis>
The FE is running in HA mode hot Standby
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>CEFailoverPolicyValues</name>
<synopsis>
The possible values of CE failover policy
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>CEFailoverPolicy0</name>
<synopsis>
The FE should stop functioning immediate and
transition to the FE OperDisable state
</synopsis>
</specialValue>
<specialValue value="1">
<name>CEFailoverPolicy1</name>
<synopsis>
The FE should continue forwarding even without an
associated CE for CEFTI. The FE goes to FE
OperDisable when the CEFTI expires and no
association. Requires graceful restart support.
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>FEHACapab</name>
<synopsis>
The supported HA features
</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>GracefullRestart</name>
<synopsis>
The FE supports Graceful Restart
</synopsis>
</specialValue>
<specialValue value="1">
<name>HA</name>
<synopsis>
The FE supports HA
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>CEStatusType</name>
<synopsis>Status values. Status for each CE</synopsis>
<atomic>
<baseType>uchar</baseType>
<specialValues>
<specialValue value="0">
<name>Disconnected</name>
<synopsis>No connection attempt with the CE yet
</synopsis>
</specialValue>
<specialValue value="1">
<name>Connected</name>
<synopsis>The FE connection with the CE at the TML
has been completed
</synopsis>
</specialValue>
<specialValue value="2">
<name>Associated</name>
<synopsis>The FE has associated with the CE
</synopsis>
</specialValue>
<specialValue value="3">
<name>IsMaster</name>
<synopsis>The CE is the master (and associated)
</synopsis>
</specialValue>
<specialValue value="4">
<name>LostConnection</name>
<synopsis>The FE was associated with the CE but
lost the connection
</synopsis>
</specialValue>
<specialValue value="5">
<name>Unreachable</name>
<synopsis>The CE is deemed as unreachable by the FE
</synopsis>
</specialValue>
</specialValues>
</atomic>
</dataTypeDef>
<dataTypeDef>
<name>StatisticsType</name>
<synopsis>Statistics Definition</synopsis>
<struct>
<component componentID="1">
<name>RecvPackets</name>
<synopsis>Packets Received</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="2">
<name>RecvErrPackets</name>
<synopsis>Packets Received from CE with errors
</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="3">
<name>RecvBytes</name>
<synopsis>Bytes Received from CE</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="4">
<name>RecvErrBytes</name>
<synopsis>Bytes Received from CE in Error</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="5">
<name>TxmitPackets</name>
<synopsis>Packets Transmitted to CE</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="6">
<name>TxmitErrPackets</name>
<synopsis>
Packets Transmitted to CE that incurred
errors
</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="7">
<name>TxmitBytes</name>
<synopsis>Bytes Transmitted to CE</synopsis>
<typeRef>uint64</typeRef>
</component>
<component componentID="8">
<name>TxmitErrBytes</name>
<synopsis>Bytes Transmitted to CE incurring errors
</synopsis>
<typeRef>uint64</typeRef>
</component>
</struct>
</dataTypeDef>
<dataTypeDef>
<name>AllCEType</name>
<synopsis>Table Type for AllCE component</synopsis>
<struct>
<component componentID="1">
<name>CEID</name>
<synopsis>ID of the CE</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="2">
<name>Statistics</name>
<synopsis>Statistics per CE</synopsis>
<typeRef>StatisticsType</typeRef>
</component>
<component componentID="3">
<name>CEStatus</name>
<synopsis>Status of the CE</synopsis>
<typeRef>CEStatusType</typeRef>
</component>
</struct>
</dataTypeDef>
</dataTypeDefs>
<LFBClassDefs>
<LFBClassDef LFBClassID="2">
<name>FEPO</name>
<synopsis>
The FE Protocol Object, with new CEHA
</synopsis>
<version>1.1</version>
<components>
<component componentID="1" access="read-only">
<name>CurrentRunningVersion</name>
<synopsis>Currently running ForCES version</synopsis>
<typeRef>uchar</typeRef>
</component>
<component componentID="2" access="read-only">
<name>FEID</name>
<synopsis>Unicast FEID</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="3" access="read-write">
<name>MulticastFEIDs</name>
<synopsis>
the table of all multicast IDs
</synopsis>
<array type="variable-size">
<typeRef>uint32</typeRef>
</array>
</component>
<component componentID="4" access="read-write">
<name>CEHBPolicy</name>
<synopsis>
The CE Heartbeat Policy
</synopsis>
<typeRef>CEHBPolicyValues</typeRef>
</component>
<component componentID="5" access="read-write">
<name>CEHDI</name>
<synopsis>
The CE Heartbeat Dead Interval in millisecs
</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="6" access="read-write">
<name>FEHBPolicy</name>
<synopsis>
The FE Heartbeat Policy
</synopsis>
<typeRef>FEHBPolicyValues</typeRef>
</component>
<component componentID="7" access="read-write">
<name>FEHI</name>
<synopsis>
The FE Heartbeat Interval in millisecs
</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="8" access="read-write">
<name>CEID</name>
<synopsis>
The Primary CE this FE is associated with
</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="9" access="read-write">
<name>BackupCEs</name>
<synopsis>
The table of all backup CEs other than the
primary
</synopsis>
<array type="variable-size">
<typeRef>uint32</typeRef>
</array>
</component>
<component componentID="10" access="read-write">
<name>CEFailoverPolicy</name>
<synopsis>
The CE Failover Policy
</synopsis>
<typeRef>CEFailoverPolicyValues</typeRef>
</component>
<component componentID="11" access="read-write">
<name>CEFTI</name>
<synopsis>
The CE Failover Timeout Interval in millisecs
</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="12" access="read-write">
<name>FERestartPolicy</name>
<synopsis>
The FE Restart Policy
</synopsis>
<typeRef>FERestartPolicyValues</typeRef>
</component>
<component componentID="13" access="read-write">
<name>LastCEID</name>
<synopsis>
The Primary CE this FE was last associated
with
</synopsis>
<typeRef>uint32</typeRef>
</component>
<component componentID="14" access="read-write">
<name>HAMode</name>
<synopsis>
The HA mode used
</synopsis>
<typeRef>HAModeValues</typeRef>
</component>
<component componentID="15" access="read-only">
<name>AllCEs</name>
<synopsis>The table of all CEs</synopsis>
<array type="variable-size">
<typeRef>AllCEType</typeRef>
</array>
</component>
</components>
<capabilities>
<capability componentID="30">
<name>SupportableVersions</name>
<synopsis>
the table of ForCES versions that FE supports
</synopsis>
<array type="variable-size">
<typeRef>uchar</typeRef>
</array>
</capability>
<capability componentID="31">
<name>HACapabilities</name>
<synopsis>
the table of HA capabilities the FE supports
</synopsis>
<array type="variable-size">
<typeRef>FEHACapab</typeRef>
</array>
</capability>
</capabilities>
<events baseID="61">
<event eventID="1">
<name>PrimaryCEDown</name>
<synopsis>
The primary CE has changed
</synopsis>
<eventTarget>
<eventField>LastCEID</eventField>
</eventTarget>
<eventChanged/>
<eventReports>
<eventReport>
<eventField>LastCEID</eventField>
</eventReport>
</eventReports>
</event>
<event eventID="2">
<name>PrimaryCEChanged</name>
<synopsis>A New primary CE has been selected
</synopsis>
<eventTarget>
<eventField>CEID</eventField>
</eventTarget>
<eventChanged/>
<eventReports>
<eventReport>
<eventField>CEID</eventField>
</eventReport>
</eventReports>
</event>
</events>
</LFBClassDef>
</LFBClassDefs>
</LFBLibrary>