IP Storage Working Group Charles Monia INTERNET DRAFT Rod Mullendore Expires January 2002 Josh Tseng Nishan Systems Franco Travostino Victor Firoiu Nortel Networks David Robinson Sun Microsystems Wayland Jeong Troika Networks Rory Bolt Quantum/ATL Paul Rutherford ADIC Mark Edwards Eurologic July 2001 iFCP - A Protocol for Internet Fibre Channel Storage Networking Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [RFC2026]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Comments Comments should be sent to the ips mailing list (ips@ece.cmu.edu) or to the author(s). Monia, et al. Standards Track [Page 1] iFCP Revision 3 July 2001 Status of this Memo...................................................1 Comments..............................................................1 1. Abstract.....................................................4 2. About This Document..........................................4 2.1 Conventions used in this document............................4 2.2 Purpose of this document.....................................4 3. iFCP Introduction............................................4 3.1 Definitions..................................................5 4. Fibre Channel Communication Concepts.........................6 4.1 The Fibre Channel Network....................................7 4.2 Fabric Topologies............................................8 4.2.1 Switched Fibre Channel Fabrics..............................9 4.2.2 Mixed Fibre Channel Fabric..................................9 4.3 Fibre Channel Layers and Link Services......................10 4.3.1 Fabric-Supplied Link Services..............................11 4.4 Fibre Channel Devices.......................................11 4.5 Fibre Channel Information Elements..........................12 4.6 Fibre Channel Frame Format..................................12 4.7 Fibre Channel Transport Services............................13 4.8 Login Processes.............................................14 5. The iFCP Network Model......................................14 5.1 Fabric Topologies Supported by the iFCP Network Architecture 15 5.2 iFCP Transport Services.....................................16 5.2.1 Fibre Channel Transport Services Supported by iFCP.........16 5.3 The N_PORT Addressing Model.................................16 5.3.1 Operation in Address Transparent Mode......................19 5.3.2 Operation in Address Translation Mode......................20 6. iFCP Protocol...............................................23 6.1 Overview....................................................23 6.1.1 iFCP Transport Services....................................23 6.1.2 iFCP Support for Link Services............................24 6.2 TCP Stream Transport of iFCP Frames.........................24 6.2.1 iFCP Session Model.........................................25 6.2.2 iFCP Session Management....................................25 6.2.3 Terminating an N_PORT Login Session........................27 6.3 TCP Port Numbers............................................28 6.4 Encapsulation of Fibre Channel Frames.......................28 6.4.1 Encapsulation Header Format................................29 6.4.2 SOF and EOF Delimiter Fields...............................32 6.4.3 Frame Encapsulation........................................33 6.4.4 Frame De-encapsulation.....................................33 7. Fibre Channel Link Services.................................34 7.1 Augmented Link Service Messages.............................35 7.2 Augmented Link Services Requiring Payload Address Translation35 7.3 Augmented Link Services.....................................37 7.3.1 Abort Exchange (ABTX)......................................38 7.3.2 Discover Address (ADISC)...................................39 7.3.3 Discover Address Accept (ADISC ACC)........................40 7.3.4 FC Address Resolution Protocol Reply (FARP-REPLY)..........40 7.3.5 FC Address Resolution Protocol Request (FARP-REQ)..........41 7.3.6 Logout (LOGO)..............................................42 Monia Standards Track [Page 2] iFCP Revision 3 July 2001 7.3.7 Port Login (PLOGI).........................................43 7.3.8 Read Exchange Concise......................................44 7.3.9 Read Exchange Concise Accept...............................45 7.3.10 Read Exchange Status Block (RES).........................46 7.3.11 Read Exchange Status Block Accept........................46 7.3.12 Read Link Error Status (RLS).............................47 7.3.13 Read Sequence Status Block (RSS).........................48 7.3.14 Reinstate Recovery Qualifier (RRQ).......................49 7.3.15 Request Sequence Initiative (RSI)........................49 7.3.16 Third Party Process Logout (TPRLO).......................50 7.4 FLOGI Service Parameters Supported by an iFCP Gateway.......51 8. TCP Session Control Messages................................54 8.1 Connection Bind (CBIND).....................................55 8.2 Unbind Connection (UNBIND)..................................58 9. iFCP Error Detection........................................59 9.1 Overview....................................................59 9.2 Stale Frame Prevention......................................59 9.2.1 Enforcing R_A_TOV Limits...................................59 10. Fabric Services Supported by an iFCP implementation.........61 10.1 iFCP Support for the FC Broadcast Service...................61 11. Security....................................................62 11.1 Overview....................................................62 11.2 Physical Security...........................................62 11.3 Controlling Access..........................................62 11.4 Authentication and Encryption...............................63 11.5 Storage Firewalls...........................................63 12. Quality of Service Considerations...........................64 12.1 Minimal requirements........................................64 12.2 High-assurance..............................................64 13. References..................................................65 13.1 Relevant SCSI (T10) Specifications..........................65 13.2 Relevant Fibre Channel (T11) Specifications.................66 13.3 Relevant RFC Documents......................................66 13.4 Other Reference Documents...................................67 14. Author's Addresses..........................................67 A. iFCP Support for Fibre Channel Link Services................69 A.1 Basic Link Services.........................................69 A.2 Link Services Processed Transparently.......................69 A.3 Augmented Link Services.....................................70 B. Implementing an IP to FC Router with an iFCP Fabric.........72 C. Performance of The Multi-Connection iFCP Session Model......73 C.1 Relationship of Throughput to Packet Losses.................73 C.2 Background..................................................75 Full Copyright Statement.............................................76 Monia Standards Track [Page 3] iFCP Revision 3 July 2001 1. Abstract This document specifies an architecture and gateway-to-gateway protocol for the implementation of Fibre Channel fabric functionality on a network in which TCP/IP switching and routing elements replace Fibre Channel components. The protocol enables the attachment of existing Fibre Channel storage products to an IP network by supporting the fabric services required by such devices. 2. About This Document 2.1 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. All frame formats are in big endian network byte order. 2.2 Purpose of this document This is a standards-track document, which specifies a protocol for the implementation of Fibre Channel transport services on a TCP/IP network. Some portions of this document contain material from standards controlled by NCITS T10 and T11. This material is included here for informational purposes only. The authoritative information is given in the appropriate NCITS standards document. The authoritative portions of this document specify the protocol for mapping standards-compliant fibre Channel storage and adapter implementations to TCP/IP. This mapping includes sections of this document which describe the "iFCP Protocol" (see section 6). 3. iFCP Introduction iFCP is a gateway-to-gateway protocol, which provides Fibre Channel fabric services to Fibre Channel devices over a TCP/IP network. iFCP uses TCP to provide congestion control, error detection and recovery. iFCP's primary objective is to allow interconnection and networking of existing Fibre Channel devices at wire speeds over an IP network. The protocol and method of frame translation described in this document permit the attachment of Fibre Channel storage devices to an IP-based fabric by means of transparent gateways. The protocol achieves this transparency through a process that allows normal Fibre Channel frame traffic to pass through the gateway directly, with provisions, where necessary, for Monia Standards Track [Page 4] iFCP Revision 3 July 2001 intercepting and emulating the fabric services required by an FCP device. 3.1 Definitions Terms needed to clarify the concepts presented in this document are presented here. Address-translation mode - A mode of gateway operation in which the scope of N_PORT fabric addresses for locally attached devices are local to the iFCP gateway. Address-transparent mode - A mode of gateway operation in which the scope of N_PORT fabric addresses for all Fibre Channel devices are unique to the logical fabric to which the gateway belongs. Gateway Region - The portion of the iFCP storage network accessed through an iFCP gateway. Devices in the region consist of all Fibre Channel devices locally attached to the gateway. Logical Fabric - The union of two or more gateway regions configured to interoperate together in address-transparent mode. Fibre Channel Device - A device attached to a Fibre Channel fabric by means of the N_PORT interface described in [FC-FS]. Fibre Channel Network - A native Fibre Channel fabric and all attached Fibre Channel devices. Fabric - The components of a Fibre Channel network that provides the transport services defined in [FC-FS]. A fabric may be implemented in the IP framework by means of the architecture and protocols discussed in this document. Fabric Port - The interface through which an N_PORT accesses a Fibre Channel fabric. The type of fabric port depends on the Fibre Channel fabric topology. In this specification, all fabric port interfaces are functionally equivalent. FC-2 - The Fibre Channel transport services layer described in [FC- FS]. FCP Portal - An IP-addressable entity representing the point at which a logical or physical iFCP device is attached to the IP network. N_PORT - An iFCP or Fibre Channel entity representing the interface to Fibre Channel device functionality. This interface implements the Fibre Channel N_PORT semantics specified in [FC-FS]. Fibre Channel defines several variants of this Monia Standards Track [Page 5] iFCP Revision 3 July 2001 interface that are dependant on the Fibre Channel fabric topology. As used in this document, the term applies equally to all variants. N_PORT fabric address - The address of an N_PORT within the Fibre Channel fabric. N_PORT Alias -- The N_PORT address assigned by a gateway to represent a remote N_PORT accessed via the iFCP network. When routing frame traffic in address translation mode, the gateway automatically converts N_PORT aliases to N_PORT network addresses and vice versa. N_PORT Network Address - The address of an N_PORT in the IP fabric. This address consists of the IP address of the FCP Portal and the N_PORT ID of the locally-attached Fibre Channel device. F_PORT - The interface used by an N_PORT to access Fibre Channel switched fabric functionality. iFCP - The protocol discussed in this document. Logical FCP Device - The abstraction representing a single Fibre Channel device as it appears on an iFCP network. iSNS - The protocol by which storage name services are implemented. Resolution of Fibre Channel network object names is provided by an iSNS name server. N_PORT Session - An association created when two N_PORTS have executed a PLOGI operation. It is comprised of the N_PORTs and TCP connection that carries traffic between them. iFCP Frame - The frame inserted into the TCP stream which contains the Fibre Channel frame and iFCP header. Port Login (PLOGI) - The Fibre Channel Extended Link Service (ELS) that establishes an N_PORT login session through the exchange of identification and operation parameters between an originating N_PORT and a responding N_PORT. DOMAIN_ID - The value contained in the high-order byte of a 24-bit N_PORT Fibre Channel address. 4. Fibre Channel Communication Concepts Fibre Channel is a frame-based, serial technology designed for peer-to-peer communication between devices at gigabit speeds and with low overhead and latency. Monia Standards Track [Page 6] iFCP Revision 3 July 2001 This section contains a discussion of the Fibre Channel concepts that form the basis for the iFCP network architecture and protocol described in this document. Readers familiar with this material may skip to section 5. Material presented in this section is drawn from the following T11 specifications: -- The Fibre Channel Framing and Signaling Interface, [FC-FS] -- Fibre Channel Switch Fabric -2, [FC-SW2] -- Fibre Channel Generic Services, [FC-GS3] -- Fibre Channel Fabric Loop Attachment, [FC-FLA] The reader will find an in-depth treatment of the technology in [Kembel]. 4.1 The Fibre Channel Network The fundamental entity in Fibre Channel is the Fibre Channel network. Unlike a layered network architecture, a Fibre Channel network is largely specified by functional elements and the interfaces between them. As shown in Figure 1, these consist, in part, of the following: a) N_PORTs -- The end points for Fibre Channel traffic. In the FC standards, N_PORT interfaces have several variants, depending on the topology of the fabric to which they are attached. As used in this specification, the term applies to any one of the variants. b) FC Devices - The Fibre Channel devices to which the N_PORTs provide access. c) Fabric_Ports -- The interface within a fabric that provides Fibre Channel attachment for an N_PORT. The types of fabric port depend on the fabric topology and are discussed in section 4.2. d) The fabric infrastructure for carrying frame traffic between N_PORTs. e) Within a switched or mixed fabric (see section 4.2), a set of auxiliary servers, including a name server for device discovery and network address resolution. The types of service depend on the network topology. Monia Standards Track [Page 7] iFCP Revision 3 July 2001 +--------+ +--------+ +--------+ +--------+ | FC | | FC | | FC | | FC | | Device | | Device |<-------->| Device | | Device | |........| |........| |........| |........| | N_PORT | | N_PORT | | N_PORT | | N_PORT | +---+----+ +----+---+ +----+---+ +----+---+ | | | | +---+----+ +----+---+ +----+---+ +----+---+ | Fabric | | Fabric | | Fabric | | Fabric | | Port | | Port | | Port | | Port | +========+===+========+==========+========+==+========+==== | Fabric | | & | | Fabric Services | +-----------------------------------------------------+ Figure 1 -- A Fibre Channel Network The following sections describe Fibre Channel fabric topologies and give an overview of the Fibre Channel communications model. 4.2 Fabric Topologies The principal Fibre Channel fabric topologies consist of the following: a) Arbitrated Loop -- A series of N_PORTs connected together in daisy-chain fashion. Data transmission between N_PORTs requires arbitration for control of the loop in a manner similar to a token ring network. b) Switched Fabric -- A fabric consisting of switching elements, as described in section 4.2.1. c) Mixed Fabric -- A fabric consisting of switches and "fabric- attached" loops. A description can be found in [FC-FLA]. Depending on the topology, the N_PORT and fabric port variants through which a Fibre Channel device is attached to the network may be one of the following: Fabric Topology Fabric Port Type N_PORT Variant --------------- ---------------- -------------- Loop L_PORT NL_PORT Switched F_PORT N_PORT Mixed FL_PORT NL_PORT F_PORT N_PORT Monia Standards Track [Page 8] iFCP Revision 3 July 2001 The differences in each N_PORT variant and its corresponding fabric port are confined to the interactions between them. To an external N_PORT, all fabric ports are transparent and all remote N_PORTs are functionally identical. 4.2.1 Switched Fibre Channel Fabrics The internals of a multi-switch Fibre Channel fabric are shown below. +----------+ +----------+ | FC | | FC | | Device | | Device | |..........| |..........| | N_PORT |<........>| N_PORT | +----+-----+ +-----+----+ | | +----+-----+ +-----+----+ | F_PORT | | F_PORT | ==========+==========+==========+==========+============== | FC | | FC | | Switch | | Switch | +----------+ +----------+ Fibre Channel |Inter- | |Inter- | Fabric |Switch | |Switch | |Interface | |Interface | +-----+----+ +-----+----+ | | | | +-----+----+----------+-----+----+ |Inter- | |Inter- | |Switch | |Switch | |Interface | |Interface | +----------+ +----------+ | FC Switch | | | +--------------------------------+ Figure 2 -- Multi-Switch Fibre Channel Fabric The interface between switch elements is either proprietary or the standards-compliant E_PORT interface described by the FC-SW2 specification, [FC-SW2]. 4.2.2 Mixed Fibre Channel Fabric A mixed fabric contains one or more arbitrated loops connected to a switched fabric as shown in Figure 3. Monia Standards Track [Page 9] iFCP Revision 3 July 2001 +----------+ +----------+ +---------+ | FC | | FC | | FC | | Device | | Device | | Device | |..........| |..........| |.........| | N_PORT |<........>| NL_PORT +---+ NL_PORT | +----+-----+ +-----+----+ +----+----+ | | FC Loop | +----+-----+ +-----+----+ | | F_PORT | | FL_PORT +--------+ | | | | ==========+==========+==========+==========+============== | FC | | FC | | Switch | | Switch | +----------+ +----------+ |Inter- | |Inter- | |Switch | |Switch | |Interface | |Interface | +-----+----+ +-----+----+ | | | | +-----+----+----------+-----+----+ |Inter- | |Inter- | |Switch | |Switch | |Interface | |Interface | +----------+ +----------+ | FC Switch | | | +--------------------------------+ Figure 3 -- Mixed Fibre Channel Fabric As noted previously, the protocol for communications between peer N_PORTs is independent of the fabric topology, N_PORT variant and type of fabric port to which an N_PORT is attached. 4.3 Fibre Channel Layers and Link Services Fibre channel consists of the following layers: FC0 -- The interface to the physical media, FC1 -- The encoding and decoding of data and out-of-band physical link control information for transmission over the physical media, FC2 -- The transfer of frames, sequences and exchanges comprising protocol information units. FC3 -- Common Services, FC4 -- Application protocols, such as FCP, the Fibre Channel SCSI protocol. Monia Standards Track [Page 10] iFCP Revision 3 July 2001 In addition to the layers defined above, Fibre Channel defines a set of auxiliary operations, some of which are implemented within the transport layer fabric, called link services. These are required to manage the Fibre Channel environment, establish communications with other devices, retrieve error information, perform error recovery and other similar services. Some link services are executed by the N_PORT. Others are implemented internally within the fabric. These internal services are described in the next section. 4.3.1 Fabric-Supplied Link Services Servers internal to a switched fabric handle certain classes of Link Service requests. The servers appear as N_PORTS located at well-known N_PORT fabric addresses. Service requests use the standard Fibre Channel mechanisms for N_PORT-to-N_PORT communications. All switched fabrics must provide the following services: Fabric F_PORT server - Services an N_PORT request to access the fabric for communications. Fabric Controller -- Provides state change information to other N_PORTS. Used to inform other FC devices when an N_PORT exits or enters the fabric. Directory/Name Server - Allows N_PORTs to register information in a database or retrieve information about other N_PORTs. A switched fabric may also implement the following optional services: Broadcast Address/Server -- Transmits single-frame, class 3 sequences to all N_PORTs. Time Server -- Intended for the management of fabric-wide expiration timers or elapsed time values and is not intended for precise time synchronization. Management Server - Collects and reports management information, such as link usage, error statistics, link quality and similar items. Quality of Service Facilitator - Performs fabric-wide bandwidth and latency management. 4.4 Fibre Channel Devices A Fibre Channel device has one or more fabric-attached N_PORTs. The device and its N_PORTs have the following associated identifiers: Monia Standards Track [Page 11] iFCP Revision 3 July 2001 a) A world-wide unique identifier for the device, b) A world-wide unique identifier for each N_PORT attached to the device, c) For each N_PORT, a fabric-assigned N_PORT fabric address provided when the device is granted fabric access. This address is unique within the scope of the fabric. Each identifier is a 64-bit binary quantity having the format defined in [FC-FS]. Information about a Fibre Channel device, such as the Fibre Channel addresses and world wide names of its N_PORTs, can be discovered through the appropriate Fibre Channel name service queries. 4.5 Fibre Channel Information Elements The fundamental element of information in Fibre Channel is the frame. A frame consists of a fixed header and up to 2112 bytes of payload having the structure described in section 4.6. The maximum frame size that may be transmitted between a pair of Fibre Channel devices is negotiable up to the payload limit, based on the size of the frame buffers in each Fibre Channel device and the path MTU supported by the fabric. Operations involving the transfer of information between N_PORT pairs are performed through exchanges. In an exchange, information is transferred in one or more ordered series of frames referred to as sequences. Within this framework, an upper layer protocol is defined in terms of transactions carried by exchanges. Each transaction, in turn, consists of protocol information units, each of which is carried by an individual sequence within an exchange. 4.6 Fibre Channel Frame Format A Fibre Channel frame consists of the payload and a header containing the control information necessary to route frames between N_PORTs and manage exchanges and sequences. The following diagram gives a highly simplified view of the frame. Monia Standards Track [Page 12] iFCP Revision 3 July 2001 +-----------------------------+ | Start-of-frame Delimiter | +-----+-----------------------+<----+ | | Destination N_PORT | | | | Fabric Address (D_ID) | | | | (24-bits) | | +-----+-----------------------+ 24-byte | | Source N_PORT | Frame | | Fabric Address (S_ID) | Header | | (24 bits) | | +-----+-----------------------+ | | Control information for | | | frame type, exchange | | | management, IU | | | segmentation and | | | re-assembly | | +-----------------------------+<----+ | | | Frame payload | | (0 - 2112 bytes) | | | | | | | +-----------------------------+ | End-of-Frame Delimiter | +-----------------------------+ Figure 4 -- Fibre Channel Frame Format The source and destination N_PORT fabric addresses are embedded in the S_ID and D_ID fields respectively. 4.7 Fibre Channel Transport Services The Fibre Channel standard ([FC-FS]) defines the following classes of service provided by a fabric implementation: Class 1 - A dedicated physical circuit connecting two N_PORTs. Class 2 - A frame-multiplexed connection with end-to-end flow control and delivery confirmation. Class 3 - A frame-multiplexed connection with no provisions for end-to-end flow control or delivery confirmation. Class 3 service is similar to UDP or IP datagram service. Fibre channel storage devices using this class of service rely on the ULP implementation to detect and recover from transient device and transport errors. In addition to the above services, fabrics may implement additional quality of service policies. Monia Standards Track [Page 13] iFCP Revision 3 July 2001 For service classes other than class 1, the Fibre Channel fabric is not required to provide in-order delivery of frames. Within an exchange, it is the responsibility of the frame recipient to reconstruct the order in which frames were sent based on sequence information in the frame header. 4.8 Login Processes The Login processes are the means whereby an N_PORT establishes the operating environment necessary to communicate with the fabric and other N_PORTs. Fabric login and destination N_PORT login are established through procedures by which an N_PORT exchanges operating parameters with the fabric or another N_PORT. 5. The iFCP Network Model The purpose of the iFCP protocol is to enable the implementation of Fibre Channel mixed or switched fabric functionality on an IP network in which IP components and technology replace the Fibre Channel switching and routing infrastructure described in section 4.2. The following diagram shows a Fibre Channel fabric with attached devices. These are connected to the fabric through an N_PORT interface attached to a Fabric Port whose behavior is specified in [FC-FS]. In this case, the N_PORT and Fabric Port represent any of the variants described in section 4.2. Within the Fibre Channel device domain, fabric-addressable entities consist of other N_PORTs and devices internal to the fabric that perform the fabric services defined in [FC-GS3]. Fibre Channel Network +--------+ +--------+ | FC | | FC | | Device | | Device | |........| |........| Fibre Channel | N_PORT |<......>| N_PORT | Device Domain +---+----+ +----+---+ ^ | | | +---+----+ +----+---+ | | Fabric | | Fabric | | | Port | | Port | | ==========+========+========+========+============== | Fabric & | | | Fabric Services | v | | Fibre Channel +--------------------------+ Fabric Domain Figure 5 -- A Fibre Channel Fabric Monia Standards Track [Page 14] iFCP Revision 3 July 2001 Gateway Region Gateway Region +--------+ +--------+ +--------+ +--------+ | FC | | FC | | FC | | FC | | Device | | Device | Fibre | Device | | Device | Fibre |........| |........| Channel |........| |........| Channel | N_PORT | | N_PORT |<.........>| N_PORT | | N_PORT | Device +---+----+ +---+----+ Traffic +----+---+ +----+---+ Domain | | | | ^ +---+----+ +---+----+ +----+---+ +----+---+ | | Fabric | | Fabric | | Fabric | | Fabric | | | Port | | Port | | Port | | Port | | =+========+==+========+===========+========+==+========+========== | iFCP Layer |<--------->| iFCP Layer | | |....................| ^ |....................| | | FCP Portal | | | FCP Portal | v +--------+-----------+ | +----------+---------+ IP iFCP|Gateway Control iFCP|Gateway Fabric | Data | | | | | |<------Encapsulated Frames------->| | +------------------+ | | | | | +------+ IP Network +--------+ | | +------------------+ Figure 6 -- An iFCP Network with iFCP Gateways The above diagram shows one implementation of an equivalent iFCP fabric. Two gateway regions are shown. Each consists of Fibre Channel devices directly connected to the iFCP fabric through fabric ports implemented as part of the edge switch or gateway. Looking into the fabric port on the Fibre Channel side of the gateway, the network appears as a Fibre Channel fabric. Here, the gateway presents remote N_PORTs as directly attached devices. Conversely, on the IP network side, the gateway presents each locally connected N_PORT as a logical Fibre Channel device. 5.1 Fabric Topologies Supported by the iFCP Network Architecture An important property of this gateway architecture is that the fabric configuration and topology within the gateway region are opaque to the IP network and other gateway regions. That is, the topology in the gateway region, whether it is loop- or switch- based, is hidden from the IP network and from other gateway regions. As a result, support for specific FC fabric topologies becomes a gateway implementation issue. In such cases, the gateway Monia Standards Track [Page 15] iFCP Revision 3 July 2001 incorporates whatever functionality is required to distill and present locally attached N_PORTs as logical iFCP devices. Regarding fabric topologies, the examples in this specification show an N_PORT directly attached to a gateway fabric port, this is done to keep the illustrations simple and does not reflect any fundamental limitation in the fabric configuration that an implementation can support. 5.2 iFCP Transport Services N_PORT to N_PORT communications that traverse a TCP/IP network require the intervention of the iFCP layer within the gateway. This consists of the following operations: a) Execution of the frame addressing and mapping functions described in section 5.3. b) Execution of fabric-supplied link services addressed to one of the well-known Fibre Channel N_PORT addresses. c) Encapsulation of Fibre Channel frames for injection into the TCP/IP network and de-encapsulation of Fibre Channel frames received from the TCP/IP network. d) Establishment of an N_PORT login session in response to a PLOGI directed to a remote device. The following sections discuss the frame addressing mechanism and the way in which it is used to achieve communications transparency between N_PORTs. 5.2.1 Fibre Channel Transport Services Supported by iFCP An iFCP fabric supports Class 2 and Class 3 Fibre Channel transport services as specified in [FC-FS]. An iFCP fabric does not support Class 1 (dedicated connection). 5.3 The N_PORT Addressing Model This section discusses the role of the N_PORT addressing model in the routing of frames between locally and remotely attached N_PORTs. In the case of a remote N_PORT, where the frame traffic must traverse the IP network, the gateway must perform this routing transparently with respect to the locally attached N_PORT. To provide such transparency, the gateway maintains an association between the Fibre Channel address of a remote N_PORT, as seen by a locally attached device, and the corresponding address of the remote device on the IP network. To establish this association the Monia Standards Track [Page 16] iFCP Revision 3 July 2001 iFCP gateway assigns and manages Fibre Channel N_PORT fabric addresses as described in the following sections. The fabric address of an N_PORT device is a 24-bit value having the following format defined by the Fibre Channel specification [FC- FS]: Bit 23 16 15 8 7 0 +-----------+------------+----------+ | Domain ID | Area ID | Port ID | +-----------+------------+----------+ Figure 7 -- Fibre Channel Address Format Such addresses are volatile and subject to change based on modifications in the fabric configuration. In a Fibre Channel fabric, each switch element has a Domain I/D assigned by the principal switch. The value of the Domain I/D ranges from 1 to 239 (0xEF). Each switch element in turn administers a block of addresses divided into area and port IDs. N_PORTs logging into the fabric receive a unique fabric address consisting of the switch's Domain I/D concatenated with switch- assigned area and port I/Ds. These N_PORT addresses are carried in the Fibre Channel frame as shown in the following diagram. Bit 31 24 23 0 +--------+-----------------------------------+ Word 0 | | Destination N_PORT Address (D_ID) | +--------+-----------------------------------+ Word 1 | | Source N_PORT Address (S_ID) | +--------+-----------------------------------+ . | | . | Control information | . | and Payload | Word 534 +--------------------------------------------+ (Max) Figure 8 -- Fibre Channel Address Fields within a Frame The D_ID and S_ID fields represent the fabric addresses of the source and destination N_PORTs respectively. In an iFCP storage fabric, the iFCP gateway performs the address assignment and frame routing functions of an FC switch element. Unlike an FC switch, however, an iFCP gateway must also route frames to external devices attached to remote gateways on the IP network. In order to be transparent to FC devices, the gateway must route such frames using only the embedded 24-bit address. By exploiting Monia Standards Track [Page 17] iFCP Revision 3 July 2001 its control of address allocation and access to frame traffic entering or leaving the gateway region, it is able to achieve the necessary transparency. The gateway may allocate device addresses in one of two ways: a) Address Translation Mode - A mode of address assignment in which the gateway allocates an N_PORT device address that is unique to the gateway region. The address of a remote device is represented in that gateway region by a gateway assigned N_PORT alias. b) Address Transparent Mode - A mode of address assignment in which the gateway allocates an N_PORT address that is unique across the set of gateway regions comprising a logical fabric. Gateways within a logical fabric cooperate in the assignment of addresses to locally attached N_PORTs. Each gateway in control of a region is responsible for obtaining and distributing unique domain I/Ds from the address assignment authority as described in section 5.3.1.1. Consequently, within the scope of the logical fabric, the address of each N_PORT is unique. For that reason, gateway- assigned aliases are not required to represent remote N_PORTs. All iFCP implementations MUST support operation in address translation mode. Support for address transparent mode is optional. The choice of addressing mode involves the tradeoffs between scalability and transparency discussed below. The scalability constraints are a consequence of the Fibre Channel address allocation policy described above. As noted, an IP fabric using this address allocation scheme is limited to a combined total of 239 gateways and Fibre Channel switch elements. As the system expands, an IP fabric may consist of many switch elements distributed throughout the enterprise, each of which controls a small number of devices. In this case, the limitation in switch count may become a barrier to extending and fully integrating the storage network. Address Translation mode avoids this limitation by decoupling N_PORT fabric addresses from the constraints of fabric-wide address space management. Consequently, a virtually unlimited number of iFCP gateways, Fibre Channel devices and switch elements may be internetworked. This mode of address allocation also simplifies management of the IP storage fabric configuration by eliminating the need for a centralized address-assignment authority. A consequence of address translation mode is that the 24-bit N_PORT address is no longer unique across the storage network. As a result, when processing frame traffic to or from remote N_PORTs, the gateway must intervene to translate the 24-bit N_PORT addresses Monia Standards Track [Page 18] iFCP Revision 3 July 2001 between the sending and receiving gateway regions. These address operations involve: a) Translating the N_PORT I/Ds in the frame header and b) Translating N_PORT I/Ds carried in the payload of certain extended link service messages. The process of N_PORT I/D translation for the frame header is described in section 5.3.2. The processing for link services with frame addresses in the payload is described in section 7.1. The details of the address transparent and address translation operational modes are discussed in the following sections. 5.3.1 Operation in Address Transparent Mode In addition to the scalability limits discussed above, the following considerations and requirements apply to this mode of operation: a) There is increased dependency on the services of a central address assignment authority, such as iSNS. If connectivity with the server is lost, new DOMAIN_ID values cannot be automatically allocated as gateways and Fibre Channel switch elements are added to the logical fabric. As a result, new gateways and switch elements cannot be automatically added to the ip fabric. Of course, it is always possible to add and manage such additional components manually. b) Coordination of iSNS servers is required. Multiple iFCP gateways set up with independently-administered address servers must be completely torn down and slaved under a single iSNS name server before they can be configured into the same logical fabric. In contrast, operation in address translation mode requires only that the independent iSNS servers import client attributes from other iSNS servers, before clients under different iSNS authorities can be made to interoperate. c) iFCP gateways in transparent mode will not interoperate with iFCP gateways that are not in transparent mode. d) When interoperating with locally attached Fibre Channel fabrics, the iFCP gateway MUST assume control of DOMAIN_ID assignments in accordance with the appropriate Fibre Channel standard or specification. As described in section 5.3.1.1, DOMAIN_ID values assigned to FC switches in attached fabrics must be issued by the iSNS server or manually assigned. e) When operating in address transparent Mode, no Fibre Channel address translation SHALL take place, and no link service Monia Standards Track [Page 19] iFCP Revision 3 July 2001 Messages shall be augmented with additional information by the iFCP layer. The process for establishing the TCP/IP context associated with an N_PORT login session in this mode is similar to that specified for address translation mode (section 5.3.2). 5.3.1.1 Transparent Mode Domain I/D Management As described above, each gateway and Fibre Channel switch in a logical fabric must have a unique domain I/D. In a gateway region containing Fibre Channel switch elements, each element obtains a domain I/D by querying the principal switch as described in [FC- SW2] -- in this case the iFCP gateway itself. The gateway in turn may obtain domain I/Ds on demand from a central address allocation authority, such as an iSNS name server or manually from a pre- assigned block of IDs. In that sense, the address authority (e.g., iSNS) assumes the role of master switch for the logical fabric. 5.3.1.2 Incompatibility with Address Translation Mode iFCP gateways in address transparent mode shall not originate or accept frames that do not have the TRN bit set to one in the iFCP flags field of the encapsulation header (see section 6.4.1). The iFCP gateway shall immediately terminate any N_PORT sessions with the iFCP gateway from which it receives such frames. 5.3.2 Operation in Address Translation Mode This section summarizes the process for managing the assignment of addresses within a gateway region, including the modification of FC frame addresses embedded in the frame header for frames sent and received from remotely attached N_PORTs. As described above, the scope of N_PORT addresses in this mode is local to the gateway region. A principal switch within the gateway region, possibly the iFCP gateway itself, oversees the assignment of such addresses in accordance with the rules specified in [FC- SW2] and [FC-FLA]. The assignment of N_PORT addresses to locally attached devices is controlled by the switch element to which the device is connected. When a remotely attached N_PORT is accessed, the gateway assigns a locally significant N_PORT alias. This alias is used in place of the N_PORT I/D assigned by the remote gateway. To perform address conversion and enable the appropriate routing, the gateway maintains a table mapping N_PORT aliases to the appropriate TCP/IP connection context and N_PORT ID of all remotely accessed N_PORTs. Monia Standards Track [Page 20] iFCP Revision 3 July 2001 The gateway opportunistically builds the table of N_PORT addresses and TCP/IP connections for remotely attached devices in the IP fabric by: a) Intercepting name service requests issued by locally-attached N_PORTs as described below or, b) Intercepting incoming N_PORT login requests from external Fibre Channel devices and outgoing N_PORT login requests directed to remote N_PORTs. Such requests are used to establish the N_PORT login session as described in section 6.2.1. In response to name server requests, the iSNS server returns the IP address and N_PORT ID pair of the remote device. The IP address is mapped to the connection context. After saving the context and N_PORT ID, the iFCP layer creates the 24-bit N_PORT alias that is returned to the local N_PORT as the Fibre Channel address of the remotely attached device. 5.3.2.1 Translation Table Maintenance The contents of the gateway's address translation tables are updated opportunistically, in response to the name service queries and PLOGI requests described previously. There is no need to invalidate entries in response to changes in the fabric configuration, since any potentially stale entries caused by such events are self-correcting as described below. Once a fabric has achieved steady-state operation, any event that causes a change in the Fibre Channel address of a device also causes the device to terminate all N_PORT sessions. In the process of resuming operation, the status of the device, including its new address, is reflected in the name server's database. The new state of the device is advertised using the appropriate state change notifications. These, in turn, trigger the series of port login operations described below. For inbound PLOGI requests, the iFCP gateway simply updates the translation table, generates the N_PORT alias and forwards the request to the local N_PORT for processing as described above. For outbound requests, a fabric-attached Fibre Channel device usually precedes the PLOGI with a name server query to obtain the device's new N_PORT address. At this point, the iFCP gateway intercepts such a request, performs the necessary iSNS query, creates the translation table entry and returns the assigned N_PORT alias to the requester. After issuing the PLOGI, the N_PORT verifies that it has logged in with the expected device by checking the device name returned in the PLOGI response. Monia Standards Track [Page 21] iFCP Revision 3 July 2001 An N_PORT that attempts to execute a PLOGI without first querying the name server is still required to confirm the device name as described above. 5.3.2.2 Frame Address Translation For outbound frames, the table of external N_PORT network addresses are referenced to map the Destination N_PORT alias and Source N_PORT ID to the TCP connection context and the N_PORT ID assigned by the remote gateway. The translation process for outbound frames is shown below. Raw Fibre Channel Frame +--------+-----------------------------------+ +--------------+ | | Destination N_PORT Alias |--->| Lookup TCP | +--------+-----------------------------------+ | connection | | | Source N_PORT ID | | context | +--------------------------------------------+ | and N_PORT ID| | | +------+-------+ | Control information, | | TCP | Payload and FC CRC | | conn | | | context +--------------------------------------------+ | & | N_PORT | ID | After Address Translation and Encapsulation | +--------------------------------------------+ | | FC Encapsulation Header | | +--------------------------------------------+ | | SOF Delimiter Word | | +============================================+ | | | Destination N_PORT ID |<----------+ +--------+-----------------------------------+ | | Source N_PORT ID | +--------+-----------------------------------+ | | | Control information, Payload | | and FC CRC | +============================================+ | EOF Delimiter Word | +--------------------------------------------+ Figure 9 -- Outbound Frame Address Translation For inbound frames, a translation table lookup is performed to regenerate the N_PORT alias from the TCP connection context and N_PORT ID contained in the encapsulated FC frame. The translation process for inbound frames is shown below. Monia Standards Track [Page 22] iFCP Revision 3 July 2001 Network Format of Inbound Frame +--------------------------------------------+ TCP | FC Encapsulation Header | Connection +--------------------------------------------+ Context | SOF Delimiter Word | | +============================================+ V | | Destination N_PORT ID | +---+----+ +--------+-----------------------------------+ | Lookup | | | Source N_PORT ID |---->| Source | +--------+-----------------------------------+ | N_PORT | | | | Alias | | Control information, Payload | +----+---+ | and FC CRC | | Source +============================================+ | N_PORT | EOF Delimiter Word | | Alias +--------------------------------------------+ | | | Frame after Address Translation and De-encapsulation | +--------+-----------------------------------+ | | | Destination N_PORT ID | | +--------+-----------------------------------+ | | | Source N_PORT Alias |<---------+ +--------+-----------------------------------+ | | | Control information, Payload, | | and FC CRC | +--------------------------------------------+ Figure 10 -- Inbound Frame Address Translation 5.3.2.3 Incompatibility with Address Transparent Mode iFCP gateways in address translation mode shall not originate or accept frames that have the TRN bit set to one in the iFCP flags field of the encapsulation header. The iFCP gateway shall immediately abort any N_PORT login sessions with the iFCP gateway from which it receives such frames as described in section 6.2.3.2. 6. iFCP Protocol 6.1 Overview 6.1.1 iFCP Transport Services The main function of the iFCP protocol layer is to transport Fibre Channel frame images between locally and remotely attached N_PORTs. When transporting frames to a remote N_PORT, the iFCP layer encapsulates and routes the Fibre Channel frames comprising each Fibre Channel Information Unit via a predetermined TCP connection for transport across the IP network. Monia Standards Track [Page 23] iFCP Revision 3 July 2001 When receiving Fibre Channel frame images from the IP network, the iFCP layer de-encapsulates and delivers each frame to the appropriate N_PORT. The iFCP layer processes the following types of traffic: a) FC4 frame images associated with a Fibre Channel application protocol. b) FC2 frames comprising Fibre Channel link service requests and responses c) Fibre Channel broadcast frames d) iFCP control messages required to setup or terminate an iFCP session. For FC4 N_PORT traffic and most FC2 messages the iFCP layer never interprets the contents of the frame payload. iFCP does interpret and process iFCP control messages and certain FC2 extended link service messages as described in section 6.1.2 6.1.2 iFCP Support for Link Services iFCP must intervene in the processing of those Fibre Channel Extended Link Service (ELS) messages which contain N_PORT addresses in the message payload or require other special handling, such as an N_PORT login request (PLOGI). In the former case, an iFCP gateway operating in address translation mode must supplement the payload with additional information that enables the receiving gateway to convert such embedded N_PORT addresses to its frame of reference. For out-bound Fibre Channel frames comprising such an ELS, the iFCP layer creates the supplemental information based on frame content, modifies the frame payload, then transmits the resulting Fibre Channel frame with supplemental data through the appropriate TCP connection. For incoming iFCP frames containing supplemented Fibre Channel ELSs, iFCP interprets the frame, including any supplemental information, modifies the frame content, and forwards the resulting frame to the destination N_PORT for further processing. Section 7.1 describes the processing of these Extended Link Service messages in detail. 6.2 TCP Stream Transport of iFCP Frames Monia Standards Track [Page 24] iFCP Revision 3 July 2001 6.2.1 iFCP Session Model An iFCP session consists of the pair of N_PORTs comprising the session endpoints joined by a single TCP/IP connection. An N_PORT is identified by its network address consisting of: a) The N_PORT I/D assigned by the gateway to which the N_PORT is locally attached and b) The IP address of the gateway's iFCP Portal. Since only one iFCP session may exist between a pair of N_PORTs, the iFCP session is uniquely identified by the network addresses of the session end points. TCP connections that may be used for iFCP sessions between pairs of iFCP portals are either "bound" or "unbound". An unbound connection is a TCP connection that is not actively supporting an iFCP session. A pool of unbound connections may be established to reduce the session setup time. Pre-existing TCP connections between FCP Portals remain unbound and uncommitted until allocated to an iFCP session through a CBIND message (see section 8.1). When the iFCP layer detects a Port Login (PLOGI) message creating an iFCP session between a pair of N_PORTs, it may select an existing unbound TCP connection or establish a new TCP connection, and send the CBIND message down that TCP connection. This allocates the TCP connection to that PLOGI login session. 6.2.2 iFCP Session Management This section describes the protocols for establishing and terminating an N_PORT login session. 6.2.2.1 Creating an iFCP Session An iFCP session may be in one of the following states: a) OPEN -- The session state in which Fibre Channel frame images may be sent and received. b) OPEN PENDING -- The session state after a gateway has issued a CBIND request but no response has yet been received. No Fibre Channel frames may be sent. The gateway SHALL initiate the creation of an iFCP session in response to a PLOGI ELS directed to a remote N_PORT from a locally attached N_PORT as described in the following steps. a) If no iFCP session exists, allocate a TCP connection to the remote gateway. An existing connection in the Unbound state may Monia Standards Track [Page 25] iFCP Revision 3 July 2001 be used or a new connection may be created and placed in the Unbound state. b) If a connection cannot be allocated or created, the gateway SHALL terminate the PLOGI with an LS_RJT response. The Reason Code field in the LS_RJT message shall be set to 0x09 (Unable to Perform Command Request) and the Reason Explanation SHALL be set to 0x29 (Insufficient Resources to Support Login). c) If an iFCP session in the OPEN state already exists to the remote N_PORT, the gateway SHALL forward the PLOGI ELS using the existing session. d) If the iFCP session does not exist, the gateway SHALL issue a CBIND session control message (see section 8.1) and place the session in the OPEN PENDING state. e) In the event that a CBIND response is returned with one of the following statuses, the PLOGI shall be terminated with an LS_RJT message. Depending on the CBIND failure status, the Reason Code and Reason Explanation SHALL be set to the following values specified in [FC-FS]. CBIND Failure LS_RJT Reason LS_RJT Reason Code Status Code Explanation ------------- ------------- ------------------ Unspecified Unable to Perform No additional Reason (16) Command Request explanation (0x00) (0x09) No Such Device Unable to Perform Invalid N_PORT Name (17) Command Request (0x0D). (0x09) Lack of Unable to Perform Insufficient Resources (19) Command Request Resources to Support (0x09). Login (0x29). Incompatible Unable to Perform No additional address Command Request Explanation (0x00) translation mode (0x09) (20) Incorrect Unable to Perform No additional protocol version Command Request explanation (0x00) number (21) (0x09) f) A CBIND response with a CBIND STATUS of "N_PORT session already exists" indicates that the remote gateway has concurrently initiated a CBIND request to create an iFCP session between the same pair of N_PORTs. The receiving gateway SHALL terminate this Monia Standards Track [Page 26] iFCP Revision 3 July 2001 attempt and prepare to respond to an incoming CBIND request as described below. The gateway receiving a CBIND request SHALL respond as follows: a) If the receiver has a duplicate iFCP session in the OPEN PENDING state, then the receiving gateway SHALL compare the IP address of the sender with the receiver's IP address. b) If the sender's IP address is greater, the receiver shall issue a CBIND response of "Success" and SHALL place the session in the OPEN state. c) If the sender's IP address is less, the receiver shall issue a CBIND RESPONSE of Failed - N_PORT session already exists. The state of the receiver-initiated iFCP session SHALL BE unchanged. d) If there is no duplicate iFCP session, the receiving gateway SHALL issue a CBIND response. If a status of Success is returned, the receiving gateway SHALL create the iFCP session and place it in the OPEN state. 6.2.3 Terminating an N_PORT Login Session An N_PORT login session SHALL be terminated or aborted in response to one of the following events: a) An LS_RJT response is returned to the gateway that issued the PLOGI ELS. The gateway SHALL forward the LS_RJT to the local N_PORT and complete the session as described in section 6.2.3.1. b) An ACC received from a remote device in response to a LOGO. The gateway SHALL forward the ACC to the local N_PORT and complete the session as described in section 6.2.3.1. c) For an FC frame received from the IP network, a gateway detects a CRC error in the encapsulation header. The gateway shall abort the session as described in section 6.2.3.2. d) The TCP connection associated with the login session fails for any reason. The gateway detecting the failed connection shall abort the session as described in section 6.2.3.2. 6.2.3.1 N_PORT Login Session Completion An N_PORT login session is completed in response to a rejected PLOGI request or a successful LOGO ELS. The gateway receiving one of the above responses shall issue an Unbind session control ELS as described in section 8.2. An Unbind error shall be considered a fatal gateway error. Monia Standards Track [Page 27] iFCP Revision 3 July 2001 In response to the Unbind message, either gateway may choose to close the connection or return it to the pool of unbound connections. 6.2.3.2 Aborting an N_PORT Login Session An N_PORT login session SHALL be aborted if the TCP connection is spontaneously terminated or whenever one of the following occurs: a) An encapsulation error is detected as described in section 6.4.3. b) The gateway receives an encapsulated frame from a gateway operating in an incompatible address translation mode as specified in section 5.3.2.3 or 5.3.1.2. In any event, the TCP connection shall be closed. If the local N_PORT has logged in to the remote N_PORT, the gateway SHALL send a LOGO to the local N_PORT. 6.3 TCP Port Numbers An FCP Portal uses a single port number to receive TCP connection requests for iFCP over TCP. All TCP connections established between FCP Portals must be directed to the registered well known port number assigned by the IANA. An FCP Portal may use any TCP port number consistent with its implementation of the TCP/IP stack to initiate a TCP connection, but each port number must be unique. 6.4 Encapsulation of Fibre Channel Frames This section describes the iFCP encapsulation of Fibre Channel frames. The encapsulation is based on the common encapsulation format defined in [ENCAP]. The format of an encapsulated frame is shown below: +--------------------+ | Header | +--------------------+-----+ | SOF | f | +--------------------+ F r | | FC frame content | C a | +--------------------+ m | | EOF | e | +--------------------+-----+ Figure 11 -- Encapsulation Format The encapsulation consists of a 7-word header, an SOF delimiter word, the FC frame (including the Fibre Channel CRC), and an EOF Monia Standards Track [Page 28] iFCP Revision 3 July 2001 delimiter word. The header and delimiter formats are described in the following sections. 6.4.1 Encapsulation Header Format W|------------------------------Bit------------------------------| o| | r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 | d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0| Protocol# | Version | -Protocol# | -Version | +---------------+---------------+---------------+---------------+ 1| Reserved (must be zero) | +---------------+---------------+---------------+---------------+ 2| LS_COMMAND | iFCP Flags | SOF | EOF | +-----------+---+---------------+-----------+---+---------------+ 3| Flags | Frame Length | -Flags | -Frame Length | +-----------+-------------------+-----------+-------------------+ 4| Time Stamp [integer] | +---------------------------------------------------------------+ 5| Time Stamp [fraction] | +---------------------------------------------------------------+ 6| CRC | +---------------------------------------------------------------+ Common Encapsulation Fields: Monia Standards Track [Page 29] iFCP Revision 3 July 2001 Protocol# IANA-assigned protocol number identifying the protocol using the encapsulation. For iFCP the value is (/TBD/). Version Encapsulation version -Protocol# Ones complement of the protocol# -Version Ones complement of the version Flags Encapsulation flags (see 6.4.1.1) Frame Length Contains the length of the entire FC Encapsulated frame including the FC Encapsulation Header and the FC frame (including SOF and EOF words) in units of 32-bit words. -Flags Ones-complement of the Flags field. -Frame Length Ones-complement of the Frame Length field. Time Stamp [integer] Integer component of the frame time stamp in SNTP format [RFC2030]. Time Stamp Fractional component of the time stamp [fraction] in SNTP format [RFC2030]. CRC Header CRC. MUST be valid for iFCP. The time stamp fields are used to enforce the limit on the lifetime of a Fibre Channel frame as described in section 9.2.1. iFCP-specific fields: Monia Standards Track [Page 30] iFCP Revision 3 July 2001 LS_COMMAND For an augmented ELS ACC response, the LS_COMMAND field SHALL contain bits 31 through 24 of the LS_COMMAND to which the ACC applies. Otherwise the LS_COMMAND field shall be set to zero. iFCP Flags iFCP-specific flags (see below) SOF Copy of the SOF delimiter encoding (see section 6.4.2) EOF Copy of the EOF delimiter encoding (see section 6.4.2) The iFCP flags word has the following format: |------------------------Bit----------------------------| | | | 23 22 21 20 19 18 17 16 | +------+------+------+------+------+------+------+------+ | Reserved | SES | TRN | AUG | +------+------+------+------+------+------+------+------+ Figure 12 -- iFCP Flags Word iFCP Flags: SES 1 = Session control frame (TRN and AUG MUST be 0) TRN 1 = Address transparent mode enabled 0 = Address translation mode enabled AUG 1 = Augmented frame. 6.4.1.1 Common Encapsulation Flags The iFCP usage of the common encapsulation flags is shown below: |------------------------Bit--------------------------| | | | 31 30 29 28 27 26 | +--------------------------------------------+--------+ | Reserved | CRCV | +--------------------------------------------+--------+ For iFCP, the CRC field MUST be valid and CRCV MUST be set to one. Monia Standards Track [Page 31] iFCP Revision 3 July 2001 6.4.2 SOF and EOF Delimiter Fields The format of the delimiter fields is shown below. W|------------------------------Bit------------------------------| o| | r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 | d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0| +---------------+---------------+-------------------------------+ 0| SOF | SOF | -SOF | -SOF | +---------------+---------------+-------------------------------+ 1| | +----- FC frame content -----+ | | +---------------+---------------+-------------------------------+ n| EOF | EOF | -EOF | -EOF | +---------------+---------------+-------------------------------+ Figure 13 -- FC Frame Encapsulation Format SOF (bits 31-24 and bits 23-16 in word 0): iFCP uses the following subset of the SOF fields described in [ENCAP]. +-------+----------+ | FC | | | SOF | SOF Code | +-------+----------+ | SOFi2 | 0x2D | | SOFn2 | 0x35 | | SOFi3 | 0x2E | | SOFn3 | 0x36 | +-------+----------+ Table 1-- Translation of FC SOF Values to SOF Field Contents -SOF (bits 15-8 and 7-0 in word 0): The -SOF fields contain the ones complement of the value in the SOF fields. EOF (bits 31-24 and 23-16 in word n): iFCP uses the following subset of EOF fields specified in [ENCAP]. +-------+----------+ | FC | | | EOF | EOF Code | +-------+----------+ | EOFn | 0x41 | | EOFt | 0x42 | +-------+----------+ Table 2 -- Translation of FC EOF Values to EOF Field Contents -EOF (bits 15-8 and 7-0 in word n): The -EOF fields contain the one's complement of the value in the EOF fields. Monia Standards Track [Page 32] iFCP Revision 3 July 2001 iFCP implementations SHALL place a copy of the SOF and EOF delimiter codes in the appropriate header fields. 6.4.3 Frame Encapsulation A Fibre Channel Frame to be encapsulated MUST first be validated as described in [FC-FS]. Any frames received from a locally attached Fibre Channel device that do not pass the validity tests in [FC-FS] SHALL be discarded by the gateway. Frames types submitted for encapsulation and forwarding on the IP network SHALL have one of the SOF delimiters in Table 1 and an EOF delimiter from Table 2. Other valid frame types MUST be processed internally by the gateway as specified in the appropriate Fibre Channel Specification. Prior to submitting a frame for encapsulation, a gateway in address translation mode SHALL replace the D_ID address, and, if processing an augmented ELS, SHALL format the frame payload and add the supplemental information as specified in section 7.1. The gateway SHALL then calculate a new FC CRC on the reformatted frame. A gateway in address transparent mode MAY encapsulate and transmit the frame image without recalculating the FC CRC. The frame originator MUST then create and fill in the header and the SOF and EOF delimiter words as specified above. 6.4.4 Frame De-encapsulation The receiving gateway SHALL perform de-encapsulation as follows: Upon receiving the encapsulated frame, the gateway SHALL check the header CRC. If the header CRC is invalid, the gateway SHALL terminate the N_PORT login session as described in section 6.2.3.2. After validating the header CRC, the receiving gateway MAY verify the frame propagation delay as described in section 9.2.1. If the propagation delay is too long, the frame SHALL be discarded. Otherwise, the gateway SHALL check the SOF and EOF in the encapsulation header. A frame shall be discarded if it has an SOF code that is not in Table 1 or an EOF code that is not in Table 2. The gateway shall then de-encapsulate the frame. If operating in address translation mode, the gateway shall: a) Check the FC CRC and discard the frame if the CRC is invalid. b) Replace the S_ID with the N_PORT alias of the frame originator Monia Standards Track [Page 33] iFCP Revision 3 July 2001 c) If processing an augmented ELS, replace the ELS frame with a copy whose payload has been modified as specified in section 7.1. The resulting frame SHALL then be delivered to the N_PORT specified in the D_ID field. If the frame contents have been modified by the receiving gateway, a new FC CRC SHALL be calculated. 7. Fibre Channel Link Services Link services provide a set of Fibre Channel functions that allow a port to send control information or request another port to perform a specific control function. Each Link Service message (request and reply) is carried by a Fibre Channel sequence, and can be segmented into multiple frames. The iFCP Layer is responsible for transporting link service messages across the IP fabric. This includes mapping Link Service messages appropriately from the domain of the Fibre Channel transport to that of the IP network. This process may require special processing and the inclusion of augmented data by the iFCP layer. Each link service or extended link service is processed according to one of the following rules: a) Transparent - The link service message and reply MUST be transported to the receiving N_PORT by the iFCP gateway without altering the message payload. The link service message and reply are not processed by the iFCP implementation. b) Augmented - Applies to an extended link service reply or request containing Fibre Channel addresses in the payload or requiring other special processing by the iFCP implementation. The processing for augmented link services is described in this section. c) Rejected - When issued by a locally attached N_PORT, the specified link service request MUST be rejected by the iFCP implementation. The gateway SHALL respond to a rejected link service message by returning an LS_RJT response with a Reason Code of 0x0B (Command Not Supported) and a Reason Code Explanation of 0x0 (No Additional Explanation). This section describes the processing for augmented link services, including the manner in which augmentation data is transmitted over the IP network. Appendix A enumerates all link services and the iFCP processing policy that applies to each. Monia Standards Track [Page 34] iFCP Revision 3 July 2001 7.1 Augmented Link Service Messages Augmentation applies to extended link service requests that require the intervention of the iFCP layer. Such intervention is required in order to: a) Service any ELS that requires special handling, such as a PLOGI. b) In address translation mode only, service any ELS which has an N_PORT address in the payload. Such ELS messages are transmitted in a Fibre Channel frame having the following format: Word 31 24 23 0 +----------+------------------------------------------------+ 0| R_CTL | D_ID | | [22] | [Destination of extended link Service request] | +----------+------------------------------------------------+ 1| CS_CTL | S_ID | | | [Source of extended link service request] | +----------+------------------------------------------------+ 2| TYPE | F_CTL | +----------+------------------+-----------------------------+ 3| SEQ_ID | DF_CTL | SEQ_CNT | +----------+------------------+-----------------------------+ 4| OX_ID | RX_ID | +-----------------------------+-----------------------------+ 5| Parameter | | [ 00 00 00 00 ] | +-----------------------------------------------------------+ 6| LS_COMMAND | | [Extended Link Service Command Code] | +-----------------------------------------------------------+ 7| | .| Additional Service Request Parameters | .| ( if any ) | n| | +-----------------------------------------------------------+ Figure 14 -- Format of Extended Link Service Frame 7.2 Augmented Link Services Requiring Payload Address Translation This section describes the handling for ELS frames containing N_PORT addresses in the ELS payload. Such addresses SHALL only be translated when the gateway is operating in address translation mode. When operating in address transparent mode, these addresses SHALL NOT be translated and such ELS messages SHALL not be sent as augmented frames unless other special processing is required. Monia Standards Track [Page 35] iFCP Revision 3 July 2001 Supplemental data includes information required by the receiving gateway to convert an N_PORT address in the payload to an N_PORT address in the receiving gateway's address space. The following rules define the manner in which such supplemental data is packaged and referenced. For an N_PORT address field, the gateway originating the frame MUST set the value in the payload to identify the address translation type as follows: 0x00 00 01 - The gateway receiving the frame from the IP network MUST replace the contents of the field with the N_PORT alias of the frame originator. This translation type MUST be used when the address to be converted is that of the source N_PORT. 0x00 00 02 - The gateway receiving the frame from the IP network MUST replace the contents of the field with the N_PORT I/D of the destination N_PORT. This translation type MUST be used when the address to be converted is that of the destination N_PORT 0x00 00 03 - The gateway receiving the frame from the IP network MUST reference augmentation data to set the field contents. The augmentation information is the 64-bit world wide identifier of the N_PORT as set forth in the Fibre Channel specification [FC-FS]. If not otherwise part of the ELS, this information MUST be appended as described below. This translation type SHALL NOT be used when the address to be converted corresponds to that of the frame originator or recipient. Since Fibre Channel addressing rules prohibit the assignment of fabric addresses with a domain I/D of 0, the above codes will never correspond to valid N_PORT fabric IDs. For translation type 3, the receiving gateway SHALL obtain the information needed to fill in the ELS field by converting the specified N_PORT world-wide identifier to a gateway IP address and N_PORT ID. This information MUST be obtained through a name server query. If the N_PORT is locally attached, the gateway MUST fill in the field with the N_PORT ID. If the N_PORT is remotely attached, the gateway MUST assign and fill in the field with an N_PORT alias. If an N_PORT alias has already been assigned, it MUST be reused. In the event that the sending gateway cannot obtain the world wide identifier of an N_PORT, or a receiving gateway cannot obtain the IP address and N_PORT ID, the gateway detecting the error SHALL terminate the request with an LS_RJT message as described in [FCS]. The Reason Code SHALL be set to 0x07 (protocol error) and the Monia Standards Track [Page 36] iFCP Revision 3 July 2001 Reason Explanation SHALL be set to 0x1F (Invalid N_PORT identifier). [Editor's note: Such errors, when detected by the receiving gateway, may be indicative of a serious problem requiring a more drastic response. Therefore, this section should be regarded as tentative.] Supplemental data is sent with the ELS request or ACC frames in one of the following ways: a) By appending the necessary data to the end of the ELS frame. b) By extending the sequence with the addition of additional frames. In the first case, a new frame SHALL be created whose length includes the supplemental data. The procedure for extending the ELS sequence with additional frames is /TBS/. After applying the supplemental data, the receiving gateway SHALL forward the resulting ELS to the destination N_PORT with the supplemental information removed. When the ACC response must be augmented, the receiving gateway MUST act as a proxy for the originator, retaining the state needed to process the response from the N_PORT to which the request was directed. 7.3 Augmented Link Services The following Link Service Messages must receive special processing or be supplemented with additional control data. An encapsulated Fibre Channel frame that is part of an augmented ELS MUST have the AUG bit set to one in the iFCP FLAGS field of the encapsulation header as specified in section 6.4.1. The supplemental data (if any) MUST be appended as described in the following section. An ELS ACC frame that is augmented must be similarly formatted. Monia Standards Track [Page 37] iFCP Revision 3 July 2001 Link Service Message LS_COMMAND Mnemonic -------------------- ---------- -------- Abort Exchange 0x06 00 00 00 ABTX Discover Address 0x52 00 00 00 ADISC Discover Address Accept 0x02 00 00 00 ADISC ACC FC Address Resolution Protocol 0x55 00 00 00 FARP-REPLY Reply FC Address Resolution Protocol 0x54 00 00 00 FARP-REQ Request Logout 0x05 00 00 00 LOGO Port Login 0x30 00 00 00 PLOGI Read Exchange Concise 0x13 00 00 00 REC Read Exchange Concise Accept 0x02 00 00 00 REC ACC Read Exchange Status Block 0x08 00 00 00 RES Read Exchange Status Block 0x02 00 00 00 RES ACC Accept Read Link Error Status Block 0x0F 00 00 00 RLS Read Sequence Status Block 0x09 00 00 00 RSS Reinstate Recovery Qualifier 0x12 00 00 00 RRQ Request Sequence Initiative 0x0A 00 00 00 RSI Third Party Process Logout 0x24 00 00 00 TPRLO The formats of each augmented ELS, including supplemental data where applicable, are shown in the following sections. Each ELS diagram shows the basic format, as specified in the applicable FC standard, followed by supplemental data as shown below. +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | LS_COMMAND | +------+------------+------------+-----------+----------+ | 1 | | | . | | | . | ELS Payload | | | | | n | | +======+============+============+===========+==========+ | n+1 | | | . | Supplemental Data | | . | (if any) | | n+k | | +======+================================================+ ELS Diagram (single FC Frame Format) 7.3.1 Abort Exchange (ABTX) ELS Format: Monia Standards Track [Page 38] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x6 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | RRQ Status | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID of Tgt exchange | RX_ID of tgt exchange| +------+------------+------------+-----------+----------+ | 3-10 | Optional association header (32 bytes | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type (see (type 3 only) ------------------- section 7.2) ------------ ----------- Exchange Originator 1, 2 N/A S_ID Other Special Processing: None 7.3.2 Discover Address (ADISC) Format of ADISC ELS: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x52 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Reserved | Hard address of ELS Originator | +------+------------+------------+-----------+----------+ | 2-3 | Port Name of Originator | +------+------------+------------+-----------+----------+ | 4-5 | Node Name of originator | +------+------------+------------+-----------+----------+ | 6 | Rsvd | N_PORT I/D of ELS Originator | +======+============+============+===========+==========+ Monia Standards Track [Page 39] iFCP Revision 3 July 2001 Fields Requiring Translation Supplemental Data Address Translation Type (see (type 3 only) ------------------- section 7.2) ------------ ------------- N_PORT I/D of ELS 1 N/A Originator Other Special Processing: The Hard Address of the ELS originator shall be set to 0. 7.3.3 Discover Address Accept (ADISC ACC) Format of ADISC ACC ELS: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x20 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Reserved | Hard address of ELS Originator | +------+------------+------------+-----------+----------+ | 2-3 | Port Name of Originator | +------+------------+------------+-----------+----------+ | 4-5 | Node Name of originator | +------+------------+------------+-----------+----------+ | 6 | Rsvd | N_PORT I/D of ELS Originator | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type (see (type 3 only) ------------------- section 7.2) ------------ ------------ N_PORT I/D of ELS 1 N/A Originator Other Special Processing: The Hard Address of the ELS originator SHALL be set to 0. 7.3.4 FC Address Resolution Protocol Reply (FARP-REPLY) The FARP-REPLY ELS is used in conjunction with the FARP-REQ ELS (see section 7.3.5) to perform the address resolution services required by the FC-VI protocol [FC-VI]. Monia Standards Track [Page 40] iFCP Revision 3 July 2001 Format of FARP-REPLY ELS: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x55 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Match Addr | Requesting N_PORT Identifier | | | Code Points| | +------+------------+------------+-----------+----------+ | 2 | Responder | Responding N_PORT Identifier | | | Action | | +------+------------+------------+-----------+----------+ | 3-4 | Requesting N_PORT Port_Name | +------+------------+------------+-----------+----------+ | 5-6 | Requesting N_PORT Node_Name | +------+------------+------------+-----------+----------+ | 7-8 | Responding N_PORT Port_Name | +------+------------+------------+-----------+----------+ | 9-10 | Responding N_PORT Node_Name | +------+------------+------------+-----------+----------+ | 11-14| Requesting N_PORT IP Address | +------+------------+------------+-----------+----------+ | 15-18| Responding N_PORT IP Address | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type (see (type 3 only) ------------------- section 7.2) ----------------- ------------- Requesting N_PORT 2 N/A Identifier Responding N_PORT 1 N/A identifier Other Special Processing: None. 7.3.5 FC Address Resolution Protocol Request (FARP-REQ) The FARP-REQ ELS is used to in conjunction with the FC-VI protocol [FC-VI] to perform IP and FC address resolution in an FC fabric. The FARP-REQ ELS is usually directed to the fabric broadcast server Monia Standards Track [Page 41] iFCP Revision 3 July 2001 at well-known address 0xFF-FF-FF for retransmission to all attached N_PORTs. Section 10.1 describes the iFCP implementation of FC broadcast server functionality in an iFCP fabric. Format of FARP_REQ ELS: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x54 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Match Addr | Requesting N_PORT Identifier | | | Code Points| | +------+------------+------------+-----------+----------+ | 2 | Responder | Responding N_PORT Identifier | | | Action | | +------+------------+------------+-----------+----------+ | 3-4 | Requesting N_PORT Port_Name | +------+------------+------------+-----------+----------+ | 5-6 | Requesting N_PORT Node_Name | +------+------------+------------+-----------+----------+ | 7-8 | Responding N_PORT Port_Name | +------+------------+------------+-----------+----------+ | 9-10 | Responding N_PORT Node_Name | +------+------------+------------+-----------+----------+ | 11-14| Requesting N_PORT IP Address | +------+------------+------------+-----------+----------+ | 15-18| Responding N_PORT IP Address | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type (see (type 3 only) ------------------- section 7.2) ----------------- ----------- Requesting N_PORT 3 Requesting N_PORT Identifier Port Name Other Special Processing: None. 7.3.6 Logout (LOGO) ELS Format: Monia Standards Track [Page 42] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x5 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | N_PORT I/D being logged out | +------+------------+------------+-----------+----------+ | 2-3 | Port name of the LOGO originator (8 bytes) | +======+============+============+===========+==========+ This ELS shall always be sent as an augmented ELS regardless of the translation mode in effect. Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) -------------- ----------- N_PORT I/D Being 1 N/A Logged Out Other Special Processing: See section 6.2.3.1. 7.3.7 Port Login (PLOGI) PLOGI provides the mechanism for establishing a login session between two N_PORTs. In iFCP, a PLOGI request addressed to a remotely attached N_PORT may trigger the creation of an iFCP session, if one does not already exist. Otherwise, the PLOGI and ACC payloads MUST be passed transparently to the destination N_PORT. The PLOGI request and ACC response carry information identifying the originating N_PORT, including specification of its capabilities and limitations. If the destination N_PORT accepts the login request, it sends an accept (an ACC frame with PLOGI payload), specifying its capabilities and limitations. This exchange establishes the operating environment for the two N_PORTs. The following figure is duplicated from [FC-FS], and shows the PLOGI message format for both request and accept (ACC) response. A port will reject a PLOGI request by transmitting an LS_RJT message, which contains no payload. Monia Standards Track [Page 43] iFCP Revision 3 July 2001 Byte Offset +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | COMMON SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 20 | PORT NAME | 8 Bytes +----------------------------------+ 28 | NODE NAME | 8 Bytes +----------------------------------+ 36 | CLASS 1 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 52 | CLASS 2 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 68 | CLASS 3 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 86 | CLASS 4 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 102 | VENDOR VERSION LEVEL | 16 Bytes +----------------------------------+ Total Length = 116 bytes Figure 15 -- Format of PLOGI Request and ACC Payloads Details on the above fields, including common and class-based service parameters, can be found in [FC-FS]. 7.3.8 Read Exchange Concise ELS Format: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x13 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID | RX_ID | +======+============+============+===========+==========+ | 3-4 |Port name of the exchange originator (8 bytes) | | | (present only for translation type 3) | +======+============+============+===========+==========+ Monia Standards Track [Page 44] iFCP Revision 3 July 2001 Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1, 2 or 3 Port Name of the S_ID Exchange Originator Other Special Processing: None. 7.3.9 Read Exchange Concise Accept Format of ACC Response: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Acc = 0x02 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | OX_ID | RX_ID | +------+------------+------------+-----------+----------+ | 2 | Rsvd | Exchange Originator N_PORT ID | +------+------------+------------+-----------+----------+ | 3 | Rsvd | Exchange Responder N_PORT ID | +------+------------+------------+-----------+----------+ | 4 | Data Transfer Count | +------+------------+------------+-----------+----------+ | 5 | Exchange Status | +======+============+============+===========+==========+ | 6-7 |Port name of the Exchange Originator (8 bytes) | +======+============+============+===========+==========+ | 8-9 |Port name of the Exchange Responder (8 bytes) | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1, 2 or 3 Port Name of the N_PORT I/D Exchange Originator Exchange Responder 1, 2 or 3 Port Name of the N_PORT I/D Exchange Responder Monia Standards Track [Page 45] iFCP Revision 3 July 2001 When supplemental data is required, the ELS shall always be extended by 4 words as shown above. If the translation type for the Exchange Originator N_PORT I/D or the Exchange Responder N_PORT I/D is 1 or 2, the corresponding 8-byte port name SHALL be set to all zeros.Other Special Processing: None. 7.3.10 Read Exchange Status Block (RES) ELS Format: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x13 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID | RX_ID | +------+------------+------------+-----------+----------+ | 3-10 | Association header (may be optionally req'd) | +======+============+============+===========+==========+ | 11-12| Port name of the Exchange Originator (8 bytes) | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1, 2 or 3 Port Name of the S_ID Exchange Originator Other Special Processing: None. 7.3.11 Read Exchange Status Block Accept Format of ELS Accept Response: Monia Standards Track [Page 46] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Acc = 0x02 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | OX_ID | RX_ID | +------+------------+------------+-----------+----------+ | 2 | Rsvd | Exchange Originator N_PORT ID | +------+------------+------------+-----------+----------+ | 3 | Rsvd | Exchange Responder N_PORT ID | +------+------------+------------+-----------+----------+ | 4 | Exchange Status Bits | +------+------------+------------+-----------+----------+ | 5 | Reserved | +------+------------+------------+-----------+----------+ | 6-n | Service Parameters and Sequence Statuses | | | as described in [FCS] | +======+============+============+===========+==========+ |n+1- | Port name of the Exchange Originator (8 bytes) | |n+2 | | +======+============+============+===========+==========+ |n+3- | Port name of the Exchange Responder (8 bytes) | |n+4 | | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1, 2 or 3 Port Name of the N_PORT I/D Exchange Originator Exchange Responder 1, 2 or 3 Port Name of the N_0ORT I/D Exchange Responder When supplemental data is required, the ELS SHALL be extended by 4 words as shown above. If the translation type for the Exchange Originator N_PORT I/D or the Exchange Responder N_PORT I/D is 1 or 2, the corresponding 8-byte port name SHALL be set to all zeros. Other Special Processing: None. 7.3.12 Read Link Error Status (RLS) ELS Format: Monia Standards Track [Page 47] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x0F | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | N_PORT Identifier | +======+============+============+===========+==========+ | 2-3 | Port name of the N_PORT (8 bytes) | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data (type Address Translation Type(see 3 only) ------------------- section 7.2) ------------------ ----------- N_PORT Identifier 1, 2 or 3 Port Name of the N_PORT Other Special Processing: None. 7.3.13 Read Sequence Status Block (RSS) ELS Format: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x09 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | SEQ_ID | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID | RX_ID | +======+============+============+===========+==========+ | 3-4 |Port name of the Exchange Originator (8 bytes) | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1, 2 or 3 Port Name of the S_ID Exchange Originator Other Special Processing: Monia Standards Track [Page 48] iFCP Revision 3 July 2001 None. 7.3.14 Reinstate Recovery Qualifier (RRQ) ELS Format: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x12 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID | RX_ID | +------+------------+------------+-----------+----------+ | 3-10 | Association header (may be optionally req'd) | +======+============+============+===========+==========+ Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1 or 2 N/A S_ID Other Special Processing: None. 7.3.15 Request Sequence Initiative (RSI) ELS Format: +------+------------+------------+-----------+----------+ | Word | Bits 31-24 | Bits 23-16 | Bits 15-8 | Bits 7-0 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0x0A | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Rsvd | Exchange Originator S_ID | +------+------------+------------+-----------+----------+ | 2 | OX_ID | RX_ID | +------+------------+------------+-----------+----------+ | 3-10 | Association header (may be optionally req'd) | +======+============+============+===========+==========+ Monia Standards Track [Page 49] iFCP Revision 3 July 2001 Fields Requiring Translation Supplemental Data Address Translation Type(see (type 3 only) ------------------- section 7.2) ------------------ ----------- Exchange Originator 1 or 2 N/A S_ID Other Special Processing: None. 7.3.16 Third Party Process Logout (TPRLO) TPRLO provides a mechanism for an N_PORT (third party) to remove one or more process login sessions that exists between the destination N_PORT and other N_PORTs specified in the command. This command includes one or more TPRLO LOGOUT PARAMETER PAGEs, each of which when combined with the destination N_PORT identifies a SCSI login session which shall be terminated by the command. Byte Offset +----------------------------------+ 0 | LS_COMMAND | 1 Byte +----------------------------------+ 1 | PAGE LENGTH (0x10) | 1 Byte +----------------------------------+ 2 | PAYLOAD LENGTH (0x14) | 2 Bytes +----------------------------------+ 4 | TPRLO LOGOUT PARAMETER PAGE 1 | 2-4 Bytes +----------------------------------+ | . . . . | M Bytes +----------------------------------+ | TPRLO LOGOUT PARAMETER PAGE N | 2-4 Bytes +----------------------------------+ Total Length = Variable Each TPRLO LOGOUT PARAMETER PAGE identifies a remote N_PORT which when combined with the destination N_PORT identifies a SCSI session to be terminated. The TPRLO LOGOUT PARAMETER PAGE is of the following format: Monia Standards Track [Page 50] iFCP Revision 3 July 2001 Byte Offset +----------------------------------+ 0 | TYPE CODE | 1 Byte +----------------------------------+ 1 | TYPE CODE EXTENSION | 1 Byte +----------------------------------+ 2 | TPRLO FLAGS | 2 Bytes +----------------------------------+ 4 | ORIG PROCESS ASSOC (if present) | 4 Bytes +----------------------------------+ 8 | RESP PROCESS ASSOC (if present) | 4 Bytes +----------------------------------+ 12 | RESERVED | 1 Byte +----------------------------------+ 13 | THIRD PARTY ORIGINATOR N_PORT ID | 3 Bytes +----------------------------------+ When the iFCP header contains a TPRLO message (including the ACC response), iFCP supplemental data field will contain the PORT_NAME(s) (WWPN) identifying the N_PORT described by the equivalent TPRLO LOGOUT PARAMETER PAGE(s). If more than one TPRLO LOGOUT PARAMETER PAGE is contained in the Link Service message, the corresponding PORT_NAME shall also be included. PORT_NAMEs shall be listed in the same order as the equivalent TPRLO LOGOUT PARAMETER PAGEs in the original Link Service message. [The format for passing supplemental data is /TBS/] Additionally, the THIRD PARTY ORIGINATOR N_PORT ID field in each TPRLO LOGOUT PARAMETER PAGE shall be cleared when it is sent by the originating gateway. This applies to both the original Link Service message and the ACC response. When the iFCP layer receives a TPRLO message, it shall use the latter to replace the THIRD PARTY ORIGINATOR N_PORT ID in the original Link Service message, before forwarding it on to the upper Fibre Channel layers. Additional information on TPRLO can be found in [FC-FS]. 7.4 FLOGI Service Parameters Supported by an iFCP Gateway The FLOGI ELS is issued by an N_PORT that wishes to access the fabric transport services. The format of the FLOGI request and FLOGI ACC payloads are identical to the PLOGI request and ACC payloads described in section 7.3.7. The figure in that section is duplicated below for convenience. Monia Standards Track [Page 51] iFCP Revision 3 July 2001 Byte Offset +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | COMMON SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 20 | PORT NAME | 8 Bytes +----------------------------------+ 28 | NODE NAME | 8 Bytes +----------------------------------+ 36 | CLASS 1 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 52 | CLASS 2 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 68 | CLASS 3 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 86 | CLASS 4 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 102 | VENDOR VERSION LEVEL | 16 Bytes +----------------------------------+ Figure 16 -- FLOGI Request and ACC Payload Format A full description of each parameter is given in [FC-FS]. This section tabulates the service parameters supported by a fabric port attached to an iFCP gateway. FC service parameters are carried in the payload of a PLOGI or FLOGI extended link service request having the following format: Monia Standards Track [Page 52] iFCP Revision 3 July 2001 +-----------------------------------------+---------------+ | | Fabric Login | | Service Parameter | Class | | +---+---+---+---+ | | 1 | 2 | 3 | 4 | +-----------------------------------------+---+---+---+---+ | Class Validity | X | a | a | n | +-----------------------------------------+---+---+---+---+ | Service Options | | +-----------------------------------------+---+---+---+---+ | Intermix Mode | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Stacked Connect-Requests | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Sequential Delivery | n | M | M | n | +-----------------------------------------+---+---+---+---+ | Dedicated Simplex | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Camp on | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Buffered Class 1 | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Priority | n | a | a | n | +-----------------------------------------+---+---+---+---+ | Initiator Control | | +-----------------------------------------+---+---+---+---+ | Initial Responder Process Associator | n | n | n | n | +-----------------------------------------+---+---+---+---+ | ACK_0 capable | n | n | n | n | +-----------------------------------------+---+---+---+---+ | ACK generation assistance | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Clock synchronization ELS capable | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Recipient Control | | +-----------------------------------------+---+---+---+---+ | ACK_0 capable | n | n | n | n | +-----------------------------------------+---+---+---+---+ | X_ID interlock | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Error policy support | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Categories per sequence | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Clock synchronization ELS capable | n | n | n | n | +-----------------------------------------+---+---+---+---+ | Reserved - Fabric Specific | n | y | y | n | +-----------------------------------------+---+---+---+---+ Notes: 1) "y" indicates a parameter that applies to an iFCP gateway Monia Standards Track [Page 53] iFCP Revision 3 July 2001 2) "n" indicates the parameter is not applicable to an iFCP gateway 3) "M" indicates a parameter that MUSY be supported by an iFCP gateway 4) "X" indicates a parameter that MUST NOT be supported by an iFCP gateway 8. TCP Session Control Messages TCP session control messages are used to create and manage an iFCP session as described in section 6.2.2. They are passed between peer FCP Portals, and are only processed within the iFCP layer. The message format is based on the extended link service message template shown below. Word 3124 23<---------------Bits------------------------->0 +----------+------------------------------------------------+ 0| R_CTL | D_ID [0x00 00 00] | |[Req = 22]| [Destination of extended link Service request] | |[Rep = 23]| | +----------+------------------------------------------------+ 1| CS_CTL | S_ID [0x00 00 00] | | [0x0] | [Source of extended link service request] | +----------+------------------------------------------------+ 2|TYPE [0x1]| F_CTL [0] | +----------+------------------+-----------------------------+ 3|SEQ_ID | DF_CTL [0x00] | SEQ_CNT [0x00] | |[0x0] | | | +----------+------------------+-----------------------------+ 4| OX_ID [0x0000] | RX_ID_[0x0000] | +-----------------------------+-----------------------------+ 5| Parameter | | [ 00 00 00 00 ] | +-----------------------------------------------------------+ 6| LS_COMMAND | | [Session Control Command Code] | +-----------------------------------------------------------+ 7| | .| Additional Session Control Parameters | .| ( if any ) | n| | +===========================================================+ n| Fibre Channel CRC | +| | 1+===========================================================+ Figure 17 -- Format of Session Control Message Monia Standards Track [Page 54] iFCP Revision 3 July 2001 The LS_COMMAND value for the response remains the same as that used for the request. The session control ELS frame is terminated with a Fibre Channel CRC. The encapsulation header for the link Service frame carrying a TCP ELS message SHALL be set as follows: Encapsulation Header Fields: LS_COMMAND 0 iFCP Flags SES = 1 TRN = 0 AUG = 0 SOF code SOFi3 encoding (0x2E) EOF code EOFt encoding (0x42) Time Stamp Integer 0,0 and Fraction fields The SOF and EOF delimiter words SHALL be set based on the SOF and EOF codes specified above. The following lists the session control messages and their corresponding LS_COMMAND values. Request LS_COMMAND Short Name iFCP Support ------- ---------- ---------- ----------- Connection Bind 0xE0 CBIND REQUIRED Unbind Connection 0xE4 UNBIND REQUIRED 8.1 Connection Bind (CBIND) As described in section 6.2.2.1, the CBIND message and response are used to bind an N_PORT login session to a specific TCP connectionand establish iFCP session. In the CBIND request message, the source and destination N_Ports are identified by the N_PORT network address (iFCP portal address and N_PORT ID). The following shows the format of the CBIND request. Monia Standards Track [Page 55] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Byte 0 | Byte 1 | Byte 2 | Byte 3 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0xE0 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Reserved | Addr Mode | iFCP Ver | +------+-------------------------+-----------+----------+ | 2 | User Info | +------+------------+------------+-----------+----------+ | 3 | | +------+ SOURCE PORT NAME | | 4 | | +------+------------------------------------------------+ | 5 | | +------+ DESTINATION PORT NAME | | 6 | | +------+------------------------------------------------+ Addr Mode - The address translation mode of the originating gateway. 0 = Address Translation mode, 1 = Address Transparent mode. iFCP Ver - iFCP version number. SHALL be set to 1. USER INFO - Contains any data desired by the requester. This info MUST be echoed by the recipient in the CBIND response message. SOURCE PORT NAME - Contains the originating N_PORT's World Wide Port Name (WWPN). DESTINATION PORT NAME - Contains the destination N_PORT's World Wide Port Name (WWPN). The following shows the format of the CBIND response. Monia Standards Track [Page 56] iFCP Revision 3 July 2001 +------+------------+------------+-----------+----------+ | Word | Byte 0 | Byte 1 | Byte 2 | Byte 3 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0xE0 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | Reserved | Addr Mode | iFCP Ver | +------+-------------------------+-----------+----------+ | 2 | User Info | +------+------------+------------+-----------+----------+ | 3 | | +------+ SOURCE PORT NAME | | 4 | | +------+------------------------------------------------+ | 5 | | +------+ DESTINATION PORT NAME | | 6 | | +------+-------------------------+----------------------+ | 7 | Reserved | CBIND Status | +------+-------------------------+----------------------+ | 8 | Reserved | CONNECTION HANDLE | +------+-------------------------+----------------------+ Total Length = 32 Addr Mode - The address translation mode of the responding gateway. 0 = Address Translation mode, 1 = Address Transparent mode. iFCP Ver - iFCP version number of the responding gateway. SHALL be set to 1. USER INFO - Contains the same value received in the USER INFO field of the CBIND request message. DESTINATION PORT NAME - Contains the destination N_PORT's World Wide Port Name (WWPN). CBIND STATUS - Indicates success or failure of the CBIND request. CBIND values are shown below. Value Description ----- ----------- 0 Successful - No other status 1 - 15 Reserved 16 Failed - Unspecified Reason 17 Failed - No such device 18 Failed - N_PORT session already exists 19 Failed - Lack of resources 20 Failed - Incompatible address translation mode 21 Failed - Incorrect protocol version number Others Reserved Monia Standards Track [Page 57] iFCP Revision 3 July 2001 CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the connection. 8.2 Unbind Connection (UNBIND) UNBIND is used to release a bound TCP connection and return it to the pool of unbound TCP connections. This message is transmitted in the connection that is to be unbound. The following is the format of the UNBIND request message. +------+------------+------------+-----------+----------+ | Word | Byte 0 | Byte 1 | Byte 2 | Byte 3 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0xE4 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | User Info | +------+------------+------------+-----------+----------+ | 2 | Reserved | Connection Handle | +------+------------+------------+----------------------+ | 3 | Reserved | +------+------------+------------+-----------+----------+ | 4 | Reserved | +------+------------+------------+-----------+----------+ CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the connection The following shows the format of the UNBIND response message. +------+------------+------------+-----------+----------+ | Word | Byte 0 | Byte 1 | Byte 2 | Byte 3 | +------+------------+------------+-----------+----------+ | 0 | Cmd = 0xE4 | 0x00 | 0x00 | 0x00 | +------+------------+------------+-----------+----------+ | 1 | User Info | +------+------------+------------+-----------+----------+ | 2 | Reserved | Connection Handle | +------+------------+------------+-----------+----------+ | 3 | Reserved | +------+------------+------------+-----------+----------+ | 4 | Reserved | +------+------------+------------+-----------+----------+ | 5 | Reserved | UNBIND Status | +------+------------+------------+-----------+----------+ UNBIND STATUS - Indicates the success or failure of the UNBIND request. Monia Standards Track [Page 58] iFCP Revision 3 July 2001 Value Description ----- ----------- 0 Successful - No other status 1 - 15 Reserved 16 Failed - Unspecified Reason 17 Failed - No such device 18 Failed - Connection ID Invalid Others Reserved CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the unbound connection. 9. iFCP Error Detection 9.1 Overview [FC-FS] defines error detection and recovery procedures. These Fibre Channel-defined mechanisms continue to be available in the iFCP environment. 9.2 Stale Frame Prevention Recovery from Fibre Channel protocol error conditions requires that frames associated with a failed or aborted exchange drain from the fabric before exchange resources can be safely reused. Since a Fibre Channel fabric does not preserve frame order, there is no deterministic way to purge such frames. Instead, the fabric guarantees that frame the lifetime will not exceed a specific limit (R_A_TOV). R_A_TOV is defined in [FC-FS] as "the maximum transit time within a fabric to guarantee that a lost frame will never emerge from the fabric". For example, a value of 2 x R_A_TOV is the minimum time that the originator of an ELS request or FC4 ELS request must wait for the response to that request. The Fibre Channel default value for R_A_TOV is 10 seconds. The iFCP fabric MAY actively enforce limits on R_A_TOV as described in section 9.2.1. 9.2.1 Enforcing R_A_TOV Limits The R_A_TOV limit on frame lifetimes MAY be enforced by means of the time stamp in the encapsulation header (see section 6.4.1) as described in this section. The budget for R_A_TOV SHOULD include allowances for the propagation delay through the gateway regions of the sending and receiving N_PORTs plus the propagation delay through the IP Monia Standards Track [Page 59] iFCP Revision 3 July 2001 network. This latter component is referred to in this specification as IP_TOV. If enforced by a gateway, IP_TOV SHOULD be set well below the value of R_A_TOV specified for the iFCP fabric and SHOULD be stored in the iSNS server. A rule of thumb is to set IP_TOV to 50 percent of R_A_TOV. The following paragraphs describe the requirements for synchronizing gateway time bases and the rules for measuring and enforcing propagation delay limits. The protocol for synchronizing a gateway time base is SNTP [RFC2030]. In order to insure that all gateways are time-aligned, a gateway SHOULD obtain the address of an SNTP-compatible time server via an iSNS query. If multiple time server addresses are returned by the query, the servers must be synchronized and the gateway may use any server in the list. Alternatively, the server may return a multicast group address in support of operation in Anycast mode. Implementation of Anycast mode is as specified in RFC 2030, including the precautions defined in that document. Multicast mode SHOULD NOT be used. An SNTP server may use any one of the time reference sources listed in RFC 2030. The resolution of the time reference MUST be 125 milliseconds or better. With regard to the time base, the gateway is in either the synchronized or unsychronized state. When in the unsynchronized state, the gateway SHALL: a) Set the time stamp field to 0,0 for all outgoing frames b) Ignore the time stamp field for all incoming frames. When in the synchronized state, the gateway SHALL a) Set the time stamp field for each outgoing frame in accordance with the gateway's internal time base b) Check the time stamp field of each incoming frame, following validation of the encapsulation header CRC as described in section 6.4.4. c) If the incoming frame has a time stamp of 0,0, the receiving gateway SHALL NOT test the frame to determine if it is stale. d) If the incoming frame has a non-zero time stamp, the receiving gateway SHALL compute the absolute value of the time in flight and SHALL compare it against the value of IP_TOV specified for the IP fabric. Monia Standards Track [Page 60] iFCP Revision 3 July 2001 e) If the result in step (d) exceeds IP_TOV, the encapsulated frame shall be discarded. Otherwise, the frame shall be de- encapsulated as described in section 6.4.4. A gateway SHALL enter the synchronized state upon receiving a successful response to an SNTP query. A gateway shall enter the unsynchronized state: a) Upon power up and before successful completion of an SNTP query b) Whenever the gateway looses contact with the SNTP server. If synchronization is lost, the gateway MAY choose to abort all N_PORT login sessions with all remote gateways. 10. Fabric Services Supported by an iFCP implementation An iFCP gateway implementation MUST support the following fabric services: N_PORT ID Value Description Section --------------- ----------- ------- 0xFF-FF-FE F_PORT Server /TBS/ 0xFF-FF-FD Fabric Controller /TBS/ 0xFF-FF-FC Directory/Name Server /TBS/ In addition, an iFCP gateway MAY support the FC broadcast server functionality described in section 10.1. 10.1 iFCP Support for the FC Broadcast Service In Fibre Channel, frames are broadcast by addressing them to the broadcast server at well-known address 0xFF-FF-FF. The broadcast server then replicates and delivers the frame to each attached N_PORT in all zones to which the originating device belongs. Only class 3 (datagram) service is supported. In an iFCP/iSNS system, outgoing frames to be broadcast are directed to the gateway-resident broadcast server by locally attached N_PORTs. The broadcast server then redistributes such frames as follows: a) One copy is sent to each locally attached N_PORT in the same discovery domain as the originator. Monia Standards Track [Page 61] iFCP Revision 3 July 2001 b) One copy is sent to the broadcast server in each remote gateway via a UDP datagram. The D_ID field is set to the well-known address of the FC broadcast server. The datagram encapsulation format is identical to the iFCP encapsulation format described in section 6.4. The UDP datagram SHALL be sent with the DF bit set to 0 in the IP header to allow IP fragmentation (see [RFC791]). On receiving an iFCP broadcast datagram via UDP, the broadcast server SHALL: a) Validate the encapsulation header as described in section 6.4.3. If the header is invalid, the frame SHALL be discarded. b) Convert the S_ID N_PORT address in the frame to an N_PORT alias as described in section 5.3.2, if address translation mode is in effect. c) If the AUG bit is set in the iFCP flags field, perform any special processing required by the ELS, including translation of any addresses in the payload. d) Replicate and redistribute the frame to all locally attached N_PORTs in the discovery domain of the sender. If no broadcast server is implemented, the receiving gateway SHALL discard an incoming broadcast frame from a remote gateway. Broadcast frames received from locally attached N_PORTs shall be processed as specified in[FC-GS3]. 11. Security 11.1 Overview As with any other IP-based network, an iFCP storage network has security issues which must be addressed with the appropriate security policies and enforcement resources. There are various levels of security paradigms which when applied appropriately to an iFCP network can provide sufficient levels of security, including data integrity, authentication, and privacy, depending on user needs. 11.2 Physical Security Most existing SCSI and Fibre Channel interconnections are deployed in private, physically isolated environments where hostile entities are not provided access to the SCSI and Fibre Channel interconnects. This is the most basic security mechanism, and may be a sufficient model in some cases for an iFCP network. 11.3 Controlling Access Monia Standards Track [Page 62] iFCP Revision 3 July 2001 A second level of security is the use of zoning. Zoning specifies which devices are allowed to communicate, and is similar in concept to VLAN (Virtual Local Area Network) technology. Zoning information is maintained in a Name Server. 11.4 Authentication and Encryption Where additional levels of data integrity and privacy are required for iFCP, existing IPSec specifications can be applied to iFCP. Because IPSec is a layer-3 technology and has no knowledge of TCP, UDP, or higher-level protocols such as iFCP and FCP, it can be applied transparently to iFCP. The following IETF documents describe the operational framework and automatic keying mechanisms for IPSec. [RFC2401] Security Architecture for the Internet Protocol [RFC2402] IP Authentication Header [RFC2406] IP Encapsulating Security Payload [RFC2407] The Internet IP Security Domain of Interpretation for ISAKMP [RFC2408] Internet Security Association and Key Management Protocol (ISAKMP) [RFC2409] The Internet Key Exchange (IKE) 11.5 Storage Firewalls Firewalls are a common and proven methodology for securing access to IP-based networks, and they can be appropriate for use in IP- based storage networks as well. A firewall is a choke point through which all transit traffic must transit in order to pass between two separate networks. Since all iFCP traffic uses a well- known IANA-assigned TCP port number, it can easily be recognized and inspected. Access to storage resources can be secured by setting up a single gateway through which all outside non-secured traffic must pass through in order to access resources in the storage network. Such a firewall can be a proxy host operating at the session or application layer, requiring authentication before allowing traffic to pass. It can also be a stateful inspection gateway which understands the iFCP protocol, and can passively inspect and discover security threats as they transit the gateway. A third option is to use a standard router access control list to filter authorized traffic based upon static parameters such as IP addresses and TCP port numbers. Monia Standards Track [Page 63] iFCP Revision 3 July 2001 12. Quality of Service Considerations 12.1 Minimal requirements Conforming iFCP protocol implementations SHALL correctly communicate gateway-to-gateway even across one or more intervening best-effort IP regions. The timings with which such gateway-to gateway communication is performed, however, will greatly depend upon BER, packet losses, latency, and jitter experienced throughout the best-effort IP regions. The higher these parameters, the higher will be the gap measured between iFCP observed behaviors and baseline iFCP behaviors (i.e., as produced by two iFCP gateways directly attached to one another). 12.2 High-assurance It is expected that many iFCP deployments will benefit from a high degree of assurance on the behaviors of the intervening IP regions, with resulting high-assurance on the overall end-to-end path, as directly experienced by Fibre Channel applications. Such assurance on the IP behaviors stems from the intervening IP regions supporting standard Quality-of-Service (QoS) techniques, fully complementary to iFCP, such as: a) Congestion avoidance by over-provisioning of the network b) Integrated Services [IntServ] QoS c) .Differentiated Services [DiffServ] QoS d) .Multi-Protocol Label Switching [MPLS] In the most general definition, two iFCP gateways are separated by one or more independently managed IP regions, some of which implement some of the QoS solutions mentioned above. The IP regions with these QoS solutions are said to support Service Level Agreements (SLAs). Such agreements finalize requirements on network parameters such as bandwidth, loss, latency, jitter, burst length. The requirements may be expressed in absolute or relative terms, and apply to a unidirectional flow of packets. Depending on the QoS techniques available, the dynamic stipulation of a SLA may require the iFCP gateway to interact with network ancillary functions such admission control and bandwidth brokers (with RSVP or other signalling protocols that an IP region may accept). Due to the fact that Fibre Channel Class 2 and Class 3 do not currently support fractional bandwidth guarantees, and that iFCP is committed to supporting Fibre Channel semantics, it is impossible for an iFCP gateway to autonomously infer bandwidth requirements from streaming Fibre Channel traffic. Rather, the requirements on bandwidth or other network parameters need to be injected out-of- band into a iFCP gateway (or the node that will actually negotiate Monia Standards Track [Page 64] iFCP Revision 3 July 2001 the SLA on the gateway's behalf) through mechanisms outside the scope of this specification (e.g., through a management interface into the iFCP gateway). The administrator of a iFCP gateway MAY thus stipulate a Service Level Agreement with the local IP region for one, several, or all of an iFCP gateway's TCP sessions used by iFCP. Alternately, this responsibility may be delegated to a node downstream. Since one TCP connection is dedicated to each N_PORT login session , an individual N_PORT to N_PORT flow can enjoy a customized SLA. To render the best emulation of Fibre Channel possible over IP, it is anticipated that typical SLAs will specify a fixed amount of bandwidth, null losses, and, to a lesser degree of relevance, low latency, and low jitter. For example, an IP region using DiffServ QoS may support SLAs of this nature by applying EF DSCPs to the iFCP traffic. For the same SLA, another IP region might as well use a different DSCP or different QoS techniques alltogether. The way different QoS techniques are re-mapped at the edge of different intervening IP regions is beyond the scope of this specification. [00-603] describes a proposal to add fractional bandwidth guarantees to Class 2 and 3 (migrating it from Class 4). In such proposal, the bandwidth parameters would surface in the FLOGI request and accept, and PLOGI request and accept. In this case, it will become possible for an iFCP gateway to trap this information and autonomously remap it onto the SLA negotiation mechanism required by the local IP region, without resorting to out-of-band QoS management. Such an in-band QoS mechanism would result in true end-to-end provisioning of network resources. Forthcoming revisions of this iFCP specification will build upon this new opportunity. 13. References 13.1 Relevant SCSI (T10) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854-7179 or (303) 792-2181, Fax: (303) 792-2192 [SAM] SCSI-3 Architecture Model (SAM), ANSI X3.270-1996 [SAM-2] SCSI Architecture Model-2 (SAM-2), Project 1157-D, revision 11 [SPC] SCSI Primary Commands (SPC), ANSI X3.301-1997 [SPC-2] SCSI Primary Commands-2 (SPC-2), Project 1236-D, revision 16 [FCP] Fibre Channel Protocol for SCSI (FCP), ANSI X3.269-1996 Monia Standards Track [Page 65] iFCP Revision 3 July 2001 [FCP-2] Fibre Channel Protocol for SCSI, Second Revision (FCP- 2), Project 1144D, revision 04 13.2 Relevant Fibre Channel (T11) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854-7179 or (303) 792-2181, Fax: (303) 792-2192 [FC-PH] Fibre Channel Physical and Signaling Interface (FC-PH) Rev 4.3, ANSI X3.230:1994 [FC-PH-2] Fibre Channel Physical and Signaling Interface (FC-PH- 2) Rev 7.4, ANSI X3.297:1997 [FC-PH-3] Fibre Channel Physical and Signaling Interface (FC-PH- 3) Rev 9.4, ANSI X3.303:1998 [FC-FG] Fibre Channel Generic Requirements (FC-FG) Rev 3.5 ANSI X3.289:1996 [FC-GS-2] Fibre Channel Generic Services (FC-GS-2) Rev 5.2, ANSI NCITS 288 [FC-AL] Fibre Channel Arbitrated Loop (FC-AL) Rev 4.5, ANSI X3.272:1996 [FC-AL-2] Fibre Channel Arbitrated Loop (FC-AL-2) Rev 7.0, NCITS 32:1999 [FC-PLDA] Fibre Channel Private Loop SCSI Direct Attachment (FC LDA), NCITS TR-19:1998 [FC-FLA] Fibre Channel Fabric Loop Attachment (FC-FLA), NCITS TR-20:1998 [FC-TAPE] Fibre Channel Tape and Tape Medium Changers (FC-TAPE), NCITS TR-24:1999 13.3 Relevant RFC Documents [RFC768] User Datagram Protocol [RFC791] Internet Protocol, DARPA Internet Program Protocol Specification Monia Standards Track [Page 66] iFCP Revision 3 July 2001 [RFC2401] Security Architecture for Internet Protocol [RFC2402] IP Authentication Header [RFC2406] Encapsulating Security Protocol (ESP) [RFC2407] The Internet IP Security Domain for ISAKMP [RFC2408] Internet Security Association and Key Management Protocol (ISAKMP) [RFC2409] The Internet Key Exchange (IKE) [RFC2460] Internet Protocol, Version 6 (IPv6) Specification 13.4 Other Reference Documents The Fibre Channel Consultant, A Comprehensive Introduction, Robert W. Kembel, Northwest Learning Associates, ISBN 0-931836-82-6 The Fibre Channel Consultant, Arbitrated Loop, Robert W. Kembel, Connectivity Solutions, a division of Northwest Learning Associates, ISBN 0-931836-84-0 14. Author's Addresses Charles Monia Franco Travostino Rod Mullendore Director, Content Josh Tseng Internetworking Lab, Nishan Systems Victor Firoiu 3850 North First Street San Jose, CA 95134 Nortel Networks Phone: 408-519-3986 3 Federal Street Email: Billerica, MA 01821 cmonia@nishansystems.com Phone: 978-288-7708 Email: travos@nortelnetworks.com Monia Standards Track [Page 67] iFCP Revision 3 July 2001 David Robinson Wayland Jeong Sun Microsystems Troika Networks Senior Staff Engineer Vice President, Hardware M/S UNWK16-301 Engineering 901 San Antonio Road 2829 Townsgate Road Suite Palo Alto, CA 94303-4900 200 Phone: 510-936-2337 Westlake Village, CA 91361 Email: Phone: 805-370-2614 David.Robinson@sun.com Email: wayland@troikanetworks.com Rory Bolt Paul Rutherford Quantum/ATL ADIC Director, System Design Vice President, Technology & 101 Innovation Drive Software Irvine, CA 92612 1143 Willows Road N.E. Phone: 949-856-7760 P.O. Box 97057 Email: rbolt@atlp.com Redmond, WA 98073-9757 Phone: 425-881-8004 Email: paul.rutherford@adic.com Mark Edwards Senior Systems Architect Eurologic Development, Ltd. 4th Floor, Howard House Queens Ave, UK. BS8 1SD Phone: +44 (0)117 930 9600 Email: medwards@eurologic.com Monia Standards Track [Page 68] iFCP Revision 3 July 2001 Appendix A A. iFCP Support for Fibre Channel Link Services For reference purposes, this appendix enumerates all the Fibre Channel link services and the manner in which each shall be processed by an iFCP implementation. The iFCP processing policies are defined in section 7. A.1 Basic Link Services The basic link services are shown in the following table. Basic Link Services Name Description iFCP Policy ---- ----------- ---------- ABTS Abort Sequence Transparent BA_ACC Basic Accept Transparent BA_RJT Basic Reject Transparent NOP No Operation Transparent PRMT Preempted Rejected (Applies to Class 1 only) RMC Remove Connection Rejected (Applies to Class 1 only) A.2 Link Services Processed Transparently The following link service requests and responses MUST be processed transparently as defined in section 7. ELSs Processed Transparently Name Description ---- ----------- ACC Accept ADVC Advise Credit CSR Clock Synchronization Request CSU Clock Synchronization Update ECHO Echo ESTC Estimate Credit ESTS Establish Streaming FACT Fabric Activate Alias_ID FAN Fabric Address Notification FDACT Fabric Deactivate Alias_ID FDISC Discover F_Port Service Parameters Monia Standards Track [Page 69] iFCP Revision 3 July 2001 FLOGI F_Port Login GAID Get Alias_ID LCLM Login Control List Management LINIT Loop Initialize LIRR Link Incident Record Registration LPC Loop Port Control LS_RJT Link Service Reject LSTS Loop Status NACT N_Port Activate Alias_ID NDACT N_Port Deactivate Alias_ID PDISC Discover N_Port Service Parameters PRLI Process Login PRLO Process Logout QoSR Quality of Service Request RCS Read Connection Status RLIR Registered Link Incident Report RNC Report Node Capability RNFT Report Node FC-4 Types RNID Request Node Identification Data RPL Read Port List RPS Read Port Status Block RPSC Report Port Speed Capabilities RSCN Registered State Change Notification RTIN Request Topology Information RTV Read Timeout Value RVCS Read Virtual Circuit Status SBRP Set Bit-error Reporting Parameters SCL Scan Remote Loop SCN State Change Notification SCR State Change Registration TEST Test TPLS Test Process Login State A.3 Augmented Link Services The following extended link services are augmented with additional data and processed by the iFCP implementation as described in the referenced section listed in the table. Augmented Link Services Name Description Section ---- ----------- ------- ABTX Abort Exchange 7.3.1 ADISC Discover Address 7.3.2 Monia Standards Track [Page 70] iFCP Revision 3 July 2001 ADISC Discover Address Accept 7.3.3 ACC FARP- Fibre Channel Address 7.3.4 REPLY Resolution Protocol Reply FARP-REQ Fibre Channel Address 7.3.5 Resolution Protocol Request LOGO N_PORT Logout 7.3.6 PLOGI Port Login 7.3.7 REC Read Exchange Concise 7.3.8 REC ACC Read Exchange Concise Accept 7.3.9 RES Read Exchange Status Block 7.3.10 RES ACC Read Exchange Status Block 7.3.11 Accept RLS Read Link Error Status Block 7.3.12 RRQ Reinstate Recovery Qualifier 7.3.14 RSI Request Sequence Initiative 7.3.15 RSS Read Sequence Status Block 7.3.13 TPRLO Third Party Process Logout 7.3.16 Monia Standards Track [Page 71] iFCP Revision 3 July 2001 Appendix B B. Implementing an IP to FC Router with an iFCP Fabric This informational appendix describes a method for implementing IP- to-FC router functionality in an iFCP gateway. An IP-to-FC router is a two-port device that converts IP datagrams received over a GBE link to FC frames in accordance with RFC 2625 [RFC2625]. This RFC specifies the all the elements needed to internetwork Fibre Channel devices to an IP network and access IP protocol stacks including: a) The Fibre Channel encapsulation of IP datagrams received from gigbit ethernet. b) Support for ARP, the address resolution protocol specified by [RFC826] c) Use of FARP-REQ and FARP-REPLY for address resolution in a Fibre Channel fabric. In the Fibre Channel device domain, the router has appears as a Fibre Channel N_PORT with a standard 24-bit N_PORT I/D. In the IP network domain, the router has an IP address which MUST be different from the gateway's iFCP portal address. / The remainder of this section is TBS / Monia Standards Track [Page 72] iFCP Revision 3 July 2001 Appendix C C. Performance of The Multi-Connection iFCP Session Model This appendix provides a quantitative analysis of the claim that N TCP connections carrying the traffic of all the sessions active between gateways provide significantly higher aggregate average throughput than a single TCP connection carrying the same sessions. The analysis shows that the difference is proportional to the square of the number of TCP sessions, N. This analyses is based on three fundamental assumptions: (i) all the available bandwidth in a link is available to iFCP traffic, (ii) the sender has always data ready to send (as is most likely the case with a backup application), and (iii) the maximum window size at the two TCP ends (i.e., the iFCP gateways) is set to the link nominal capacity multiplied by the round-trip-time (so as to have the highest chances of saturating the link yet without unduly raising buffering requirements at the end nodes). The N^2 factor that emerges from this analysis is essentially due to the way TCP congestion control reacts to packet losses. C.1 Relationship of Throughput to Packet Losses There are several reasons for packet losses: network congestion, link errors and network errors. Network congestion is pervasive in current IP networks, where the only way to control congestion is through dropping packets. Techniques for loss prevention, such as traffic engineering, admission control and bandwidth reservation, are not widely deployed and hence are not a factor in the behavior of existing networks. Even in a perfectly engineered network, link errors occur. Assuming a link error rate equal to that specified for Fibre Channel (10^- 12) and a 10Gb/s link, there is one error every 100 seconds. Network errors also occur with significant frequency in IP networks. Jonathan Stone and Craig Partridge recently reported in Sigcomm 2000 that network errors caught by the TCP checksum occur with significant frequency. Between one packet in 1100 and 1 in 32000 have errors get past the link CRC and are detected by the TCP/IP checksum. [Editor's note: During the 5/31/2001 internal review, the reviewers requested that the editor clarify the wording in the above paragraph regarding the incidence of TCP/IP checksum errors. The requested clarification will be included in a subsequent revision.] TCP throughput is impacted by each packet loss. Following TCP's congestion control algorithm (supported by the Tahoe, Reno, New- Monia Standards Track [Page 73] iFCP Revision 3 July 2001 Reno, and SACK implementations) each packet loss results in the TCP sender's congestion window being reduced to half of its current value, and therefore (assuming constant Round Trip Time), TCP's throughput is halved. After that, the window increases by roughly one packet every two Round Trip Times (assuming the widely-used Delayed-Acknowledgement algorithm). The temporary decrease in TCP's rate translates into a missed opportunity to transmit a given amount of data. As we show in the following Background section, for N storage connections sharing an IP "pipe" of rate E, the amount of data missing the opportunity to be transmitted due to a packet loss is: D(N) = E^2/(N^2)*RTT^2/(256*M) where RTT = Round Trip Time, M = packet size. For example, for a set of N=100 connections totaling E=10Gb/s, RTT=10ms, M=1500B, the data not transmitted in time due to a packet loss is D(N)=2.6MB. For the same set transported over one TCP session, the data not sent in time is D(1)= 26GB, a 10,000 fold increase. The time interval for TCP to recover its sending rate to its initial value after a packet loss is I(N)= 0.833 seconds in the case N TCP connections, and I(1)=83.3seconds in the case of a single TCP connection. Observe that in the latter case, the time to recover its rate, I(1)=83.3s, is of the same order of magnitude as the time between two packet losses due exclusively to a link Bit Error Rate of 10^-12. In other words, a packet loss occurs almost immediately after TCP has recovered its rate. This means that a single TCP connection delivers on average about 3/4 of the required 10Gb/s rate, since 1/4 of the rate is lost during the time the TCP rate is increasing linearly from 1/2 to full rate. (More precisely, the effective rate is 8.27Gb/s because 1/4 of the rate is lost during 83.3s, and the time between two errors is now 120.825s due to a decreased sending rate). By comparison, N TCP connections deliver approximately 9.99979Gb/s (i.e., lost 1/4 of one TCP full rate of 100Mb/s during 0.833s out of a 100s interval). If the impact of TCP checksum errors is also considered, the TCP sending rate is limited to an average of (8M/RTT)sqrt(3/4p), where p is the probability of packet loss (see [Padhye] for details). For M=1500, RTT=10ms and p=1/32000, TCP throughput is about 240Mb/s. For p=1/1100, maximum TCP throughput is 34.4Mb/s. Therefore, to fill a 10Gb/s line, about 42 simultaneous TCP flows are required (in the case where p=1/32000) or 291 TCP flows (in the case where p=1/1100). Practically, for these reasons the iFCP protocol supports combinations of M tuples using N TCP connections, with M, N >= 1, and with an individual tuple using at most one TCP connection (thus M >= N). Monia Standards Track [Page 74] iFCP Revision 3 July 2001 C.2 Background. For a TCP session to sustain a rate of C bits/second, the TCP's maximum congestion window W (measured in number of packets) has to be at least W0=RTT*C/(8*M) where RTT = Round Trip Time in seconds, M = packet size in Bytes. The following analyses assumes W=W0. Later, the problems with the alternative W>W0 are discussed. The time needed by the TCP sender to recover from a single packet loss and have its sending rate reach the previous C value is I = 2*RTT*W/2 = RTT*W = RTT^2*C/(8*M). The total amount of data (in Bytes) missing the opportunity to be transmitted in this time interval I is: D = C/8*I/4 = C^2*RTT^2/(256*M) Consider a set of tuples sharing an IP "pipe" of rate E to be transported in N TCP sessions. Assuming all connections are processed equally, each TCP session sends at a rate of E/N. One packet loss impacts only one TCP session, and thus, the total amount of data missing the opportunity to be transmitted due to a packet loss is D(N) = E^2/(N^2)*RTT^2/(256*M). On the other hand, if the same set of tuples sharing an IP "pipe" of rate E is transported in one TCP session only, the total amount of data losing the opportunity to be transmitted due to a packet loss is D(1) = E^2*RTT^2/(256*M) = D(N)*N^2. The impact of packet losses on the single-TCP solution can be reduced by configuring the maximum congestion window to be larger than the bandwidth*delay product, W>W0. But in this case, only W0 packets can be in transit on the line, while the rest (up to the current window size) need to be stored in a queue at the line's ingress. In order to provide full line rate utilization assuming periodic losses, the maximum congestion window should be at least 2*W0, due to TCP's congestion Monia Standards Track [Page 75] iFCP Revision 3 July 2001 Full Copyright Statement "Copyright (C) The Internet Society, July 2001. All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implmentation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [FC-FS] dpANS X3.XXX-200X, "Fibre Channel Framing and Signaling Interface", Revision 1.2, NCITS Project 1331-D, February, 2001 [FC-SW2] dpANS X3.XXX-2000X, "Fibre Channel Switch Fabric -2 (FC- SW2)", revision 5.2, NCITS Project 1305-D, May, 2001 [FC-GS3] dpANS X3.XXX-200X, "Fibre Channel Generic Services -3 (FC- GS3)", revision 7.01, NCITS Project 1356-D, November, 2000 [FC-FLA] TR-20-199X, "Fibre Channel Fabric Loop Attachment (FC- FLA)", revision 2.7, NCITS Project 1235-D, August, 1997 Monia Standards Track [Page 76] iFCP Revision 3 July 2001 [Kembel] Kembel, R., "Fibre Channel, A Comprehensive Introduction", Northwest Learning Associates Inc., 2000, ISBN 0-931836-84-0 Kembel, R., "The Fibre Channel Consultant, Arbitrated Loop", Robert W. Kembel, Northwest Learning Associates, 2000, ISBN 0-931836-84-0 [ENCAP] Weber, et-al., "FC Frame Encapsulation", draft-ietf-ips- fcencapsulation-01.txt, May, 2001 [RFC2030] Mills, D., RFC 2030, "Simple Network Time Protocol (SNTP)" Version 4, October, 1996 [RFC791] Postel, J., RFC 791, "The Internet Protocol", September, 1981 [00-603] Stephens, G., Warden, G. T11/00-603, "Fractional Bandwidth, Class 2, Class 3", October, 2000 [RFC768] Postel, J., RFC 768, "User Datagram Protocol", August, 1980 [RFC2401] Kent, S., Atkinson, R., RFC 2401, "Security Architecture for the Internet Protocol", November, 1998 [RFC2402] Kent, S., Atkinson, R., RFC 2402, "IP Authentication Header", November, 1998 [RFC2406] Kent, S., Atkinson, R., RFC 2406, "Encapsulating Security Protocol", November, 1998 [RFC2407] Piper, D., RFC 2407, " The Internet IP Security Domain of Interpretation for ISAKMP", November, 1998 [RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J., RFC 2408, "Internet Security Association and Key Management Protocol (ISAKMP)" November 1998 [RFC2409] D. Harkins, D. Carrel, RFC 2409, "The Internet Key Exchange (IKE)", November, 1998 [RFC2460] Deering, S., Hinden, R., RFC 2460 "Internet Protocol, Version 6 (IPv6) Specification", December 1998 [RFC2625] Rajagopal, M, et.al., RFC 2625, "IP and ARP over Fibre Channel", June 1999 [RFC826] Plummer, D.C., RFC 826 "Ethernet Address Resolution Protocol: Or converting network protocol addresses to 48.bit Ethernet address for transmission on Ethernet hardware", November, 1982. Monia Standards Track [Page 77] iFCP Revision 3 July 2001 [Padhye] Padhye, J, Firoiu, D, Kurose, J., "Modeling TCP Reno Performance: A Simple Model and its Empirical Validation." J. Padhye, V. Firoiu, D. Towsley and J. Kurose, IEEE/ACM Transactions on Networking, April 2000. Monia Standards Track [Page 78]