IPFC Working Group M. Rajagopal, R. Bhagwat, W. Rickard INTERNET-DRAFT Gadzoox Networks Elizabeth Rodriguez (Expires November 15, 2000) Lucent Technologies Fibre Channel Over IP (FCIP) Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC 2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as Reference material or to cite them other than as ``work in progress''. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/lid-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html 1. Abstract Fibre Channel(FC) is a dominant technology used in Storage Area Networks(SAN). The purpose of this draft is to specify a standard way of encapsulating FC frames over IP and to describe mechanisms that allow islands of FC SANs to be interconnected over IP-based networks running over very reliable data links. FC over IP relies on IP-based network services to provide the connectivity between the SAN islands over LANs, MANs, or WANs. The FC over IP specification is independent of the link level transport protocol such as Gigabit Ethernet, SONET, ATM, or DWDM, used for carrying the IP packets. This specification treats all classes of FC frames like datagrams. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [2]. 3. Motivation and Objectives Fibre Channel (FC) is a gigabit speed networking technology primarily used for Storage Area Networking (SAN). FC is standardized under American National Standard for Information Systems of the National Rajagopal, et al. [Page 1] Internet-Draft Fibre Channel over IP March 2000 Committee for Information Technology Standards (ANSI-NCITS) and has specified a number of documents describing its protocols, operations, and services [13]. The motivation behind connecting remote sites include disk or tape backup and live mirroring, or simply distance extension between two FC devices or FC Switch clusters (SAN islands). A fundamental assumption made in this specification is that the FC encapsulated IP packets are carried over very reliable data links and may span LANs, MANs, and WANs. This main objectives of this document are to: 1) specify the IPv4 encapsulation, mapping and routing of FC frames 2) apply the mechanism described in 1) to a FC backbone network or generally between any two FC devices The goal of this specification is to utilize the existing IP suite of protocols and address any FC concerns such as security, data integrity (loss), and performance when running over IP-based networks. 5. FCIP Protocol 5.1 FCIP Device In this specification, the term FCIP device generally refers to any device that encapsulates FC frames into IP packets and decapsulates IP packets to regenerate FC frames. Note: In an actual implementation, the FCIP device may be a stand- alone box or integrated with an FC device such as a FC Backbone Switch or integrated with any IP device such as an IP Switch or an IP Router. The FCIP device is a transparent translation point. The IP network is not aware of the FC payload that it is carrying. Likewise, the FC Fabric and the FC end nodes are unaware of the IP-based transport. 5.2 Protocol The FCIP protocol specifies the IPv4 encapsulation, mapping and routing of FC frames and applies these mechanisms to a FC backbone network or generally between any two FC devices. The FCIP protocol is summarized below: 1. All FCIP protocol devices are peers and communicate using IP. Each FCIP device behaves like an IP host from the perspective of the IP-based network. That is, these devices do not perform IP routing or IP switching but simply forward FC frames. Rajagopal, et al. [Page 2] Internet-Draft Fibre Channel over IP March 2000 2. There is no requirement for an FCIP device to establish a login with a peer before communication begins. However, FCIP devices may authenticate the IP packet before accepting it using the IPSec protocols. Each IPv4 datagram is treated independently and a FCIP device receiver simply listens to the Protocol value (Fibre Channel) contained in the IPv4 header. 3. Each FCIP device may be statically or dynamically configured with a list of IP addresses corresponding to all the participating FCIP devices. It is outside the scope of this specification to describe any dynamic scheme for configuring the FCIP device with an IP address or the list of IP addresses of other participating FCIP devices. 4. The reachable FC addresses behind each FCIP device and its IP address association can be statically configured or dynamically learnt from any FC layer routing protocol exchanged between these devices. In the case when the FCIP device is a Border Switch, the DMP routing protocol can provide this information. Routing in the IP plane and the FC plane are largely independent. The exact path (route) taken by the IP packet follows the normal procedures of any IP packet. From the perspective of the FCIP devices this communication is between only two FCIP for any given packet. 5. A FCIP device may send FC encapsulated IP packets to more than one FCIP device. However, these are treated as separate instances and are not correlated in any way in the FCIP Protocol device. The FCIP device routes its packets based on the 3-byte FC Destination ID (D_ID) address contained in each FC frame. 6. An IP packet may make use of the IPSec protocols to provide secure communications across the IP-based network. 7. Any reordering of data link frames due to MTU fragmentation will be recovered in accordance with a normal IP host behavior. Any reordering of FC frames due to IP packet reordering will be recovered at the FC end nodes. 8. FCIP assumes that error recovery due to any data loss of IP packets will be done at the FC end nodes. FCIP is expected to run on very reliable data links making the probability of data loss due to line Bit Error Rates extremely small and no worse than that of a FC optic link. Note: If the underlying data link is unreliable, then use of an upper layer protocol such as TCP is suggested. However, it is Rajagopal, et al. [Page 3] Internet-Draft Fibre Channel over IP March 2000 beyond the scope of this draft to discuss any such error recovery and retransmission scheme. 9. IPv4 packets shall indicate the use of the Premium Service in the DSCP bits in the IPv4 header. 6. FCIP Encapsulation 6.1 FC Frame Format All FC frames have a standard format much like LAN 802.x protocols. However, the exact size of each frame varies depending on the size of the variable fields. The size of the variable field ranges from 0 to 2112-bytes as shown in the FC Frame Format in Fig. 1 resulting in the minimum size FC Frame of 36 bytes and the maximum size FC frame of 2148 bytes. +------+--------+-----------+----//-------+------+------+ | SOF |Frame |Optional | Frame | CRC | EOF | | (4B) |Header |Header | Payload | (4B) | (4B) | | |(24B) |<----------------------->| | | | | | Data Field = (0-2112B) | | | +------+--------+-----------+----//-------+------+------+ Fig. 1 FC Frame Format SOF and EOF Delimiters: On a FC link, SOF and EOF are called Ordered Sets and are sent as special out-of-band words constructed from the 10-bit comma character (K28.5) followed by 3 additional 10-bit data characters. On a non-Fibre Channel link the Start of Frame (SOF) and End of Frame (EOF) delimiters are both byte-encoded and 4-bytes long. On a FC link the SOF delimiter serves to identify the beginning of a frame and prepares the receiver for frame reception. The correct SOF must be used that corresponds to the frame's Class of Service, position within a sequence and in some cases whether connection is established or not. The EOF delimiter identifies the end of the frame. It also identifies the final frame of a sequence. In connection-oriented classes of service, it is used to end the connection. Besides the above uses, it also serves to force the running disparity to negative. It is therefore important to preserve the information conveyed by the delimiters across the IP-based network, so that the receiving FCIP device can correctly construct the FC frame in its original SOF and EOF format before forwarding it to its ultimate FC destination on the FC link. Start of Frame (SOF) and End of Frame (EOF) byte- encodings are defined in Annex A. Although, the SOF and EOF codes are 32-bits,the Rajagopal, et al. [Page 4] Internet-Draft Fibre Channel over IP March 2000 format makes use of a single-byte to represent each FC Ordered Set. Frame Header: The Frame Header is 24-bytes long and has several fields that are associated with the identification and control of the payload. Current FC Standards allow up to 3 Optional Header fields [4]: - Network_Header (16-bytes) - Association_Header (32-bytes) - Device_Header (up to 64-bytes). Frame Payload: The FC Frame Payload is transparent to the FCIP device. An FC application level payload is called an Information Unit at the FC-4 Level. This is mapped into the Frame Payload of the FC Frame. A large Information Unit is segmented using a structure consisting of FC Sequences. Typically, a Sequence consists of more than one FC frame. FCIP does not maintain any state information regarding the relationship of frames within a FC Sequence. CRC: The CRC is 4-bytes long and uses the same 32-bit polynomial used in FDDI and is specified in ANSI X3.139 Fiber Distributed Data Interface. Note: When FC frames are encapsulated into IP packets, the CRC is untouched. 6.2 FC Frame Mapping to IP Packet Fig.2 shows the mapping of the FC frame into an IPv4 Packet. The FC to IP mapping (and reverse) mapping is one-to-one since the maximum size of the encapsulated FC Frame along with the header fields does not exceed 2148 bytes. The minimum size FC Frame is 36 bytes resulting in a maximally minimum IP MTU size of 96 bytes. (The Maximally minimum MTU size is the IP packet with the minimum size payload and the maximum size IP headers). The maximum size FC frame is 2148 bytes resulting in an (nominal) IP packet size of 2168 bytes. Fig.2 shows the format of the IPv4 packet with the standard 20-byte fixed header and a 40-byte optional header. For the case of the maximum size payload of 2148 bytes, the maximum IPv4 packet size is 2208 bytes. The maximum size FC frame can cause the IP packet to be fragmented when the data link cannot support this MTU size. When an IP packet is fragmented, required parts of the header must be copied by all fragments and the option field may or may not be copied. Rajagopal, et al. [Page 5] Internet-Draft Fibre Channel over IP March 2000 +---------- -+---------------+-------------+ | IP Header | IP Opt. Header| FC Frame | | (20 bytes) | (40 bytes | (2148 bytes | | | Max) | Max) | +------------+---------------+-------------+ Fig. 2 Format of an IPv4 Payload carrying FC If IPSec is used for security it introduces its own headers and the IP packet size increase depends on the exact mode of IPSec usage (AH versus ESP, Tunnel versus Transport). However, this additional increase in the IP packet size due to IPSec headers is relatively small (see [8], [9], [10]), and the maximum size IP packet will remain within its maximum size of 65535 bytes. Adding, IPSec header may in some cases may lead to fragmentation if it exceeds the data link MTU. IP Header Field Setting: DSCP (6 bits): The Differentiated Service Code Points (DSCP) [6] shall be set to correspond to the Premium Service. This service provides "Expedited Forwarding" at each IP hop (Per Hop Behavior (PHB)). Protocol (8 bits): This 8-bit field defines the higher level protocol that uses the service of the IP layer. In this case, this is set to the Fibre Channel Protocol Value 133 defined in [12]. Source IP Address (32 bits): This is the IP address of the ingress FCIP device that is transmitting the FC encapsulated IP packet. Destination IP Address (32 bits): This the IP address of the egress FCIP device that is receiving the FC encapsulated IP packet. FCIP specification treats all classes of FC frames as datagrams. There will be no F_BSY or F_RJT sent if a Class 2 frame is lost while in transit within the IP network. FCIP may not be suitable for transport of Class 1 traffic since these frames are treated the same way as any Class 2 or 3 frame. 6.3 Fibre Channel Bit and Byte Ordering Fibre Channel's FC-1 Level defines the method used to encode data prior to transmission and subsequently decode the data upon reception. The method encodes 8-bit bytes into 10-bit transmission characters to improve the transmission characteristics of the serial data stream. In Fibre Channel data fields are aligned on word boundaries. A word in FC is defined as 4 bytes or 32 bits. The resulting transmission word after the 8-bit to 10-bit encoding consists of 40 bits. Data words or Ordered Sets (special FC-2 Level control words) from the FC-2 Level map to the FC-1 Level with no change in order and Rajagopal, et al. [Page 6] Internet-Draft Fibre Channel over IP March 2000 the bytes in the word are transmitted in the Most Significant Byte first to Least Significant Byte order. The transmission order of bits within each byte is the Least Significant Bit to the Most Significant Bit. 7. FCIP Network 7.1 FC Backbone Switches FC Standards [3] describe the operation and interaction of FC Switches. Two distinct levels of switch interconnections are specified. Autonomous Regions (AR) are defined to allow clusters of FC Switches to be connected across a backbone network called a DMP- backbone. An AR is administratively defined with each AR encompassing one or more FC Address Domains. The DMP-backbone network is formed from one or more Backbone Switches (BSW) that run the DMP routing and switch control protocol on FC links. DMP is based on OSPF and the DMP backbone may consist of an arbitrary mesh network. A BSW may communicate with multiple neighbors. As specified in [3], native FC frames traverse the DMP backbone between DMP neighbors on FC links. DMP Routing Protocol messages are exchanged between BSWs on this backbone. An example network consisting of 4 ARs and a DMP FC backbone consisting of 3 links is given in Fig. 1. There is no restriction in adding other links to this network as needed. The connection between BSWs below may in fact form a fully connected mesh. _______ _______ | | | | | AR #1 |_____ _____| AR #4 | |_______| | | |_______| ___|___ ___|___ | BSW 1 |---------------------| BSW 4 | |_______| |_______| ___|___ _______ | BSW 2 |---------------------| BSW 3 | |_______| |_______| ___ ___ | | _______ | | | | | | | AR #2 |----- -----| AR #3 | |_______| |_______| Note: BSW 1 knows it is connected to BSWs 2 and 4; BSW 2 knows it is connected to BSWs 1 and 3; BSW 4 knows it is connected to BSWs 1. Fig. 1 Example Network showing DMP Backbone Switching Architecture Rajagopal, et al. [Page 7] Internet-Draft Fibre Channel over IP March 2000 An FCIP device provides a single logical interface to the DMP protocol connecting multiple DMP neighbors on the IP network. From the DMP routing point of view, the connection to each neighbor on the IP network is treated as a separate logical FC link. In FCIP, the native FC frames are first encapsulated in IP packets which then traverse the IP-based network. The IP network provides a new transport path for each emulated DMP FC link. The IP network itself may consist of any number of hops between two FCIP devices. Also, the route taken by the IP packet between any two FCIP devices is dictated by the normal IP routing. A functional and logical diagram of an IP-based DMP backbone for the example network given in Fig. 1 is shown in Fig. 2. In this figure, each BSW is logically connected to other BSWs. _______ _______ | | | | | AR #1 |--- | AR #4 | |_______| | ______ ________ ______ |_______| __|_ __ | | | | | | ___|___ | BSW 1 |---| FCIP |--| IP |--| FCIP |--| BSW 4 | |_______| |______| | Network| |______| |_______| | | -------- ______ | | ______ ______ | | | | | | _______ | BSW 2|---| FCIP |-----| |---| FCIP |---| BSW 3 | |______| |______| |______| |_______| ________ | ___|___ | | | | | | AR #2 |__| | AR #3 | |________| |_______| Fig. 2 Example Network showing an IP-based FC Backbone Switching Architecture The IP-based network has transformed the DMP backbone into a fully connected network. From the perspective of each BSW all remote BSWs therefore appear to be neighbors. The DMP routing protocol computation would make the IP based network topology appear as a fully connected mesh. The DMP routing protocol exchanges between BSWs occur transparently to the FCIP devices. Encapsulated FC frames are routed on the IP network according to the normal IP routing procedures. In this mode, the DMP routing protocol lays over the IP network and has no knowledge of the underlying IP protocol and IP routing or the underlying technology that carries the IP datagram. This concept is shown in Fig.3 Rajagopal, et al. [Page 8] Internet-Draft Fibre Channel over IP March 2000 ________ _______ | AR #1 | | AR #2 | | |-- | | |________| | ______ ________ _____ |_______| __|___ | | | | | | ____|__ | BSW 1|---| FCIP |--| IP |--|FCIP |--| BSW 2 | | | | | | Network| | | | | |______| |______| |________| |_____| |_______| <--------------> IP Routing <----------------------------------> DMP Routing Plane Note: IP Network routing may consist of multiple paths 7.2 FC Device The protocol encapsulation and mapping of the FC frame described in earlier sections applies equally to any pair of FC device (e.g., Server-to-server) wishing to tunnel FC frames across an IP-based network. Any FC routing protocol exchanges may still occur transparent to the FCIP devices. 8. Security Considerations For Virtual Private Networks , both authentication and encryption are generally desired, because it is important both to (1) assure that unauthorized users do not penetrate the virtual private network and (2) assure that eavesdroppers on the network cannot read messages sent over the network. IPSec provides 3 main facilities: an authentication-only function, referred to as Authentication Header (AH), a combined authentication/encryption function called Encapsulating Security Payload (ESP), and a key exchange function. Because both features are generally desirable, ESP may be more suitable than AH. The key exchange function allows for manual exchange of keys as well as an automated scheme. The IPSec specifications described in [8], [9], [10], and [11] covers these topics. It is beyond the scope of this document to discuss specific use of the IPSec protocols. Note: Use of IPSec protocol is optional. 9. Data Integrity Considerations Loss: Recovery from data loss due to IP datagram loss is made at the end FC nodes. It is expected that such data losses are rare because the mechanism assumes extremely reliable data links. Rajagopal, et al. [Page 9] Internet-Draft Fibre Channel over IP March 2000 Fragmentation: IP packets as noted earlier can exceed the standard maximum Ethernet frame size of 1526 bytes. Any reordering caused as a result of fragmentation is recovered according to normal procedures at IP hosts. Ordering: FC Over IP specification treats all Classes of FC frames alike and treats each FC frame like a datagram. FCIP specification does not provide any support to maintain any ordering relationships that may exist between FC Frames. In FC Class 2 and 3 Service, the physical (temporal) ordering of the frames as it arrives at a destination can be different from that of the order sent because of traversing through a FC Network. FC frames in this sense are no different from IP datagrams. FCIP protocol does not provide any support to maintain any ordering relationships that may exist between frames related to a Sequence. FC Class 1 service requires that frames be delivered in the same order as transmitted. Since the FCIP protocol does not treat Class 1 Frames differently, it does not provide support to ensure that these frames are in order. 10. Performance Considerations Mapping the IP header DSCP bits to correspond to a Premium Service provides a preferred service at each IP Router Per Hop Behavior (PHB) [6]. Since FCIP protocol makes use of the layer 3 IP protocol rather than the layer 4 TCP, minimal buffering requirements are imposed on the FCIP device. However, this also means that no reliable transmission in the sense of retransmissions are supported. This aspect is important when engineering the data links between the FCIP devices. Note: We expect that technology advances in optics now have the ability to provide very large bandwidth links with very low error rates. Hence the need for a Layer 4 Transport protocol seems unnecessary. In the rare event, when an IP datagram is dropped (corrupted or due to congestion), then the FC end nodes are designed to recover from this situation. The FCIP protocol does not crack the FC Frame (except for attaching the correct byte-encoded SOF and EOF) nor does it do any FC payload processing. This allows any FC traffic to be tunneled across at high throughput rates. The case where there is no data link fragmentation, each FC frame which has a one to one mapping to an IP datagram also has a one-to- one mapping to a data link frame. This has the tendency to further Rajagopal, et al. [Page 10] Internet-Draft Fibre Channel over IP March 2000 improve the throughput performance. Note: Class 1 FC traffic expects a dedicated bandwidth. This specification does not address this requirement. 11. Flow Control FCIP does not provide any flow control support at the IP level. FC credit mechanism provides the required flow control at a higher level between switches. FCIP may be subject to data link level flow control when used. 12. References: [1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [3] NCITS 321-200x (ANSI) T11/Project 1305-D/Rev4.3 "Fibre Channel Switch-Fabric-2", March 2000 (www.t11.org) [4] Fibre Channel Physical and Signaling Interface -3 (FC-PH-3), Rev. 9.3, ANSI X3.xxx-199x [5] The Fibre Channel Consultant: A Comprehensive Introduction, "Robert W. Kembel", Northwest Learning Associates, 1998 [6] Nichols, K., Blake, S., Baker, F. and D. Black, " Definition of the Differentiated Services Field (DS Field) in the IPv4 and Ipv6 Headers", RFC 2474, December 1998. [7] NCITST11/Project 1238-D/Rev4.6 "Fibre Channel Backbone", April 17 2000 (www.t11.org) [8] Kent, S. and Atkinson, R., "Security Architecture for the Internet Protocol", RFC 2401, Nov 1998 [9] Kent, S. and Atkinson, R., "IP Authentication Header", RFC 2402, Nov 1998 [10] Kent, S. and Atkinson, R., "IP Encapsulating Security Payload (ESP)", RFC 2406, Nov 1998 [11] Maughan, D. et all, "Internet Security Association and Key Management Protocol (ISAKMP)", RFC 2408, Nov 1998 [12] http://www.isi.edu/in-notes/iana/assignments/protocol-numbers [13] http://www.t11.org 13. Acknowledgments Rajagopal, et al. [Page 11] Internet-Draft Fibre Channel over IP March 2000 14. Authors' Addresses Murali Rajagopal Gadzoox Networks, Inc. 16281 Laguna Canyon Road, Suite 100 Irvine, CA 92618 Phone: +1 949 789 4646 Fax: +1 949 453 1271 Email: murali@gadzoox.com Raj Bhagwat Gadzoox Networks, Inc. 16281 Laguna Canyon Road, Suite 100 Irvine, CA 92618 Phone: +1 949 789 4634 Fax: +1 949 453 1271 Email: raj@gadzoox.com Wayne Rickard Gadzoox Networks, Inc. 16281 Laguna Canyon Road, Suite 100 Irvine, CA 92618 Phone: +1 949 789 4604 Fax: +1 949 453 1271 Email: wayne@gadzoox.com Elizabeth G. Rodriguez Lucent Technologies 1202 Richardson Drive, Suite 210 Richardson, TX 75080 Phone: 972-231-0672 Email: egrodriguez@lucent.com Rajagopal, et al. [Page 12] Internet-Draft Fibre Channel over IP March 2000 APPENDIX A: Fibre Channel EOF and SOF Encodings A.1 Ordered Sets On a FC link, Ordered Sets (OS) are sent as special out-of-band words constructed of the 10-bit comma character (K28.5) followed by 3 additional 10-bit data characters. The Ordered Sets defined by FC include the Frame Delimiter, Start of Frame (SOF) and End of Frame (EOF), and other Primitive Signals. When FC frames are encapsulated in an IP packet, the Byte-encoded frame format is used. The Byte-encoded frame format uses 32-bit OS Code Words to represent valid FC frame delimiter. This format uses a single-byte OS Code to represent each FC Ordered Set. FC Over IP makes use of the OS Codes defined in Annex A of [7] for the frame delimiters. SOF and EOF codes defined in the figures (see below) in this Annex are inserted into the FC frame. Primitive Signals and Primitive Sequences are stripped at the FCIP boundary. The frame delimiters are identified by their position. An encapsulated Byte-encoded frame must use the corresponding 32-bit OS Code Word as the first and last words in the encapsulated PDU. FC frame delimiters shall be encoded in the format shown in Table below. Table 1. Frame Delimiter Format +---+----------------+----------------+----------------+--------------+ |Wrd| <31:24> | <23:16> | <15:08> | <07:00> | +---+----------------+----------------+----------------+--------------+ |0 | OS Code | Reserved | +---+----------------+----------------+----------------+--------------+ A.2 Encoded FC Frame Delimiters The SOF OS-codes are a single byte encoding of the SOF Ordered Set. The first word in an encapsulated Byte-encoded FC frame shall map the SOF Ordered Set to the corresponding 32-bit OS Code Word. The EOF OS-codes are a single byte encoding of the EOF Ordered Set. The last word in an encapsulated Byte-encoded FC frame shall map the EOF Ordered Set to the corresponding 32-bit OS Code Word. +-----------------+----------------+ | OS-Code | Delimiter Name | | (hex) | | +-----------------+----------------+ | 0x28 | SOFf | +-----------------+----------------+ Rajagopal, et al. [Page 13] Internet-Draft Fibre Channel over IP March 2000 | 0x3F | SOFc1 | +-----------------+----------------+ | 0x2F | SOFi1 | +-----------------+----------------+ | 0x37 | SOFn1 | +-----------------+----------------+ | 0x3D | SOFc2 | +-----------------+----------------+ | 0x2D | SOFi2 | +-----------------+----------------+ | 0x35 | SOFn2 | +-----------------+----------------+ | 0x3E | SOFc3 | +-----------------+----------------+ | 0x2E | SOFi3 | +-----------------+----------------+ | 0x36 | SOFn3 | +-----------------+----------------+ | 0x39 | SOFc4 | +-----------------+----------------+ | 0x29 | SOFi4 | +-----------------+----------------+ | 0x31 | SOFn4 | +-----------------+----------------+ | 0x38 | SOFcf | +-----------------+----------------+ | 0x30 | SOFnf | +-----------------+----------------+ | 0x41 | EOFn | +-----------------+----------------+ | 0x42 | EOFt | +-----------------+----------------+ | 0x46 | EOFdt | +-----------------+----------------+ | 0x44 | EOFrt | +-----------------+----------------+ | 0x49 | EOFni | +-----------------+----------------+ | 0x4E | EOFdti | +-----------------+----------------+ | 0x4F | EOFrti | +-----------------+----------------+ | 0x50 | EOFa | +-----------------+----------------+ Full Copyright Statement Copyright (C) The Internet Society (1999). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction Rajagopal, et al. [Page 14] Internet-Draft Fibre Channel over IP March 2000 of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. [draft-ietf-ipfc-fcoverip-01.txt] [This INTERNET DRAFT expires on November 15, 2000] Rajagopal, et al. [Page 15]