Internet Draft M. Wakeley Agilent Technologies Expires March 2001 September 2000 iSCSI Message Boundary Detection Proposal Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) Agilent Technologies (2000). All Rights Reserved. 1. Abstract The iSCSI working group is currently considering the framing/delimiting of iSCSI messages in a TCP stream. This I-D presents a proposal to discover iSCSI message boundaries using the TCP urgent pointer. Wakeley [Page 1] Internet-Draft iSCSI Message Boundary September 2000 2. Introduction The iSCSI I-D presents an encapsulation of the SCSI protocol onto TCP/IP. This encapsulation is accomplished by sending iSCSI "messages" that are of varying length. Unfortunately, TCP does not have a built-in mechanism for signaling message boundaries at the TCP layer. iSCSI overcomes this obstacle by placing the message length at the beginning of each message (in the iSCSI message "header"). This serves to delineate the end of the current message as well as the beginning of the next message. In situations where IP packets are delivered in-order from the network, framing issues are not an issue (messages are processed one after the other). In the presence of IP packet reordering (mostly due to frames being dropped), it is best to minimize the dependencies between TCP segments, so as to be able to do as much processing of the received out-of-order segments as possible. Such processing ensures that data can be copied to correct buffers the first time, decreasing the need for dedicated reassembly buffers as well as the latency and bandwidth related to extra copies. Another area where it would be extremely helpful to be able to delineate iSCSI messages is when using a protocol analyzer to monitor or debug an iSCSI session. Typically, an analyzer will not be snooping continuously from the time the session is established, and thus it will not be "aligned" with the iSCSI messages. An iSCSI message delimiter would enable the analyzer to discover and decode the iSCSI messages. Unfortunately, when relying solely on the "message length in the iSCSI message" scheme to delineate iSCSI messages, a missing TCP segment that contains an iSCSI message header (with the message length) makes it impossible to find message boundaries in subsequent TCP segments. The missing TCP segment must be received before later segments can be parsed and processed. There are several ways to get around this. * Make each message fixed size * Put synchronization information in the stream at fixed intervals * Use a marker byte to mark message boundaries However, each of these techniques has its drawbacks. Fixed sized messages introduce deadspace for small messages and extra control overhead for large messages. Marker bytes may work well for hardware but the escape sequences make it difficult for software. Putting information at fixed intervals in the stream introduces extra processing. This document proposes usage of the urgent field in the TCP header to delineate iSCSI messages. Wakeley [Page 2] Internet-Draft iSCSI Message Boundary September 2000 3. iSCSI Message Boundary Delineation 3.1. Use the TCP Urgent Pointer The proposal is pretty simple. The first byte of every iSCSI message is marked "urgent" - the TCP urgent pointer will point to the first byte of the iSCSI message in the TCP segment. 3.2. Restrict iSCSI PDU Size When a large iSCSI message is sent, the first TCP segment will contain the iSCSI header, but the remaining TCP segments do not contain any iSCSI framing information. To minimize the amount of buffering required when an iSCSI header is lost, it is recommended that the iSCSI PDU size be restricted to a small value (perhaps a few TCP segments in length). 3.3. Differing Interpretations of Urgent Pointer Unfortunately, there are differing interpretations of whether the Urgent pointer points to the last (only) byte of urgent data (as defined by RFC1122), or the byte after the urgent data (typically BSD implementations) (see section 20.8 in [Stevens]). In order to allow an iSCSI implementation to resolve which interpretation is being used on the data received, the following change is proposed for the iSCSI draft. Define a bit in the first byte of the iSCSI message (the opcode field) that shall always be zero. Likewise, define the same bit in the following byte (typically a "flags" field) that shall always be one. When an iSCSI implementation receives an out of order TCP segment with the Urgent pointer defined, it shall look at byte pointed to by the Urgent pointer. If the bit is clear, the sender is RFC1122 compliant. If the bit is set, the sender has implemented the BSD interpretation, and must "back up" one byte to find the beginning of the iSCSI message. The proposed bit to be used is bit 7. Bit 7 is currently used in the opcode field as the "retry" bit, and used in the "SCSI Command" as a field describing immediate data. The "retry" bit could be moved to bit 5, and the immediate data field moved elsewhere in the flags field. 4. Security considerations A malicious interceptor could modify the urgent pointer information, falsely indicating message boundaries and corrupting the stream. However, such an interceptor could probably manipulate the stream to corrupt it in other ways. In addition, the urgent pointers should only be used to attempt iSCSI message synchronization when a TCP segment is missing, and not during in order packet reception. Wakeley [Page 3] Internet-Draft iSCSI Message Boundary September 2000 5. Authors' Addresses Matt Wakeley 1101 Creekside Ridge Drive Suite 100 Roseville, CA 95661 USA Phone: 916-788-5670 Email: matt_wakeley@agilent.com References [Stevens] W Richard Stevens, "TCP/IP Illustrated, Volume 1," ISBN 0201633469. [TCP] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", RFC 793, September 1981 [Requirements] Braden, R., "Requirements for Internet Hosts -- Communication Layers", RFC 1122, October 1989 Expires February 2001 Wakeley [Page 4]