INTERNET-DRAFT C. Sapuntzakis Cisco Systems June 2000 Expires January 2001 iSCSI/VI/TCP proposal Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other docu- ments at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in pro- gress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) Cisco Systems (2000). All Rights Reserved. Abstract The iSCSI contributors are currently considering the framing/delimiting of iSCSI messages in a TCP stream, especially with an eye towards being able to process segments in a TCP stream out-of-order. Another concern in iSCSI is reducing host overhead, especially for data phase transfers. An RDMA abstraction on top of TCP has been proposed as a solution. VI/TCP addresses both those needs. It provides a generic, reliable Sapuntzakis [Page 1] Internet-Draft iSCSI/VI/TCP proposal 13 July 2000 message and RDMA interface. In addition, VI has other compelling applications, including clustering and filesystem access. As such, it is a good candidate for a generic mechanism to place on a net- work interface. 1. Introduction In situations where there is totally in-order delivery of IP pack- ets from the network, framing issues are largely a matter of imple- mentation complexity. In the presence of reordering, however, it is best to minimize the dependencies between packets, so as to be able to do as much pro- cessing up-front as possible. Such processing ensures that data can be copied to correct buffers the first time, decreasing the need for dedicated reassembly buffers as well as the latency and bandwidth related to extra copies. Decreasing the need for reassembly buffers is crucial for high- speed WAN links. On a 10 gigabit connection with a 200 millisecond round-trip time, hundreds of megabytes of re-order buffering may be necessary. The VI/TCP protocol, when used optimally with TCP, adds a self- describing header to each segment. For data transfers, this header is sufficient to place the data in the correct receiving buffer. For messages, the header is sufficient to place the data in the correct places in a message queue. 2. Changes necessary to iSCSI to support VI * The iSCSI command and ready-to-transmit PDUs must be changed to pass a 64-bit virtual address and 32-bit memory handle. * VI has a maximum message size, which may be less than the iSCSI maximum. iSCSI PDUs over the size of VI's maximum message size will need to be divided across VI messages. Problems with MTU should only come into play on unsolicited writes. However, an alternate scheme for unsolicited writes is possible with VI. The initiator could be issued a memory region (or multiple memory regions) for immediate writes. * Data transfers will no longer happen at the iSCSI layer in iSCSI Sapuntzakis [Page 2] Internet-Draft iSCSI/VI/TCP proposal 13 July 2000 PDUs, but instead be initiated as RDMAs. 3. Q & A Does using VI/TCP imply that I will have to use the VI software interface? No, the VI/TCP protocol is entirely self-contained and does not mandate any software interface. Will adopting the VI/TCP protocol place any constraints on iSCSI? Recovery from failed TCP connections: Since RDMA is at a lower layer than iSCSI, recovery in the middle of an iSCSI data phase would be harder with VI/TCP. However, iSCSI is not currently proposing to recover starting in the middle of a data phase. Will adding VI hurt performance of software-only implementations? No. It may help performance by using common RDMA code for many applications. Sapuntzakis [Page 3] Internet-Draft iSCSI/VI/TCP proposal 13 July 2000 4. Author's Address (send comments to:) Constantine Sapuntzakis Cisco Systems, Inc. 170 W. Tasman Drive San Jose, CA 95134 USA Phone: +1 408 525 5497 Email: csapuntz@cisco.com 5. References [TCP] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", RFC 793, September 1981 [VI] Virtual Interface Architecture Specification version 1.0, http://www.viarch.org/ [VI/TCP] VI/TCP (Internet VI), draft-dicecco-vitcp-00.txt Expires January 2001 Sapuntzakis [Page 4]