IP Storage Working Group M. Krueger R. Haagens Internet Draft Hewlett-Packard Corporation Category: Informational C. Sapuntzakis M. Bakke Cisco Systems Document: draft-ietf-ips-iscsi-reqmts-00.txt November 2000 iSCSI Requirements and Design Considerations Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract The IP Storage Working group is chartered with developing a protocol to transport the Small Computer Systems Interface (SCSI) protocol over the internet. The iSCSI protocol will define a mapping of SCSI transport protocol over TCP/IP so that SCSI storage controllers (principally disk and tape arrays and libraries) can be attached to IP networks, notably Gigabit Ethernet (GbE) and 10 Gigabit Ethernet (10 GbE). This document specifies the requirements the iSCSI protocol should satisfy and the design considerations guiding the iSCSI protocol development effort. In the interest of timely adoption of the iSCSI protocol, this group has chosen to work with the existing SCSI architecture and commands, and the existing TCP/IP transport layer. Krueger Informational û Expires May 2001 1 ISCSI Reqmnts and Design Considerations Nov. 2000 Both these protocols are widely-deployed and well-understood. The thought is that using these mature protocols will entail a minimum of new invention, the most rapid possible adoption, and the greatest compatibility with Internet architecture, protocols, and equipment. The iSCSI protocol is a mapping of SCSI to TCP, and constitutes a "SCSI transport" as defined by the ANSI T10 document SCSI SAM-2 document [SAM2, p. 3, "Transport Protocols"]. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. Paragraphs marked with [R] or [D] are still undergoing development. 3. Definitions Certain definitions are offered here, with references to the original document where applicable, in order to clarify the discussion of requirements. Definitions without references are the work of the authors and reviewers of this document. Logical Unit (LU): A target-resident entity that implements a device model and executes SCSI commands sent by an application client [SAM- 2, sec. 3.1.50, p. 7]. Logical Unit Number (LUN): A 64-bit identifier for a logical unit [SAM-2, sec. 3.1.52, p. 7]. SCSI Device: A device that is connected to a service delivery subsystem and supports an SCSI application protocol [SAM-2, sec. 3.1.78, p. 9]. Service Delivery Port (SDP): A device-resident interface used by the application client, device server, or task manager to enter and retrieve requests and responses from the service delivery subsystem. Synonymous with port (SAM-2 sec. 3.1.61) [SAM-2, sec. 3.1.89, p. 9]. Target: An SCSI device that receives SCSI command and directs such commands to one or more logical units for execution [SAM-2 sec. 3.1.97, p. 10]. Task: An object within the logical unit representing the work associated with a command or a group of linked commands [SAM-2, sec. 3.1.98, p. 10]. Transaction: A cooperative interaction between two objects, involving the exchange of information or the execution of some Krueger Informational û Exp. May 2001 2 ISCSI Reqmnts and Design Considerations Nov. 2000 service by one object on behalf of the other [SAM-2, sec. 3.1.109, p. 10]. [A transaction seems to be a smaller unit than a task.] 4. iSCSI Design Considerations 4.1. General Discussion The iSCSI standard SHALL specify how SCSI volume/block-oriented devices interact when attached to IP networks. The SCSI-3 command sets (defined by the ANSI NCITS T10 committee) will be mapped to TCP. TCP has been chosen as the transport protocol because it is well defined, well respected, and widely implemented in the internet community. In addition, the TCP transport provides the necessary congestion management behavior necessary to be a "good internet citizen". Traditionally, volume/block-oriented storage controllers (e.g., disk array controllers, tape library controllers) have supported the SCSI-3 protocol, and have been attached to computers through the SCSI parallel bus or through Fibre Channel. File-oriented storage controllers have supported the NFS and/or CIFS protocols, and have been attached directly to IP networks such as Ethernet. The IP infrastructure offers compelling advantages for volume/block- oriented storage attachment compared to current approaches. It offers the opportunity to take advantage of the cost/performance benefits provided by competition in the internet marketplace. This reduces the cost of storage infrastructure by: -- Increasing performance (market driven by networking demand) -- Offers richer array of management, security and QoS solutions -- Economies arising from the need to install and operate only single type of network In addition, mapping SCSI over IP provides: -- Extended distance ranges -- Connectivity to "carrier class" services that support IP The following applications for iSCSI are contemplated: -- Local storage access, consolidation, clustering and pooling (as in the data center) -- Client access to remote storage (ex. a "storage service provider") -- Local and remote synchronous and asynchronous mirroring between storage controllers -- Local and remote backup and recovery iSCSI must support the following topologies: Krueger Informational û Exp. May 2001 3 ISCSI Reqmnts and Design Considerations Nov. 2000 -- Point-to-point direct connections -- Dedicated storage LAN, consisting of one or more LAN segments -- Shared LAN, carrying a mix of traditional LAN traffic plus storage traffic -- LAN-to-WAN extension using IP routers or carrier-provided "IP Datatone" -- Private networks and the public Internet Local-area storage networks will be built using Ethernet LAN switches. These networks may be dedicated to storage, or shared with traditional Ethernet uses, as determined by cost, performance, administration, and security considerations. In the local area, TCP's adaptive retransmission timers will provide for automatic and rapid error detection and recovery, compared to alternative technologies. IP LAN-WAN routers will be used to extend the IP storage network to the wide area, permitting remote disk access (as for a storage utility), synchronous and asynchronous remote mirroring, and remote backup and restore (as for tape vaulting). In the WAN, TCP end-to- end will avoid the need for specialized equipment for protocol conversion, ensure data reliability, cope with network congestion, and automatically adapt retransmission strategies to WAN delays. The full realization of iSCSI will involve the following elements: (1) Completion of Requirements (this document) and Specification documents; (2) Development of Ethernet storage NICs and related driver and protocol software; [NOTE: high-speed applications of iSCSI are expected to require significant portions of the iSCSI/TCP/IP implementation in hardware to achieve the necessary throughput.] (3) Development of compatible storage controllers; and (4) The likely development of translating gateways to provide connectivity between the Ethernet storage network and the Fibre Channel and/or parallel-bus SCSI domains. Products will initially be offered for Gigabit Ethernet attachment, with rapid migration to 10 GbE. For performance competitive with alternative SCSI transports, it will be necessary to implement the performance path of the full protocol stack in hardware. These new storage NICs will perform full-stack processing of a complete SCSI task, analogous to today's SCSI and Fibre Channel HBAs. They typically also will support all host protocols that use TCP, including NFS, CIFS and HTTP. A key goal is not to require modifications to the current IP and Ethernet infrastructure to support storage traffic over TCP. Nevertheless, the performance and security requirements of storage will create opportunities for improvement in security protocols and QoS implementations. The addition of storage traffic to local- and wide-area internets (and even to the public Internet) may introduce Krueger Informational û Exp. May 2001 4 ISCSI Reqmnts and Design Considerations Nov. 2000 increased requirements for traffic monitoring and engineering in those environments. It is contemplated that many organizations initially will choose to operate storage networks based on iSCSI that are independent of (isolated from) their current data networks except for secure routing of storage management traffic. These organizations will benefit from the high performance/cost of IP equipment and a unified management architecture, compared to alternative means of building storage networks. As security and QoS evolve, it will become more reasonable to build combined networks with shared infrastructure; nevertheless, it is likely that sophisticated users will choose to keep their storage subnetworks isolated, for the best control of security and QoS. The charter of the IETF IP Storage Working Group (IPSWG) describes the broad goal of mapping SCSI to IP. Within that broad charter, many transport alternatives may be considered. Our initial work focuses on TCP, and this requirements document is restricted to that domain of interest. At the current time, the working group does not seek a more generic requirements statement that would justify the choice of TCP (or another protocol) as transport, since the merits of using TCP are readily evident to the working group participants. 4.2. Framing Framing refers to the addition of information in a header, or the data stream to allow implementations to locate the boundaries of an iSCSI protocol data unit (PDU). There are two technical requirements driving framing: interfacing needs, and accelerated processing needs. A framing solution that addresses the "interfacing needs" of the iSCSI protocol will facilitate the implementation of a message-based upper layer protocol (SCSI) on top of an underlying byte streaming protocol (TCP). Since TCP is a reliable transport, this can be accomplished by including a length field in the iSCSI header. That assumes that the receiver will parse from the beginning of the stream, and never make a mistake (lose alignment on packet headers). The other technical requirement for framing, "accelerated processing", stems from the need to handle increasingly higher data rates in the physical media interface. Two needs arise from higher data rates - (1) LAN environment - NIC vendors seek ways to provide "0 copy" methods of moving data directly from the wire into application buffers. (2) WAN environment- the emergence of high bandwidth, high latency, low bit error rate physical media places huge buffer requirements on the physical interface solutions. Krueger Informational û Exp. May 2001 5 ISCSI Reqmnts and Design Considerations Nov. 2000 First, vendors are producing network processing hardware that offloads network protocols to hardware solutions to achieve higher data rates. The concept of "0 copy" seeks to store blocks of data in appropriate memory locations (aligned) directly off the wire, even in when data is reordered due to packet loss. This is necessary to drive actual data rates of 10G and beyond. Secondly, in order for iSCSI to be successful in the WAN arena it must be possible to operate efficiently in high bandwidth, high delay networks. The emergence of multi-gigabit IP networks with latencies in the tens to hundreds of milliseconds presents a challenge. To fill such large pipes, tens of megabytes of outstanding requests from the application are needed. In addition, some protocols potentially require tens of megabytes at the transport layer to deal with buffering for reassembly of data when packets are received out-of-order. Consider that a network pipe at 10 Gbps . 200 msec holds 250 MB. [Assume land-based communication with a spot half way around the world at the equator. Ignore additional distance due to cable routing. Ignore repeater and switching delays; consider only a speed-of-light delay of 5 .sec / km. The circumference of the globe at the equator is approx. 40000 km (we need to consider round-trip delay to keep the pipe full). 10 Gb/sec . 40000 km . 5 .sec / km . B / 8b = 250 MB]. In a conventional TCP implementation, loss of a TCP segment means that stream processing must stop until that segment is recovered, which takes at least a time of to accomplish. Following the example above, we would be obliged to catch 250 MB of data into an anonymous buffer before we could resume stream processing; later, this data would need to be moved to its proper location. Some proponents of iSCSI seek some means of putting data directly where it belongs, and avoiding extra data movement in the case of segment drop. This is a key concept in understanding the debate behind framing methodologies. The framing of the iSCSI protocol impacts both the "interfacing needs" and the "accelerated processing needs", however, while including a length in a header may suffice for the "interfacing needs", it will not serve the "accelerated processing needs". The framing mechanism developed should allow resynchronization of packet boundaries even in the case where a packet is temporarily missing in the incoming data stream. 4.3. Performance/Cost EDITORS NOTE: Performance/Cost is frequently, but inaccurately, referred to as Cost/Performance. The Performance/Cost formulation is the correct representation, demonstrating that increasing Performance/Cost is good. Krueger Informational û Exp. May 2001 6 ISCSI Reqmnts and Design Considerations Nov. 2000 In general, iSCSI should allow implementations to equal or improve on the current state of the art for SCSI interconnects. [R] Low delay communication. [D] Conventional storage access is of a stop-and-wait or remote procedure call type. Applications typically employ very little pipelining of their storage accesses, and so storage access delay directly impacts performance. The delay imposed by current storage interconnects, including protocol processing, is generally in the range of 100 microseconds. The use of caching in storage controllers means that many storage accesses complete almost instantly, and so the delay of the interconnect can have a high relative impact on overall performance. [R] High bandwidth, bandwidth aggregation. [D] The bandwidth (transfer rate, MB/sec) supported by storage controllers is rapidly increasing, due to several factors: (1) Increase in disk spindle and controller performance; (2) Use of ever-larger caches, and improved caching algorithms; (3) Increased scale of storage controllers (number of supported spindles, speed of interconnects). Not only must the iSCSI provide for full utilization of available link bandwidth, it also must exploit parallelism (multiple connections) at the device interfaces and within the interconnect fabric. [R] Low CPU utilization, equal to or better than current technology. [D] For competitive performance, the iSCSI protocol must allow three key implementation choices to be realized: (1) iSCSI must make it possible to build I/O adapters that handle an entire SCSI task, as alternative SCSI transport implementations do. (2) The protocol must permit "zero-copy" memory architectures, where the I/O adapter reads or writes host memory exactly once per disk transaction. (3) The protocol must not impose complex operations on the host software, which would increase host instruction path length relative to alternatives. [R] Cost competitive with alternative storage network technologies. [R] Possible to move data directly f EDITORS NOTE: Performance/Cost is frequently, but inaccurately, referred to as Cost/Performance. The Performance/Cost formulation is the correct representation, demonstrating that increasing Performance/Cost is good. In general, iSCSI must allow implementations to equal or improve on the current state of the art for SCSI interconnects. [R] Low delay communication. [D] Conventional storage access is of a stop-and-wait or remote procedure call type. Applications typically employ very little Krueger Informational û Exp. May 2001 7 ISCSI Reqmnts and Design Considerations Nov. 2000 pipelining of their storage accesses, and so storage access delay directly impacts performance. The delay imposed by current storage interconnects, including protocol processing, is generally in the range of 100 microseconds. The use of caching in storage controllers means that many storage accesses complete almost instantly, and so the delay of the interconnect can have a high relative impact on overall performance. [R] High bandwidth, bandwidth aggregation. [D] The bandwidth (transfer rate, MB/sec) supported by storage controllers is rapidly increasing, due to several factors: (1) Increase in disk spindle and controller performance; (2) Use of ever-larger caches, and improved caching algorithms; (3) Increased scale of storage controllers (number of supported spindles, speed of interconnects). Not only must the iSCSI provide for full utilization of available link bandwidth, it also must exploit parallelism (multiple connections) at the device interfaces and within the interconnect fabric. [R] Low CPU utilization, equal to or better than current technology. [D] For competitive performance, the iSCSI protocol must allow three key implementation choices to be realized: (1) iSCSI must make it possible to build I/O adapters that handle an entire SCSI task, as alternative SCSI transport implementations do. (2) The protocol must permit "zero-copy" memory architectures, where the I/O adapter reads or writes host memory exactly once per disk transaction. (3) The protocol must not impose complex operations on the host software, which would increase host instruction path length relative to alternatives. [R] Cost competitive with alternative storage network technologies. 5. Ease of implementation/complexity of protocol Experience has shown that adoption of a protocol by the internet community is inversely proportional to its complexity. In addition, the simpler the protocol, the easier it is to diagnose problems. The designers of iSCSI shall strive to fulfill the requirements of the interconnect effort, while keeping the protocol as simple as possible. In the interest of simplicity, iSCSI should minimize optional features. When features are deemed necessary, the protocol should allow for feature negotiation at session establishment (login) and provide for rejection when an implementation does not support a requested feature. Krueger Informational û Exp. May 2001 8 ISCSI Reqmnts and Design Considerations Nov. 2000 6. Reliability and Availability ISCSI protocol design, while placing an emphasis on simplicity, should lead to timely recovery from failure of initiator, target, or connecting internet infrastructure (cabling, data path equipment such as routers, etc). This would provide a basis for layered technologies like high availability and clustering. The protocol specification should take into account fail-over schemes for mirrored targets or highly available storage configurations that provide paths to target data through multiple "storage servers". 7. Multiple Paths for Throughput History has shown that any single link can be saturated by storage traffic. Scientific data applications, asynchronous and synchronous data replication are examples of applications that have pushed and continue to push the limits of throughput. The iSCSI standard MUST allow the initiator and target to use multiple network interfaces and multiple paths through the network for increased throughput. Some applications, like log updates, streaming tape, and replication, require ordering of updates and thus ordering of SCSI commands. An initiator may maintain ordering by waiting for each update to complete before issuing the next (a.k.a. synchronous updates). However, the throughput of synchronous updates decreases inversely with increases in latency of the operation. To allow an initiator to maintain throughput, the SCSI task queuing mechanism allows an initiator to have multiple commands outstanding at the target simultaneously and to express ordering constraints on the execution of those commands. The task queuing mechanism is only effective if the commands arrive at the target in the order they were presented to the initiator (FIFO order). The iSCSI standard MAY provide a FIFO transport of SCSI commands, even when commands are sent along different paths. This FIFO transport mechanism MAY wish to minimize the amount of communication necessary across multiple adapters doing transport off-load. There are a few potential ways to satisfy the multiple path and ordering requirements. A popular way to satisfy the multiple-path requirement is to have a driver above the SCSI layer instantiate multiple copies of the SCSI transport, each communicating to the target along a different path. ôWedgeö drivers use this technique today to attain high performance. Unfortunately, wedge drivers must use stop-and-wait to do ordered updates. Krueger Informational û Exp. May 2001 9 ISCSI Reqmnts and Design Considerations Nov. 2000 Another approach might be for the iSCSI protocol to use multiple instances of its underlying transport (e.g. TCP). The iSCSI layer would make these independent transport instances appear as one SCSI transport instance and maintain the ability to do ordered SCSI command queuing. The document will refer to this technique as "connection binding" for convenience. The consensus of the working group is that support for connection binding is NOT a requirement for initiators and targets. (ref e-mail of David Black to ips reflector on Oct 11, 2000) There has been no explicit decision on whether the protocol is required to support connection binding. In the presence of connection binding, there are two ways to assign features to connections. In the symmetric approach, all the connections are identical from a feature standpoint. In the asymmetric model, connections have different features. For example, some connections may be used primarily for data transfers whereas others are used primarily for SCSI commands. Another point in the design space for connection binding has to do with the data transfer associated with a SCSI command. The data transfer is said to have allegiance to the command if it occurs on the same connection on which the command was sent. A data transfer can also potentially have allegiance to a channel which is different from the command was sent (and perhaps even specified in the command request). Finally, a data transfer can have no allegiance and appear across number of any connection. The question of symmetric or asymmetric has yet to be resolved by the working group. The symmetric approach potentially requires less communication between the interfaces and has simpler recovery semantics in the case of a connection failure. The asymmetric approach can simplify some aspects of the protocol and potentially yields greater throughput. The symmetric approach with command allegiance is currently being pursued. 8. Recovery The iSCSI protocol MUST provide the ability to recover from a failed, hung, or timed-out TCP connection, without the loss of the session between the initiator and target. This must particularly work for non-idempotent requests, such as operations on tape drives. If all TCP connections for a session fail, and no connections can be established, the iSCSI session shall be aborted. The iSCSI protocol SHALL attempt to provide recovery in a timely fashion from initiator and target reboots and failovers to other physical devices. Krueger Informational û Exp. May 2001 10 ISCSI Reqmnts and Design Considerations Nov. 2000 The iSCSI protocol MUST also provide a method for sessions to be gracefully terminated and restarted that can be initiated by either the initiator or target. This provides the ability to gracefully fail over an initiator or target, or to gracefully reset a target after upgrading software or performing other maintenance tasks. 8.1. Interoperability It must be possible for initiators and targets that implement the required portions of the iSCSI specification to interoperate. 8.2. Internet infrastructure The iSCSI protocol MUST: -- be compatible with both IPv4 and IPv6 -- use TCP connections conservatively, keeping in mind there may be many other users of TCP on a given machine. The iSCSI protocol MUST NOT: -- require changes to existing internet protocols 8.3. SCSI Since iSCSI is a SCSI transport, the iSCSI standard SHOULD comply with the requirements of the SCSI Architecture Model [SAM2] and SHOULD support all current SCSI command sets. Furthermore, it MUST be possible to create bridges from iSCSI to other SCSI interconnects. track changes to SCSI and the SCSI Architecture Model. iSCSI is a new SCSI "transport" [SAM2]. Being the intersection of SCSI and TCP, iSCSI has potential impact on T10 as well as on IETF. However, a stated requirement (below) is that iSCSI shall have no impact on T10 architecture or command sets. Collaboration with T10 will be necessary to achieve this requirement. Collaboration with T10 concerns three phases of T10 activity: (1) Past. For T10 work completed in the past, and well-document in T10 standards publication, the IPS working group will seek assistance in properly interpreting those standards; (2) Present. For T10 work that is ongoing, or recently completed (but not widely published), the IPS working group will seek review of our work by individuals active in T10, and/or the participation of those individuals in the IETF process; (3) Future. For compatibility with future T10 work, it is essential that iSCSI be a legitimate and recognized "SCSI transport", no less so than the several other SCSI transports. Krueger Informational û Exp. May 2001 11 ISCSI Reqmnts and Design Considerations Nov. 2000 SCSI command standards must evolve within the context of all existing SCSI transports. Storage attachment to IP networks will engender an unprecedented potential for device sharing. This alone may impact future T10 work. The iSCSI protocol MUST support all SCSI-3 command sets and device types. The primary focus is on supporting ôlargerö devices: host computers and storage controllers (disk arrays, tape libraries). However, other command sets (printers, scanners) MUST be supported. These requirements must not be construed to mean that iSCSI must be natively implementable on all of todayÆs SCSI devices, which might have limited processing power or memory. The iSCSI protocol MUST not require changes to the SCSI-3 command sets and SCSI client code except to reflect lengthier iSCSI target names and potentially lengthier timeouts. The iSCSI standard MUST allow for the construction of gateways to other SCSI transports, including parallel SCSI [SPI-X] and to SCSI- FCP[FCP, FCP-2]. It MUST be possible to construct "translating" gateways so that iSCSI hosts can talk to SCSI-X devices; so that SCSI-X devices can talk to each other over an iSCSI network; and so that SCSI-X hosts can talk to iSCSI devices (where SCSI-X refers to parallel SCSI, SCSI-FCP, or SCSI over any other transport). This requirement is implied by support for SAM-2, but is worthy of emphasis[D] These are true application protocol gateways, and not just bridge/routers. The different standards have only the SCSI-3 command set layer in common. These gateways are not mere packet forwarders. The iSCSI standard MUST reliably transport SCSI commands from the initiator to the target. According to [SAM-2, p. 17.] "The function of the service delivery subsystem is to transport an error-free copy of the request or response between the sender and the receiverà" [SAM-2, p. 22]. The iSCSI standard or its transport MUST correctly deal with packet drop, duplication, corruption, stale packets, and re-ordering. The iSCSI standard MUST support FIFO delivery of SCSI commands from the initiator to the target, so as to enable support for task ordering in SCSI Task Queuing. 9. Security Considerations 9.1. Authentication The iSCSI protocol MUST support authenticated login. Authenticated login aids the target in blocking the unauthorized use of SCSI Krueger Informational û Exp. May 2001 12 ISCSI Reqmnts and Design Considerations Nov. 2000 resources. Since block storage is considered critical in many environments and many IP networks provide easy connectivity, many organizations will want to protect their IP SCSI resources. The iSCSI authenticated login MUST be resilient against passive attacks since many IP networks are vulnerable to packet inspection. Simple, US-exportable techniques exist to satisfy this requirement. In addition, the iSCSI protocol MUST support optional authentication of its communications. This requirement may be met using IPsec or SSL/TLS or with some iSCSI-specific mechanism. The endpoints may negotiate the authentication method, optionally none. The endpoints will not be required to support any authentication algorithms. Authentication of the communications is critical since IP networks are vulnerable to source spoofing, where a malicious third party can pretend to send packets from the initiatorÆs IP address. 9.2. Data Integrity Requirements: -- The iSCSI protocol shall support the negotiation of data integrity schemes during connection login. -- The iSCSI protocol shall support the negotiation of a data integrity mechanism for SCSI data, blocks, separable from data integrity mechanisms performed on commands, status, and iSCSI headers. -- The iSCSI data integrity negotiation scheme shall be extensible to include other data integrity check mechanisms. -- The iSCSI protocol shall not preclude the use of stream data integrity mechanisms provided by IPSec. The iSCSI protocol must provide the ability to select data integrity mechanisms that are appropriate for each environment in which it is to run. For example, a layer 2 network (such as Ethernet) uses a CRC to protect each IP packet that is comparable to the CRC used to protect Fibre Channel frames. When running in this environment, it is likely that no additional data integrity mechanisms need be provided by iSCSI, so a data integrity scheme of "none" might be used. However, in a L3 or L4 routed network, the Ethernet (or other layer 2) CRC is removed and replaced at each router, and the iSCSI stream is protected only by the 16-bit TCP checksum. In some applications and networks, this still may be acceptable, but in many cases a stronger check is needed. Some of the options that have been discussed rely either on adding a TCP option for CRC, which would require work on the implementorÆs TCP stack, or would rely on data Krueger Informational û Exp. May 2001 13 ISCSI Reqmnts and Design Considerations Nov. 2000 integrity checks from a security layer such as IPsec. These are both technically workable solutions, but will not work across iSCSI proxies or gateways. In an iSCSI proxy or gateway situation, the iSCSI headers are removed and re-added, and the TCP stream is terminated on either side. This means that even the TCP checksum is removed and recomputed within the gateway. To ensure the protection of commands, data, and status, a CRC or other mechanism is required to operation on the SCSI data block itself, as well as on each command and status message. Since the iSCSI headers can be stripped and remade, the iSCSI headers cannot be included in these CRCs, and must have their own. 9.3. Data Privacy Block storage is used for storing sensitive information, like medical records, where data privacy is critical. Encrypting the data blocks before writing them to storage provides the best protection for the application. Even if the storage or communications are compromised, the attacker will have difficulty reading the data. However, for certain environments, link encryption may be sufficient or provide an extra layer of assurance of privacy. An iSCSI implementation MAY use protocols such as TLS or IPsec to provide data privacy over a link. 10. Management iSCSI devices should be manageable using IP-based management protocols (ex. SNMP, RMI). iSCSI devices may also be manageable using SCSI commands for management (ex. SCSI Enclosure Services, SES commands). The iSCSI protocol document will not define the management architecture for iSCSI networks. 10.1. Naming Whenever possible, iSCSI shall support the naming architecture of SAM-2. Deviations and uncertainties will be made explicit, and comment/resolution invited. The iSCSI protocol shall provide a means of identifying iSCSI targets by a flexible path address (URL), where the path is the combination of a DNS name or IP address, a TCP port, and an optional ASCII path name identifying the target. Krueger Informational û Exp. May 2001 14 ISCSI Reqmnts and Design Considerations Nov. 2000 The iSCSI protocol shall provide a means of identifying iSCSI targets by a world-wide unique identifier (WWUI), that is independent of the path on which it is found. This will be used to correlate alternate paths to the same device. Implementation support for the WWUI is strongly recommended, but optional. Note that LU names are discovered through SCSI-level inquiries, and are not just for Fibre Channel. There is nothing to prevent iSCSI (or parallel SCSI) from implementing the LU WWN. As such, this is outside the scope of the iSCSI protocol specification. Standard internet lookup services should be used to resolve names. For example, the Domain Name Service (DNS) MAY be used to resolve the portion of the URL to one, or multiple IP addresses. When a hostname resolves to multiple addresses, these addresses should be equivalent for functional (possibly not performance) purposes. This means that the addresses can be used interchangeably as long as performance isnÆt a concern. For example, the same set of SCSI targets must be accessible from each of these addresses. [R] Deal with the complications of the new SCSI security architecture [99-245r8]. [D] Pay attention to the proxy naming architecture defined by the new security model. In this new model, SCSI Logical Unit Numbers (LUNs) can be mapped in a manner that gives each host (more correctly, each AccessID) a unique LU map. Thus, a given LU within a target may be addressed by different LUNs. [R] Support SCSI 3rd-party operations. [D] The key issue here relates to the naming architecture for SCSI LUs. We need to determine a method of passing a name or handle between parties 10.2. Topology Discovery iSCSI shall have no impact on the use of conventional IP network discovery techniques. Various network management platforms have ways of discovering IP addresses. These techniques will be used, and will find all of the IP end points that contain iSCSI nodes. The iSCSI protocol shall provide appropriate discovery mechanisms which scale from adding single devices to an iSCSI-internal storage subsystem, up to the deployment of multi-customer, multi-utility storage outsourcing environments. iSCSI shall provide some means of determining that a discovered IP end point is an iSCSI node. It is expected that iSCSI is a point of service in a host, just as SNMP, etc are points of services, and are associated with a well known port number. One solution to this problem would be to produce an iSCSI device MIB specification. Krueger Informational û Exp. May 2001 15 ISCSI Reqmnts and Design Considerations Nov. 2000 The iSCSI protocol shall provide a method of discovering, given an IP end point on its well-known port, the list of SCSI targets available to the requestor. These targets can either be path addresses, or WWUIs. The use of this discovery service shall be optional. SCSI protocol-dependent techniques shall be used for further discovery beyond the iSCSI layer. Discovery is a complex process. SCSI provides specific hooks for doing the work, so the commands associated with this process will also work over iSCSI. Generally the SCSI discovery process involves using the Report LUNs command to determine which LUs are addressable at a given service delivery port. Subsequently, the true identity of each LU (ie, name) is discovered by reading Vital product data page 83h. By comparing LU IDs, the discovery process can find that a given LU is accessible through multiple paths. 11. Internet Accessibility 11.1. Denial of Service As with all services, the denial of service by either incorrect implementations or malicious agents is always a concern. All aspects of the iSCSI protocol should be scrutinized for potential denial of service issues, and guarded against as much as possible. 11.2. Firewalls and Proxy servers During the login phase, any login or connect command must include the full iSCSI address of the target to which the initiator wishes to connect. This includes the IP Address (or DNS name), TCP port number, and iSCSI PATH (target name), and allows an initiator to connect to a target through an iSCSI proxy server. The iSCSI protocolÆs use of IP addressing and TCP port numbers must be firewall friendly. This probably means that all connection requests should be addressed a specific, well-known TCP port. That way, firewalls can filter based on source and destination IP addresses, and destination (target) port number. The source (initiator) port number also should be well-known for the initial TCP connection. Additional TCP connections would require different source port numbers (for uniqueness), but could be opened after a security dialogue on the control channel. ItÆs important that iSCSI operate through a firewall to provide a possible means of defending against Denial of Service (DoS) assaults from less-trusted areas of the network. It is assumed that a firewall will have much greater processing power for dismissing bogus connection requests than do the end nodes. Krueger Informational û Exp. May 2001 16 ISCSI Reqmnts and Design Considerations Nov. 2000 11.3. Congestion control and Transport Selection The iSCSI protocol MUST be a good network citizen with TCP- compatible congestion control (as defined in RFC 2309). In addition, iSCSI implementations MUST not use multiple connections as a means to avoid transport-layer congestion control. 12. Virtualization Virtualization of targets and LUNs is generally handled by intelligent gateways, storage controllers, or other devices. Many vendors, especially those that build storage devices, include very advanced virtualization features that are beyond the scope of a SCSI transport layer to define, and are usually closely guarded as intellectual property. Requiring the iSCSI protocol to work within an environment that includes proxies and gateways (see earlier requirements) will provide a SCSI transport that will enable vendors to add their own virtualization features without breaking the protocol or causing interoperability problems. 13. References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 1 [SAM-2] ANSI NCITS. Weber, Ralph O., editor. SCSI Architecture Model -2 (SAM-2). T10 Project 1157-D. rev 13, 22 Mar 2000. 2 [SPC-2] ANSI NCITS. Weber, Ralph O., editor. SCSI Primary Commands û 2 (SPC-2). T10 Project 1236-D. rev 18, 21 May 2000. 3 [CAM-3] ANSI NCITS. Dallas, William D., editor. Information Technology û Common Access Method û 3 (CAM-3)). X3T10 Project 990D. rev 3, 16 Mar 1998. 4 [99-245r8] Hafner, Jim. A Detailed Proposal for Access Controls. T10/99-245 revision 8, 26 Apr 2000. 5 [SPI-X] ANSI NCITS. SCSI Parallel Interface û X. 6 [FCP] ANSI NCITS. SCSI-3 Fibre Channel Protocol [ANSI X3.269:1996] Krueger Informational û Exp. May 2001 17 ISCSI Reqmnts and Design Considerations Nov. 2000 7 [FCP-2] ANSI NCITS. SCSI-3 Fibre Channel Protocol û 2 [T10/1144- D] 14. Acknowledgements 15. Author's Addresses Address comments to: Marjorie Krueger Hewlett-Packard Corporation 8000 Foothills Blvd Roseville, CA 95747-5668, USA Phone: +1 916 785-2656 Email: marjorie_krueger@hp.com Randy Haagens Hewlett-Packard Corporation 8000 Foothills Blvd Roseville, CA 95747-5668, USA Phone: +1 916 785-4578 Email: Randy_Haagens@hp.com Costa Sapuntzakis Cisco Systems, Inc. 170 W. Tasman Dr. San Jose, CA 95134, USA Phone: +1 408 525-5497 Email: csapuntz@cisco.com Mark Bakke Cisco Systems, Inc. 6450 Wedgwood Road Maple Grove, MN 55311 Phone: +1 763 398-1054 Email: mbakke@cisco.com Krueger Informational û Exp. May 2001 18 ISCSI Reqmnts and Design Considerations Nov. 2000 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into Krueger Informational û Exp. May 2001 19