NFSv4 Working Group S. Faibish Internet-Draft EMC Corporation Intended status: draft J. Glasgow Expires: September 7, 2011 Google Updates: 5663 D. Black EMC Corporation March 7, 2011 pNFS block disk protection draft-faibish-nfsv4-pnfs-block-disk-protection-00 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on September 7, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this Faibish et al. Expires September 7, 2011 [Page 1] Internet-Draft pNFS block disk protection March 2011 document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Abstract Parallel NFS (pNFS) extends Network File Sharing version 4 (NFSv4) to allow client to directly access file data on the storage used by the NFSv4 server. This ability to bypass the server for data access can increase both performance and parallelism, but requires additional client functionality for data access, some of which is dependent of the class of storage used. The published protocol for block layout file systems describes how clients can recognize the volumes used for pNFS storage, but only after communicating with the server. This document proposes a mechanism by which clients can recognize block storage devices used by pNFS file systems, without having to communicate with the server, and therefore allows the clients to limit access to the devices at client boot time. Table of Contents 1. Introduction...................................................3 1.1. Non-pNFS clients use case.................................5 1.2. pNFS client use case......................................5 2. Conventions used in this document..............................5 3. Extending block/volume signature...............................5 3.1. Modifications to 2.3.5 RFC5663............................7 4. Problem Statement..............................................7 4.1. Non-pNFS clients..........................................7 4.2. Solution..................................................7 4.3. pNFS clients..............................................8 5. Changes to GETDEVICEINFO and or LAYOUTCOMMIT (RFC5661).........8 6. Security Considerations........................................8 7. IANA Considerations............................................9 8. Conclusions....................................................9 9. References.....................................................9 9.1. Normative References......................................9 9.2. Informative References....................................9 Authors Addresses...............................................10 Faibish et al. Expires September 7, 2011 [Page 2] Internet-Draft pNFS block disk protection March 2011 1. Introduction Figure 1 shows the overall architecture of a Parallel NFS (pNFS) system: +-----------+ |+-----------+ +-----------+ ||+-----------+ | | ||| | NFSv4.1 + pNFS | | +|| Clients |<------------------------------>| MDS | +| | | | +-----------+ | | ||| +-----------+ ||| | ||| | ||| Storage +-----------+ | ||| Protocol |+-----------+ | ||+----------------||+-----------+ Control | |+-----------------||| | Protocol | +------------------+|| Storage |------------+ +| Devices | +-----------+ Figure 1 pNFS Architecture In this document, "storage device" is used as a general term for a data server and/or storage server for the file, block or object pNFS layouts. In the current pNFS block protocol [RFC5663] the client has to contact the MDS in order to get the signature offset of the devices used by the pNFS block client. If an operating system wants to be able to identify a device/LUN being used to store pNFS data during the boot sequence, before the mount is issued, without having to contact the metadata server it is not possible because the protocol does not define a fixed location for a signature or for an identifier of a device as being used by pNFS. The MDS will send the device ID after mount time. In order for the OS to mount it needs to have the network configured and the IP address of the pNFS server and this always happens after the discovery of SCSI devices. Faibish et al. Expires September 7, 2011 [Page 3] Internet-Draft pNFS block disk protection March 2011 location of the signature is also not defined in the protocol but we can make it a configuration parameter in the client OS that will define the signature location for each pNFS block server, vendor neutral. The preferred solution would be to enhance the block protocol to include a signature unique identifying pNFS. It needs to be included in the protocol as both clients and server need to know this information. In more details the pNFS FS writes a signature to the LUN, but to find out the offset of the signature the client first needs to talk with the MDS to be told the offset that the signature is located at using GETDEVICEINFO. But this is too late and can only be done after the boot ends. An OS that wants to hide LUNs from users that contain pNFS data even if that host doesn't mount the pNFS block volume cannot do this. The problem is that there is a window of time before an OS utility/daemon can be started and allow to protect/prevent to write to the pNFS devices and the time when kernel application with higher priority than the above daemon can overwrite the pNFS devices. As a result of this it may be possible that some kernel application will write over the pNFS FS devices/LUNs. This is only a pNFS block issue and will have no secondary effect on OS kernel operation. An OS will check the signature of pNFS should be able to identify the devices and prevent from writing to them. But this enhancement should allow such a protection mechanism to be implemented. This is outside the scope of this draft to detect and protect the devices if the specific pNFS signature is detected and do so without the need to contact the MDS "if there is pNFS specific component of the signature. Even in this case we can use generic identifiers or some opaque in the protocol can recognize pNFS devices even though pNFS block protocol doesn't require any protection. Typically, storage area network (SAN) disk arrays and SAN protocols provide access control mechanisms (e.g., Logical Unit Number (LUN) mapping and/or masking), which operate at the granularity of individual hosts, not individual blocks. For this reason, block- based protection must be provided by the client software. Since block/volume storage systems are generally not capable of enforcing such file-based security, in environments where pNFS clients cannot be trusted to enforce such policies, pNFS block/volume storage layouts SHOULD NOT be used. Faibish et al. Expires September 7, 2011 [Page 4] Internet-Draft pNFS block disk protection March 2011 1.1. Non-pNFS clients use case The non-pNFS clients will need to check the signature related to pNFS block devices and prevent from WRITE data to the devices that have the pNFS identifier. The non-pNFS client OS MAY decide to implement an application that will read the pNFS signature at a known location (LBA) on the block device and write protect the device. Or it MAY just detect the pNFS devices and prevent using them for other applications that use block devices. The current draft only make these use cases possible and non-pNFS clients MAY NOT use this feature. 1.2. pNFS client use case pNFS clients MAY use this signature in cases when a pNFS MDS server sends layout extents to devices that were erroneously not signed using this pNFS identification by the server and included in valid layouts. The pNFS client MAY check the pNFS signature for devices that are included in valid layouts and report an error to the MDS in the LAYOUTCOMMIT or using a permission access mechanism similar to [PAC]. This use case is relevant in cases when virtual LUs are used as pNFS devices and they are added after the client issued a GETDEVICEINFO. Alternatively when the pNFS client detects the missing signature it may send a GETDEVICEINFO command to the MDS for the faulty device and use a permission error code to the MDS. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. 3. Extending block/volume signature The pNFS block layout identifies storage volumes by content, for example providing the means to match (unique portions of) labels used by volume managers. Volume identification is performed by matching one or more opaque byte sequences to specific parts of the stored data. Any block pNFS system using this layout MUST support a means of content-based unique volume identification that can be employed via the data structure given here. The location of the pNFS signature will be defined in the remaining of the document. Faibish et al. Expires September 7, 2011 [Page 5] Internet-Draft pNFS block disk protection March 2011 The GUID used in the GPT should not be confused with the device signature described in section 2.2.1 of [RFC5663]. The GUID is the same for all pNFS volumes where as the signature composed of pnfs_block_sig_component4 components is unique to each pNFS volume. The location on disk of these identifiers are unrelated. A pNFS signature is an array of up to "PNFS_BLOCK_MAX_DISK_SIG_COMP" (defined below) signature components. The client MUST NOT assume that all signature components are co-located within a single sector on a block device. The pNFS client block layout driver uses this volume identification to identify block devices used by pNFS file systems. The pNFS device signature may require several LBAs in order to ensure uniqueness of the signature in case that other disk signatures exist on the same location on the block device and prevent confusions with other signatures. All the new pNFS identifiers must not be used in extents covering data access and need to be read only accessible to both pNFS and non- pNFS clients. Additional, pNFS clients SHOULD NOT access/write to volumes that do not have the pNFS specific volume identifier if included in a valid layout. This can be relevant in cases of virtual LUs when they are extended to include new volumes uninitialized by the server. After the GETDEVICEINFO is received by the client the client MAY read the pNFS identifier in order to validate the extents on that device and prevent from writing to devices that do not include pNFS identifier. In this case pNFS clients will send an error to the server and inform it that the layout is invalid and or that there is a permission access error. And will fallback the I/O to the MDS. The error mechanism can be similar to the one used in the [pNFS-draft- permission-access] for lack of permission to access reinforced by the pNFS client. pNFS client implementations MAY chose not to validate that the device included in the layouts received from the server is a pNFS device. The pNFS client can OPTIONALLY check the pNFS identifier and send an error message in the layoutcommit for any device with a missing pNFS identifier. Faibish et al. Expires September 7, 2011 [Page 6] Internet-Draft pNFS block disk protection March 2011 3.1. Modifications to 2.3.5 RFC5663 The above assumption that extents are permissions may be in contradiction to the case when a device in the layout has no pNFS identity without the knowledge of the pNFS server. This because the pNFS client is the first to detect the lack of identifier while the server is unaware of this issue and assumed that the device used in the layout and the respective extent has the right permissions. The server MAY check that the device used in the layout is a legitimate pNFS device at the time it respond to the GETDEVICEINFO call from the client. Alternatively in the case when a device has no pNFS identifier the pNFS client SHOULD send an GETDEVICEINFO for that device to ensure that the device included in a valid layout can be accessed for IO. It may or may not report an error in the GETDEVICEINFO and/or in the LAYOUTCOMMIT at the end of the write I/O. The check can be made for both read-only layouts but SHOULD be done in the case of read-write layouts. The server is responsible to ensure that the pNFS id is written to the devices used by the pNFS FSID before allowing pNFS clients to mount the fsid. 4. Problem Statement 4.1. Non-pNFS clients The pNFS block layout [RFC5663] requires that storage systems allow both the server and multiple clients to access block storage devices. In principle, access control to block storage devices can be achieved via LUN masking techniques at the storage server as volumes are mounted. In practice this is not always done, and storage devices are often accessible independent of whether or not a client has mounted the associated pNFS file system. For this reason, it is desirable to enable clients to identify pNFS block storage in order to prevent non-pNFS access by the client. 4.2. Solution The pNFS block layout specification allows a client to identify the storage devices associated with a mounted volume via the GETDEVICELIST and GETDEVICEINFO operations. These operations can only be issued once a client has established contact with the server, and thus do not allow a client to block non-pNFS access to pNFS storage devices in all cases. For environments in which it is Faibish et al. Expires September 7, 2011 [Page 7] Internet-Draft pNFS block disk protection March 2011 desirable for clients to block non-pNFS access to pNFS block storage, the following SHOULD be done: - Each storage device dedicated to pNFS includes a GUID partition table (GPT). - The pNFS Block Storage partitions are identified in the GPT with GUID e5b72a69-23e5-4b4d-b176-16532674fc34. - NFS clients do not issue NFS operations for non-pNFS access to any storage identified as pNFS Block Storage by that GUID. This enables an NFS client to prevent non-pNFS access to pNFS block storage immediately upon boot. As of 2010 most current OSs support GPT including FreeBSD, Linux and Solaris [GPT]. Block/volume storage is logically at a lower layer of the I/O stack than the network stack and hence during the client boot cycle access to block devices is possible before NFSv4 security can be enforced. The client MUST prevent WRITE I/Os to devices that will be later identified as pNFS devices. It is the responsibility of those administering and deploying pNFS with a block/volume storage access protocol to ensure that appropriate protection is provided to pNFS devices identifiable by the client. This protection is similar to the mechanism that protect GPT tables written on the block devices used by different operating systems. 4.3. pNFS clients In this case the client MAY check if the devices in the GETDEVICEINFO list have pNFS signature at mount time. At I/O time the pNFS client MAY check the signature to validate that the block devices are valid and report to the server the error. 5. Changes to GETDEVICEINFO and or LAYOUTCOMMIT (RFC5661) [Need to include the changes if we agree on the need for this] 6. Security Considerations The security considerations of 2.6 [RFC5663] apply to this document. This document formalizes a mechanism which allows client operating systems to limit access to pNFS block storage devices. By doing so it allows for increase security in environments in which the client operating system is trusted. As with the issues discussed in the security section of 2.6 [RFC5663], in environments where the security Faibish et al. Expires September 7, 2011 [Page 8] Internet-Draft pNFS block disk protection March 2011 requirements are such that client-side protection from access to storage is not sufficient, pNFS block/volume storage layouts SHOULD NOT be used. Furthermore, in such environments, SAN mechanism (e.g., LUN mapping and/or masking) should be used to limit access. 7. IANA Considerations There are no IANA considerations in this document beyond pNFS IANA Considerations are covered in [RFC5661]. 8. Conclusions This draft specifies additions to the pNFS block protocol addressing protection of disks used by pNFS clients for non-pNFS clients access. 9. References 9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File System (NFS) Version 4 Minor Version 1 Protocol", http://tools.ietf.org/html/rfc5661, January 2010. [RFC5663] Black, D., Glasgow, J., Fridella, S., "Parallel NFS (pNFS) Block/Volume Layout", http://tools.ietf.org/html/rfc5663, January 2010. [RFC5664] Halevy, B., Welch, B., Zelenka, J., "Object-Based Parallel NFS (pNFS) Operations", http://tools.ietf.org/html/rfc5664, January 2010 [XDR] Eisler, M., "XDR: External Data Representation Standard", STD 67, RFC 4506, May 2006. 9.2. Informative References [GPT] http://en.wikipedia.org/wiki/GUID_Partition_Table [PAC] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-pnfs- access-permissions-check/ This document was prepared using 2-Word-v2.0.template.dot. Faibish et al. Expires September 7, 2011 [Page 9] Internet-Draft pNFS block disk protection March 2011 Authors Addresses Sorin Faibish (editor) EMC Corporation 228 South Street Hopkinton, MA 01748 US Phone: +1 (508) 305-8545 Email: sfaibish@emc.com Jason Glasgow Google 5 Cambridge Center, Floors 3-6 Cambridge, MA 02142 US Phone: +1 (617) 575 1599 Email: jglasgow@google.com David L. Black EMC Corporation 176 South Street Hopkinton, MA 01748 US Phone: +1 (508) 293-7953 Email: david.black@emc.com Faibish et al. Expires September 7, 2011 [Page 10]