| Applications Area Working Group | M. Kerwin |
| Internet-Draft | QUT |
| Obsoletes: 1738 (if approved) | November 30, 2015 |
| Intended status: Standards Track | |
| Expires: June 2, 2016 |
The file URI Scheme
draft-ietf-appsawg-file-scheme-05
This document specifies the “file” Uniform Resource Identifier (URI) scheme, obsoleting the definition in RFC 1738.
It attempts to define a common core which is intended to interoperate across the broad spectrum of existing implementations, while at the same time documenting other current practices.
Note to Readers (To be removed by the RFC Editor)
This draft should be discussed on the IETF Applications Area Working Group discussion list <apps-discuss@ietf.org>.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 2, 2016.
Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
A file URI identifies a file on a particular file system. It can be used in discussions about the file, and if other conditions are met it can be dereferenced to directly access the file.
The file URI scheme is not coupled with a specific protocol, nor with a specific media type. See Section 3 for a discussion of operations that can be performed on a file URI.
This document defines a syntax that is compatible with most extant implementations, while attempting to push towards a stricter subset of “ideal” constructs. In many cases it simultaneously acknowledges and deprecates some less common or outdated constructs.
The file URI scheme was first defined in [RFC1630], which, being an informational RFC, does not specify an Internet standard. The definition was standardised in [RFC1738], and the scheme was registered with the Internet Assigned Numbers Authority (IANA); however that definition omitted certain language included by the former that clarified aspects such as:
The Internet draft [I-D.hoffman-file-uri] was written in an effort to keep the file URI scheme on standards track when [RFC1738] was made obsolete, but that draft expired in 2005. It enumerated concerns arising from the various, often conflicting implementations of the scheme. It serves as the spiritual predecessor of this document.
Additionally the WHATWG defines a living URL standard [WHATWG-URL], which includes algorithms for interpreting file URIs (as URLs).
The Universal Naming Convention (UNC) [MS-DTYP] defines a string format that can perform a similar role to the file URI scheme in describing the location of files. A UNC filespace selector string has three parts: host, share, and path; see Appendix E. This document describes but does not specify a means of translating between UNC filespace selector strings and file URIs in Appendix C.2.
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119].
Throughout this document the term “local” is used to describe files that can be accessed directly through the local file system. It is important to note that a local file may not be physically located on the local machine, for example if a networked file system is transparently mounted into the local file system.
The file URI syntax is defined here in Augmented Backus-Naur Form (ABNF) [RFC5234], including the core ABNF syntax rule ALPHA defined by that specification, and importing the userinfo, host and path-absolute rules from [RFC3986] (as updated by [RFC6874].)
The core syntax in [RFC3986] includes path and authority components, for each of which only a subset is used in the definition of the file URI scheme. The relevant subset of path is path-absolute, and the subset of authority is file-auth, given below.
Please note Appendix C that lists some other commonly seen but nonstandard variations.
file-URI = file-scheme ":" file-hier-part
file-scheme = "file"
file-hier-part = "//" auth-path
/ local-path
auth-path = [ file-auth ] path-absolute
local-path = path-absolute
file-auth = [ userinfo "@" ] host
The syntax definition above is different from those given in [RFC1630] and [RFC1738] as it is derived from the generic syntax of [RFC3986], which post-dates all previous specifications.
As a special case, the “file-auth” rule can match the string “localhost” or the empty string; either value is interpreted as “the machine from which the URI is being interpreted,” exactly as if no authority was present. To maximise compatibility with previous specifications, implementations MAY choose to include an empty “file-auth”.
Systems exhibit different levels of case-sensitivity. Unless the file system is known to be case-insensitive, implementations MUST maintain the case of file and directory names when translating file URIs to and from the local system’s representation of file paths, and any systems or devices that transport file URIs MUST NOT alter the case of file URIs they transport.
Implementations that provide dereferencing ooperations on file URIs SHOULD, at a minimum, provide a read-like operation to return the contents of a file located by a file URI. Additional operations MAY be provided, such as writing to, creating, and deleting files. See the POSIX file and directory operations [POSIX] for examples of standardized operations that can be performed on files.
File URIs can also be translated to and from other, similar constructs, such as local file paths or UNC strings.
A file URI can be dependably dereferenced or translated to a local file path only if it is local. A file URI is considered “local” if it has a blank or no authority, or the authority is the special string “localhost”.
This specification neither defines nor forbids a mechanism for accessing non-local files. See SMB [MS-SMB], NFS [RFC7530], NCP [NOVELL] for examples of protocols that can be used to access files over a network. Also see Appendix C.2 for a discussion on translating non-local file URIs to and from UNC stings.
Below is an algorithmic description of the process used to convert a file path to a URI; see Section 4.
Differences from RFC 1738
In [RFC1738] a file URL always started with the token “file://”, followed by an (optionally blank) authority and a “/”. That “/” was not considered part of the path. This implies that the correct encoding for a file path in a UNIX-like environment would have been:
token + authority + slash + path
= "file://" + "" + "/" + "/path/to/file.txt"
= "file:////path/to/file.txt"
However that construct was never observed in practice, and in fact would have collided with the eventual encoding of UNC strings in URIs described in Appendix C.3.
Translating a non-local file path, including a UNC string, to a file URI follows the same basic algorithm as for local files, above, except that the authority MUST refer to the network-accesible node that hosts the file.
For example, in a clustered OpenVMS Files-11 system the authority would contain the node name. Where the original node reference includes a username and password in an access control string, they MAY be transcribed into the userinfo field of the authority ([RFC3986], Section 3.2.1), security considerations (Section 6) notwithstanding.
See Appendix C.2 for an explicit handling of UNC strings.
Some conventional file path formats are known to be incompatible with the file URI scheme.
The Microsoft Windows API defines Win32 Namespaces [Win32-Namespaces] for interacting with files and devices using Windows API functions. These namespaced paths are prefixed by “\\?\” for Win32 File Namespaces and “\\.\” for Win32 Device Namespaces. There is also a special case for UNC file paths in Win32 File Namespaces, referred to as “Long UNC”, using the prefix “\\?\UNC\”.
This specification does not define a mechanism for translating namespaced paths to or from file URIs.
To avoid ambiguity, a file URI SHOULD be transported as an Internationalized Resource Identifier (IRI) [RFC3987], or as a URI with non-ASCII characters encoded according to the UTF-8 character encoding [STD63] and percent-encoded as needed ([RFC3986], Section 2.5).
The encoding of a file URI depends on the file system that stores the identified file. If the file system uses a known non-Unicode character encoding, the path SHOULD be converted to a sequence of characters from the Universal Character Set [ISO10646] normalized according to Normalization Form C (NFC) [UTR15], before being translated to a file URI, and conversely a file URI SHOULD be converted back to the file system’s native encoding when dereferencing or translating to a file path.
When the file system’s encoding is not known the file URI SHOULD be transported as an Internationalized Resource Identifier (IRI) [RFC3987] to avoid ambiguity. See Appendix D for examples.
As per [RFC6454], Section 4, when determining the origin of a file URI implementations MAY return an implementation-defined value.
Historically, user agents have granted content from the file URI scheme a tremendous amount of privilege. However, granting all local files such wide privileges can lead to privilege escalation attacks. Some user agents have had success granting local files directory-based privileges, but this approach has not been widely adopted. Other user agents use globally unique identifiers for each file URI, which is the most secure option.
There are many security considerations for URI schemes discussed in [RFC3986].
File access and the granting of privileges for specific operations are complex topics, and the use of file URIs can complicate the security model in effect for file privileges. Software using file URIs MUST NOT grant greater access than would be available for other file access methods.
File systems typically assign an operational meaning to special characters, such as the “/”, “\”, “:”, “[”, and “]” characters, and to special device names like “.”, “..”, “…”, “aux”, “lpt”, etc. In some cases, merely testing for the existence of such a name will cause the operating system to pause or invoke unrelated system calls, leading to significant security concerns regarding denial of service and unintended data transfer. It would be impossible for this specification to list all such significant characters and device names. Implementers MUST research the reserved names and characters for the types of storage device that may be attached to their application and restrict the use of data obtained from URI components accordingly.
Additionally, as discussed in the HP OpenVMS Systems Documentation <http://h71000.www7.hp.com/doc/84final/ba554_90015/ch03s09.html> “access control strings include sufficient information to allow someone to break in to the remote account, [therefore] they create serious security exposure.” In a similar vein, the presence of a password in a “user:password” userinfo field is deprecated by [RFC3986]. As such, the userinfo field of a file URI, if present, MUST NOT contain a password.
This document defines the following URI scheme, so the “Permanent URI Schemes” registry has been updated accordingly. This registration complies with [BCP35].
This specification is derived from [RFC1738], [RFC3986], and [I-D.hoffman-file-uri] (expired); the acknowledgements in those documents still apply.
Additional thanks to Dave Risney, author of the informative IE Blog article <http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx>, and Dave Thaler for their comments and suggestions.
The syntax in Section 2 is intended to support file URIs that take the following forms:
Local files:
Non-local files:
This appendix is not normative; it highlights some observed behaviours and provides system-specific guidance for interacting with file URIs and paths.
There is little to say about POSIX file systems; the file URI structure already closely resembles POSIX file paths.
When mapping a DOS- or Windows-like file path to a file URI, implementations typically map the drive letter (e.g. “c:”) into the first path segment.
See Appendix C.1 for explicit (but non-normative and strictly optional) rules for interacting with DOS- or Windows-like file paths and URIs.
The HFS+ file system uses a non-standard normalization form, similar to Normalization Form D. Take care when transforming HFS+ file paths to and from URIs using Normalization Form C Section 4.
When mapping a VMS file path to a file URI, map the device name into the first path segment. Note that the dollars sign “$” is a reserved character per the definition in [RFC3986], Section 2.2, so should be percent-encoded if present in the device name.
If the VMS file path includes a node reference, use that as the authority. Where the original node reference includes a username and password in an access control string, they can be transcribed into the userinfo field of the authority ([RFC3986], Section 3.2.1), security considerations (Section 6) notwithstanding.
These variations may be encountered for historical reasons, but are not supported by the normative syntax of Section 2.
This appendix is not normative.
On Windows- or DOS-based file systems a absolute file path can begin with a drive letter. To facilitate this, the local-path rule in Section 2 can be replaced with the following:
local-path = [ drive-letter ] path-absolute drive-letter = ALPHA ":"
This is intended to support the minimal representation of a local file in a DOS- or Windows-based environment, with no authority field and an absolute path that begins with a drive letter. E.g.:
URIs of the form file:///c:/path/to/file are already supported by the path-absolute rule.
Note that comparison of drive letters in DOS or Windows file paths is case-insensitive. Some implementations therefore canonicalize drive letters in file URIs by converting them to uppercase.
In DOS- or Windows-based file systems, relative paths beginning with a slash “/” should be resolved relative to the drive letter, and resolution of “..” dot segments (per Section 5.2.4 of [RFC3986]) should not ever overwrite the drive letter.
e.g.:
base: file:///c:/path/to/file.txt rel. URI: /some/other/thing.bmp resolved: file:///c:/some/other/thing.bmp base: file:///c:/foo.txt rel. URI: ../../bar.txt resolved: file:///c:/bar.txt
Relative paths with a drive letter followed by a character other than a slash (e.g. “c:bar/baz.txt” or “c:../foo.txt”) should not be accepted as dereferenceable URIs in DOS or Windows systems.
Historically some implementations have used a vertical line character “|” instead of a colon “:” in the drive letter construct. [RFC3986] forbids the use of the vertical line, however it may be necessary to interpret or update old URIs.
For interpreting such URIs, the auth-path and local-path rules in Section 2 and the drive-letter rule above are replaced with the following:
auth-path = [ file-auth ] path-absolute
/ [ file-auth ] file-absolute
local-path = [ drive-letter ] path-absolute
/ file-absolute
file-absolute = "/" drive-letter path-absolute
drive-letter = ALPHA ":"
/ ALPHA "|"
This is intended to support regular DOS or Windows file URIs with vertical line characters in the drive letter construct. E.g.:
To update such an old URI, replace the vertical line “|” with a colon “:”.
A UNC filespace selector string can be directly translated to a URI; see Section 4. The following is an algorithmic description of the process of translating a UNC string to a file URI:
Example:
UNC String: \\host.example.com\Share\path\to\file.txt URI: file://host.example.com/Share/path/to/file.txt
It is common to encounter file URIs that encode entire UNC strings in the path, usually with all backslash “\” characters replaced with slashes “/”.
To interpret such URIs, the auth-path rule in Section 2 is replaced with the following:
auth-path = [ file-auth ] path-absolute
/ unc-authority path-absolute
unc-authority = 2*3"/" [ userinfo "@" ] file-host
file-host = inline-IP / IPv4address / reg-name
inline-IP = "%5B" ( IPv6address / IPvFuture ) "%5D"
This syntax uses the userinfo, IPv4address, IPv6address, IPvFuture, and reg-name` rules from [RFC3986].
This extended syntax is intended to support URIs that take the following forms, in addition to those in Appendix A:
Non-local files:
This representation is notably used by the Firefox web browser. See Bugzilla#107540
[Bug107540].It also further limits the set of file URIs that can be translated to a local file path to those with a path that does not encode a UNC string.
Historically some implementations have copied entire file paths into the path components of file URIs. Where DOS or Windows file paths were copied thus, resulting URI strings contained unencoded backslash “\” characters, which are forbidden by both [RFC1738] and [RFC3986].
It may be possible to translate or update such an invalid file URI by replacing all backslashes “\” with slashes “/”, if it can be determined with reasonable certainty that the backslashes are intended as path separators.
The following examples demonstrate the advantage of encoding file URIs as IRIs to avoid ambiguity (see Section 4).
Example: file IRI:
| Bytes of file IRI in a UTF-8 document: | 66 69 6c 65 3a 43 3a 2f 72 65 c3 a7 75 2e 74 78 74 | f i l e : c : / r e ( c ) u . t x t | | Interpretation: | A file named "recu.txt" with a cedilla on the "c", in the | directory "C:\" of a DOS or Windows file system. | | Character value sequences of file paths, for various file system | encodings: | | o UTF-16 (e.g. NTFS): | 0043 003a 005c 0072 0065 00e7 0075 002e 0074 0078 0074 | | o Codepage 437 (e.g. MS-DOS): | 43 3a 5c 72 65 87 75 2e 74 78 74
Counter-example: ambiguous file URI:
| Percent-encoded file URI, in any ASCII-compatible document: | "file:///%E3%81%A1" | | Possible interpretations of the file name, depending on the | encoding of the file system: | | o UTF-8: | <HIRAGANA LETTER TI (U+3061)> | | o Codepage 437: | <GREEK SMALL LETTER PI (U+03C0)> + | <LATIN SMALL LETTER U WITH DIAERESIS (U+00FC)> + | <LATIN SMALL LETTER I WITH ACUTE (U+00ED)> | | o EBCDIC: | "Ta~" | | etc.
The UNC filespace selector string is a null-terminated sequence of characters from the Universal Character Set [ISO10646].
The syntax of a UNC filespace selector string, as defined by [MS-DTYP], is given here in Augmented Backus-Naur Form (ABNF) [RFC5234] for convenience. Note that this definition is informative only; the normative description is in [MS-DTYP].
UNC = "\\" hostname "\" sharename *( "\" objectname ) hostname = netbios-name / fqdn / ip-address sharename = <name of share or resource to be accessed> objectname = <depends on resource being accessed>
The precise format of sharename depends on the protocol; see: SMB [MS-SMB], NFS [RFC7530], NCP [NOVELL].
Here are the collected syntax rules for all optional appendices, presented for convenience. This collected syntax is not normative.
file-URI = file-scheme ":" file-hier-part
file-scheme = "file"
file-hier-part = "//" auth-path
/ local-path
auth-path = [ file-auth ] path-absolute
/ [ file-auth ] file-absolute
/ unc-authority path-absolute
local-path = [ drive-letter ] path-absolute
/ file-absolute
file-auth = [ userinfo "@" ] host
unc-authority = 2*3"/" [ userinfo "@" ] file-host
file-host = inline-IP / IPv4address / reg-name
inline-IP = "%5B" ( IPv6address / IPvFuture ) "%5D"
file-absolute = "/" drive-letter path-absolute
drive-letter = ALPHA ":"
/ ALPHA "|"
This collected syntax is intended to support file URIs that take the following forms:
Local files:
Non-local files: