Network Working Group A. Phillips, Ed.
Internet-Draft webMethods, Inc.
Expires: June 16, 2004 M. Davis
IBM
December 17, 2003
Tags for Identifying Languages
draft-phillips-langtags-00
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on June 16, 2004.
Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract
This document describes a language tag for use in cases where it is
desired to indicate the language used in an information object, how
to register values for use in this language tag, and a construct for
matching such language tags, including user defined extensions for
private interchange.
Phillips & Davis Expires June 16, 2004 [Page 1]
Internet-Draft langtags December 2003
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. The Language Tag . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Language Tag Sources . . . . . . . . . . . . . . . . . . . . 4
2.2.1 Pre-Existing RFC3066 Registrations . . . . . . . . . . . . . 8
2.2.2 Possibilities for Registration . . . . . . . . . . . . . . . 8
2.3 Choice of Language Tag . . . . . . . . . . . . . . . . . . . 9
2.4 Meaning of the Language Tag . . . . . . . . . . . . . . . . 12
2.4.1 Language Range . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.2 Matching Language Tags . . . . . . . . . . . . . . . . . . . 14
3. IANA Considerations . . . . . . . . . . . . . . . . . . . . 16
3.1 Registration Procedure for Subtags . . . . . . . . . . . . . 16
4. Security Considerations . . . . . . . . . . . . . . . . . . 19
5. Character Set Considerations . . . . . . . . . . . . . . . . 20
6. Changes from RFC3066 . . . . . . . . . . . . . . . . . . . . 21
References . . . . . . . . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 24
A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25
B. Examples of Language Tags (Informative) . . . . . . . . . . 26
Intellectual Property and Copyright Statements . . . . . . . 28
Phillips & Davis Expires June 16, 2004 [Page 2]
Internet-Draft langtags December 2003
1. Introduction
Human beings on our planet have, past and present, used a number of
languages. There are many reasons why one would want to identify the
language used when presenting information.
Information about a user's preferred languages commonly needs to be
identified so that appropriate processing can be applied. For
example, the user's language preferences in a brower can be used to
select web pages appropriately. A choice of language preference can
also be used to select among tools (such as dictionaries) to assist
in the processing or understanding of content in different langauges.
In addition, knowledge about the particular language of information
content may be useful or even required by some types of information
processing; for example spell-checking, computer-synthesized speech,
Braille transcription, or high-quality print renderings.
One means of indicating the language used is by labeling the
information content with a language identifier. These identifiers can
also be used to specify user preferences when selecting information
content, or for labeling additional attributes of content and
associated resources.
These identifiers can also be used to indicate additional attributes
of content that are closely related to the language. In particular,
it is often necessary to indicate specific information about the
dialect, writing system, or orthography used in a document or
resource, as these attributes may be important for the user to obtain
information in a form that they can understand, or important in
selecting appropriate processing resources for the given content.
This document specifies an identifier mechanism, a registration
function for values to be used with that identifier mechanism, and a
construct for matching against those values. It also defines a
mechanism for private use extension and how private use, registered
values, and matching interact.
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC 2119] [9].
Phillips & Davis Expires June 16, 2004 [Page 3]
Internet-Draft langtags December 2003
2. The Language Tag
2.1 Syntax
The language tag is composed of one or more parts: A primary language
subtag and a (possibly empty) series of subsequent subtags. The
sequence of subtags has a specific structure that depends on the
length of the subtag to distinguish each tag type.
The syntax of this tag in ABNF [RFC 2234] [10] is:
= lang ["-" script] ["-" region] *("-" variant) [extensions]
=/ "x-" ALPHA * alphanumdash ; private use tag
=/ grandfathered-registrations
lang = 2*3 ALPHA ; shortest ISO 639 tag
=/ registered-lang
script = 4 ALPHA ; ISO 15924 tag
region = 2*3 ALPHA ; shortest ISO 3166 tag
variant = 5*16 alphanum
registered-lang = 5*16 alphanum
extensions = "-x" 1* ("-" key "." value)
key = ALPHA *alphanum
value = 1* utf8uri
grandfathered-registrations = ALPHA * (alphanumdash)
alphanum = (ALPHA / DIGIT)
alphanumdash = (ALPHA / DIGIT / "-")
utf8uri = (ALPHA / DIGIT / 1*4 ("%" 2 HEXDIG))
Figure 1: Language Tag ABNF
The character "-" is HYPHEN-MINUS (ABNF: %x2D). The character "." is
FULL STOP (ABNF: %x2E). The character "%" is PERCENT SIGN (ABNF:
%x25).
All tags are to be treated as case insensitive: there exist
conventions for the capitalization of some of them, but these should
not be taken to carry meaning. For instance, [ISO 3166] [4]
recommends that country codes be capitalized (MN Mongolia), while
[ISO 639] [3] recommends that language codes be written in lower case
(mn Mongolian).
For examples, see Appendix B at the end of this document.
2.2 Language Tag Sources
The namespace of language tags is administered by the Internet
Assigned Numbers Authority (IANA) [14] according to the rules in
Section 3.1.
Phillips & Davis Expires June 16, 2004 [Page 4]
Internet-Draft langtags December 2003
Terminology in this section:
o Tag or tags refers to a complete language tag, such as
'fr-Latn-CA'
o Subtag refers to a specific section of a tag, separated by hyphen,
such as 'Latn' in 'fr-Latn-CA'
o Code or codes refers to the tags defined in external standards.
For example, 'Latn' is an ISO 15924 [2] script code (which can be
used as a script subtag in a language tag)
The rules in this section apply to the various subtags within the
language tags defined in this document, excepting those
"grandfathered" tags defined in Section 2.2.1. Those tags should be
considered as exceptions to the rules presented here.
The following rules apply to the primary (language) subtag:
o All 2-character language subtags are interpreted according to
assignments found in ISO standard 639, "Code for the
representation of names of languages" [ISO639-1] [3], or
assignments subsequently made by the ISO 639 Part 1 maintenance
agency or governing standardization bodies.
o All 3-character language subtags are interpreted according to
assignments found in ISO 639 part 2, "Codes for the representation
of names of languages -- Part 2: Alpha-3 code [ISO 639-2] [1]", or
assignments subsequently made by the ISO 639 part 2 maintenance
agency or governing standardization bodies, or assignments of
3-character disambiguation registrations according to Rule 7a.
Ambiguity in Section 2.3 of this document.
o ISO639-2 reserves for private use codes the range 'qaa' through
'qtz'. These codes should be used for non-registered language
subtags.
o Subtags of 5 to 16 characters may be registered with IANA,
according to the rules in Section 3.1 of this document. (Note that
previously, in rfc3066 [15], the IANA registry contained whole tag
registrations such as de-CH-1994, whereas this document refers to
the registration of subtags such as 'klingon')
o The single character subtag "x" as the primary subtag indicates
that the whole language tag is a private use tag. The value and
semantic meaning of such a tag as a whole and of the subtags used
within such as tag are undefined by this document.
Phillips & Davis Expires June 16, 2004 [Page 5]
Internet-Draft langtags December 2003
o Other values shall not be assigned except by revision of this
document.
The following rules apply to the script subtags:
o All 4-character subtags are interpreted as ISO 15924 alpha-4
script codes from [2], or subsequently assigned by the ISO 15924
maintenance agency or governing standardization bodies, denoting
the script or writing system used in conjunction with this
language. These alpha4 tags may only occur as the second subtag in
a tag. For example: 'de-Latn' represents German written using
Latin script.
o ISO 15924 reserves the codes Qaaa-Qtzz for private use values.
These codes should be used for non-registered script values.
o Script subtags can NOT be registered using the process in Section
3.1 of this document. Variant subtags may be considered for
registration for that purpose. Note that registered subtags can
only appear in the first position or as a sequence on the end of a
language tag. This makes the associated exception handling when
parsing tags simpler.
The following rules apply to the region subtags:
o All 2-character and 3-character subtags following the primary
subtag denote the area to which this language variant relates, and
are interpreted according to assignments found in ISO 3166 alpha-
2 (or alpha-3) country codes from [4], assignments subsequently
made by the ISO 3166 maintenance agency or governing
standardization bodies, or assignments of 3-character
disambiguation registrations according to Rule 7a. [Ambiguity] in
Section 2.3 of this document.
o Region subtags must occur after any script subtags and before any
variant subtags or extensions. The shortest form ISO3166 code must
be used to form the subtag. At the time this document was written,
all alpha3 codes had a corresponding alpha2 code. Use of alpha3
codes is provided for a foreseeable future in which alpha2 codes
have been exhausted or for resolution of ambiguities. Example:
'de-Latn-CH' represents German written using Latin script for
Switzerland.
o ISO 3166 reserves the country codes AA, QM-QZ, XA-XZ and ZZ (plus
any three-character sequences starting with these codes) as
user-assigned codes. These codes should be used for private use
region subtags.
Phillips & Davis Expires June 16, 2004 [Page 6]
Internet-Draft langtags December 2003
o No region subtags can be registered using the process in Section
3.1 of this document. Variant subtags may be considered for
registration for this purpose.
The following rules apply to the variant subtags:
o Alphanumeric subtags of 5 to 16 characters may be registered with
IANA, according to the rules in Section 3.1 of this document.
Registered subtags must not begin with the character 'x', which is
reserved for private use subtags. (Note that previously, in
rfc3066, the IANA registry contained whole tag registrations such
as 'en-boont', whereas this document refers to the registration of
subtags such as 'boont')
o Alphanumeric subtags of 5 to 16 characters starting with 'x' are
reserved for private use. The semantics of these subtags must be
defined by the end users of such subtags and the semantic meaning
should be considered external to this document.
The following rules apply to the extensions:
o Extension subtags are separated from the other subtags defined in
this document by the single character subtag "x".
o Extensions must follow all language, script, region, and variant
subtags.
o Extensions consist of key-value pairs. Each key-value pair is
separated using the HYPHEN-MINUS character. The key and value are
separated by FULL STOP (also called 'period' or 'dot').
o The key must consist of an alphanumeric sequence of any length
starting with a letter (alpha character). It may not be empty.
o The value must consist of a non-empty sequence of UTF-8 [RFC3629]
[16] characters, encoded using the URI encoding rules in Section
2.1 of [RFC 2396] [11]. Note that alphanumeric values will be
encoded as themselves. So the value "abc" is encoded as "abc",
while the character LATIN SMALL LETTER A WITH GRAVE (U+00E0, ABNF
%xE0) is encoded as "%c3%a0".
o No source is defined for extensions. External agreement or
standardization of extension subtags is by private agreement and
should not be considered part of this document.
For example: 'az-Arab-x-SIL.AZE-dialect.derbend' contains two
extension subtags. The first is "SIL.AZE" and the second is
"dialect.derbend".
Phillips & Davis Expires June 16, 2004 [Page 7]
Internet-Draft langtags December 2003
2.2.1 Pre-Existing RFC3066 Registrations
Existing IANA registered language tags from RFC1766/RFC3066 that are
not defined by additions to this document maintain their validity.
IANA will maintain these tags, adding a notation that they are
"grandfathered from RFC 3066".
The rules governing existing RFC 1766 and RFC 3066 registered tags
are:
o If the formerly registered tag would now be defined by this
document, then the existing tag is marked as superseded by this
document and no subtag will be registered as a result. For
example, zh-Hans would now be defined by the addition of ISO 15924
script codes.
o If the registered tag contained one or more subtags that follow
the guidelines for registered language or variant subtags, and all
of the subtags are either now defined by this document or would be
valid to register, then each subtag not already covered by this
document will be registered automatically by IANA without further
review and the existing tag marked as superseded by this document.
For example: the tag 'en-boont' fits the pattern for a registered
variant. The variant subtag "boont" will be registered
automatically and 'en-boont' marked as superseded.
o If the registered tag contains any subtags that are not otherwise
valid for registration according to the rules in this document,
then the tag as a whole is maintained as an exceptional case (that
is, it is "grandfathered"). This includes special cases of Sign
Language tags. For example, the tag 'i-klingon' is not covered by
any addition and is grandfathered, as is sgn-BE-fr (Belgian French
Sign Language).
Users of tags that are grandfathered should consider registering the
appropriate subtags using the new format (but are not required to).
2.2.2 Possibilities for Registration
Possibilities for registration of subtags include:
o Languages not listed in ISO 639 that are not variants of any
listed language, can be registered, such as tsolyani. Before
attempting to register a language subtag, there should be a good
faith attempt to register the language with ISO 639. No language
subtags will be registered for codes that exist in ISO 639-1 or
ISO 639-2.
Phillips & Davis Expires June 16, 2004 [Page 8]
Internet-Draft langtags December 2003
o Dialect or other divisions or variations within a language, its
orthography, writing system, regional variation, or historical
usage, such as the "scouse" subtag (the Scouse dialect of
English).
This document leaves the decision on what subtags are appropriate or
not to the registration process described in Section 3.1.
ISO 639 defines a maintenance agency for additions to and changes in
the list of languages in ISO 639. This agency is:
International Information Centre for Terminology (Infoterm)
Aichholzgasse 6/12, AT-1120
Wien, Austria
Phone: +43 1 26 75 35 Ext. 312 Fax: +43 1 216 32 72
ISO 639-2 defines a maintenance agency for additions to and changes
in the list of languages in ISO 639-2. This agency is:
Library of Congress
Network Development and MARC Standards Office
Washington, D.C. 20540 USA
Phone: +1 202 707 6237 Fax: +1 202 707 0115
URL: http://www.loc.gov/standards/iso639
The maintenance agency for ISO 3166 (country codes) is:
ISO 3166 Maintenance Agency
c/o International Organization for Standardization
Case postale 56
CH-1211 Geneva 20 Switzerland
Phone: +41 22 749 72 33 Fax: +41 22 749 73 49
URL: http://www.iso.org/iso/en/prods-services/iso3166ma/index.html
The registration authority for ISO 15924 (script codes) is:
Unicode Consortium Box 391476
Mountain View, CA 94039-1476, USA
URL: http://www.unicode.org/iso15924
2.3 Choice of Language Tag
One may occasionally be faced with several possible tags for the same
body of text.
Interoperability is best served if all users send the same tag, and
use the same tag for the same language for all documents. If an
application has requirements that make the rules here inapplicable,
Phillips & Davis Expires June 16, 2004 [Page 9]
Internet-Draft langtags December 2003
the application protocol specification MUST specify how the procedure
varies from the one given here.
The text below is based on the set of tags known to the tagging
entity.
1. Use as precise a tag as possible, but no more specific than is
justified. For example, 'de' might suffice for tagging an email
written in German, while 'de-CH-1996' is probably unnecessarily
precise for such a task.
2. Avoid using subtags that add no distinguishing information about
the content. For example, the script subtag in 'en-Latn-US' is
generally unnecessary, since nearly all English texts are written
in the Latin script.
3. When a language has both an ISO 639-1 2-character code and an ISO
639-2 3-character code, you MUST use the ISO 639-1 2-character
code.
4. When a language has no ISO 639-1 2-character code, and the ISO
639- 2/T (Terminology) code and the ISO 639-2/B (Bibliographic)
codes differ, you MUST use the Terminology code. NOTE: At present
all languages that have both kinds of 3-character code also are
assigned a 2-character code, and the displeasure of developers
about the existence of two different code sets has been
adequately communicated to ISO. So this situation will hopefully
not arise.
5. You SHOULD NOT use the UND (Undetermined) code unless the
protocol in use forces you to give a value for the language tag,
even if the language is unknown. Omitting the tag is preferred.
6. You SHOULD NOT use the MUL (Multiple) tag if the protocol allows
you to use multiple languages, as is the case for the
Content-Language header in HTTP.
NOTE: In order to avoid versioning difficulties in
applications such as that experienced in RFC 1766 [7], the ISO
639 Registration Authority Joint Advisory Committee (RA-JAC)
has agreed on the following policy statement:
"After the publication of ISO/DIS 639-1 as an International
Standard, no new 2-letter code shall be added to ISO 639-1
unless a 3-letter code is also added at the same time to ISO
639-2. In addition, no language with a 3-letter code available
at the time of publication of ISO 639-1 which at that time had
no 2-letter code shall be subsequently given a 2-letter code."
Phillips & Davis Expires June 16, 2004 [Page 10]
Internet-Draft langtags December 2003
This will ensure that, for example, a user who implements
"haw" (Hawaiian), which currently has no 2-character code,
will not find his or her data invalidated by eventual addition
of a 2-character code for that language."
7. To maintain backwards compatibility, there are two provisions to
account for potential instability in ISO 639, 3166, and 15924
codes.
a) Ambiguity. In the event that one of these standards assigns a
code a new meaning or reassigns a deprecated code, the new use
of the code will not be permitted in language tags defined by
this document.
The language subtag reviewer, as described in Section 3, shall
prepare a proposal for entering in the IANA registry, as soon
as practical, a variant subtag as a surrogate value for the
new code. The form of the registered variant should be either
an ISO3166-2 alpha3 tag (if that does not itself have
ambiguity problems) or be a 3-character tag that is not
otherwise permitted by the ISO registration authority, such as
one containing a sequence number (e.g. CS1). However, should
the ISO registration authority or standard expand the
allowable 3-character tags so that this is not possible, then
this may be any other valid registration (such as CS2003,
marking the year of introduction).
The normal registration process described in Section 3.1 of
this document applies to the review and registration of such
variant subtags, except that they may be 3 characters long, as
described above. Note that these subtags should never be used
in combination with the subtag type for which they are a
surrogate. For example, a "region" variant subtag should not
be used with a region subtag.
For example:
cs-CS (Czech for Czechoslovakia)
sr-CS1 (Serbian for Serbia and Montenegro, using a
registered variant)
sr-CS-CS1 (Incorrect usage)
qx-Latn (hypothetical reassigned value 'qx')
Phillips & Davis Expires June 16, 2004 [Page 11]
Internet-Draft langtags December 2003
qx2003-Latn (hypothetical registered language subtag)
b) Stability. All other ISO codes are valid, even if they have
been deprecated. Some examples, current at the time this
document was drafted, are listed below. Where a new equivalent
code has been defined (given below on the right side after a
tilde), implementations should treat these tags as identical.
For example, some deprecated ISO 639 codes:
iw ~ he
in ~ id
ji ~ yi
sh
For example, some deprecated ISO 3166 codes:
FX
TP ~ TL
YU
2.4 Meaning of the Language Tag
The language tag always defines a language as spoken (or written,
signed or otherwise signaled) by human beings for communication of
information to other human beings. Computer languages such as
programming languages are explicitly excluded.
If a language tag B contains language tag A as a prefix, then B is
typically "narrower" or "more specific" than A. For example,
'zh-Hant-TW' is more specific than 'zh-Hant'.
This relationship is not guaranteed in all cases: specifically,
languages that begin with the same sequence of subtags are NOT
guaranteed to be mutually intelligible, although they may be. For
example, the tag 'az' shares a prefix with both 'az-Latn' (Azeri
written using the Latin script) and 'az-Cyrl' (Azeri written using
the Cyrillic script). A person fluent in one script may not be able
to read the other, even though the text might be identical. Content
tagged as 'az' most probably is written in just one script and thus
might not be intellible to a reader familiar with the other script.
Phillips & Davis Expires June 16, 2004 [Page 12]
Internet-Draft langtags December 2003
The relationship between the tag and the information it relates to is
defined by the standard describing the context in which it appears.
Accordingly, this section can only give possible examples of its
usage.
o For a single information object, it could be taken as the set of
languages that is required for a complete comprehension of the
complete object. Example: Plain text documents.
o For an aggregation of information objects, it should be taken as
the set of languages used inside components of that aggregation.
Examples: Document stores and libraries.
o For information objects whose purpose is to provide alternatives,
the set of tags associated with it should be regarded as a hint
that the content is provided in several languages, and that one
has to inspect each of the alternatives in order to find its
language or languages. In this case, a tag with multiple languages
does not mean that one needs to be multi-lingual to get complete
understanding of the document. Example: MIME multipart/
alternative.
o In markup languages, such as HTML and XML, language information
can be added to each part of the document identified by the markup
structure (including the whole document itself). For example, one
could write C'est la vie. inside a
Norwegian document; the Norwegian-speaking user could then access
a French-Norwegian dictionary to find out what the marked section
meant. If the user were listening to that document through a
speech synthesis interface, this formation could be used to signal
the synthesizer to appropriately apply French text-to-speech
pronunciation rules to that span of text, instead of misapplying
the Norwegian rules.
2.4.1 Language Range
A Language Ranges is a set of languages whose tags all begin with the
same sequence of subtags. The following definition of language-range
is derived from HTTP/1.1 [13].
language-range = language-tag / "*"
That is, a language-range has the same syntax as a language-tag, or
is the single character "*" and implicitly assumes that there is a
semantic relationship between tags that share the same subtag
prefixes.
Phillips & Davis Expires June 16, 2004 [Page 13]
Internet-Draft langtags December 2003
A language-range matches a language-tag if it exactly equals the tag,
or if it exactly equals a prefix of the tag such that the first
character following the prefix is "-".
The special range "*" matches any tag. A protocol which uses language
ranges may specify additional rules about the semantics of "*"; for
instance, HTTP/1.1 specifies that the range "*" matches only
languages not matched by any other range within an "Accept-Language:"
header.
As noted above, not all languages or content denoted by a specific
language-range may be mutually intelligible and this use of a prefix
matching rule does not imply that language tags are assigned to
languages in such a way that it is always true that if a user
understands a language with a certain tag, then this user will also
understand all languages with tags for which this tag is a prefix.
The prefix rule simply allows the use of prefix tags if this is the
case.
2.4.2 Matching Language Tags
Implementations that are searching for content or otherwise matching
language tags to a language-range [Section 2.4.1] may choose to
assume that there is a semantic relationship between two tags that
share common prefixes. This is called 'language tag fallback'. The
most common implementations follow this pattern:
1. When searching for content using language tag fallback, the
language tag is progressively truncated from the end until a
match is located. For example, starting with the tag
'en-US-boont', searchs or matches would first be performed with
the whole tag, then with 'en-US', and finally with 'en'. This
allows some flexibility in finding content in accordance with
Rules 1 and 2 in Section 2.3; allows better maintenance; and
usually provides better results when data is not available at a
specific level of tag granularity or is sparsely populated (than
if the default language for the system or content were used). Any
implementation that uses this technique should ensure that
appropriate data is available on each level.
Phillips & Davis Expires June 16, 2004 [Page 14]
Internet-Draft langtags December 2003
Tag to match: en-US-boont
1. en-US-boont
2. en-US
3. en
Figure 2: Default Fallback Pattern Example
2. The extension mechanism is orthogonal to language tag fallback.
Implementations may choose to ignore the extensions and follow
the default fallback pattern (above). Thus matching the tag
"en-US-boont-x-collation.traditional" would be exactly the same
as the example above. However, an implementation that interpreted
one or more extension subtags could either choose a different
fallback pattern, or use the extensions to interpret content in a
different fashion.
Phillips & Davis Expires June 16, 2004 [Page 15]
Internet-Draft langtags December 2003
3. IANA Considerations
This section deals with the registration of subtags for use in
language tags defined by this document, in accordance with the
requirements of RFC2434 [12].
3.1 Registration Procedure for Subtags
The procedure given here MUST be used by anyone who wants to use a
subtag not given an interpretation in Section 2.2 of this document or
previously registered with IANA.
This procedure MAY also be used to register information with the IANA
about a tag or subtag defined by this document, for instance if one
wishes to make publicly available a reference to the definition for a
language such as sgn-US (American Sign Language), or additional
information about a registration previously made via this procedure.
Variant subtags MUST NOT be registered using the pattern 2 ALPHA 4
DIGIT to accommodate the provisions in Section 2.3, rule 7a of this
document. That is, the subtag yx1234 can NOT be registered except
under the aforementioned provisions.
Subtags MUST NOT be registered that start with the letter 'x', since
this prefix is reserved for Private Use subtags.
The process starts by filling out the registration form reproduced
below.
LANGUAGE SUBTAG REGISTRATION FORM
Name of requester:
E-mail address of requester:
Subtag to be registered:
Type of Subtag: [ ] language [ ] variant
Full English name of subtag:
Intended meaning of the subtag:
If variant subtag, the intended prefix(es) of subtag:
Native name of language (transcribed into ASCII):
Reference to published description of the language (book or article):
Any other relevant information:
Figure 3
The subtag registration form MUST be sent to
for a two week review period before it can
be submitted to IANA. (This is an open list. Requests to be added
should be sent to .)
Phillips & Davis Expires June 16, 2004 [Page 16]
Internet-Draft langtags December 2003
Variant subtags are generally registered for use with a particular
prefix or set of prefixes. For example, the subtag 'boont' is
intended for use with the prefix 'en-', since Boontling is a dialect
of English. This information MUST be provided in the registration
form.
Any registered subtag MAY be incorporated into a variety of language
tags, according to the rules of Section 2.1. This makes validation
simpler and thus more uniform across implementations, and does not
require new registrations for different intended prefixes.
However, the intended prefixes for a given registered subtag will be
maintained in the IANA registry as a guide to usage. If it is
necessary to add an additional intended prefix to that list for an
existing language tag, that can be done by filing an additional
registration form. In that form, the "Any other relevant information:
" field should indicate that it is the addition of an additional
intended prefix.
When the two week period has passed, the subtag reviewer, who is
appointed by the IETF Applications Area Director, either forwards the
request to IANA@IANA.ORG, or rejects it because of significant
objections raised on the list. Note that the reviewer can raise
objections on the list himself, if he or she so desires. The
important thing is that the objection must be made publicly.
The applicant is free to modify a rejected application with
additional information and submit it again; this restarts the two
week comment period.
Decisions made by the reviewer may be appealed to the IESG [RFC 2028]
[8] under the same rules as other IETF decisions [RFC 2026] [17]. All
registered forms are available online in the directory http://
www.iana.org/numbers.html under "languages".
Updates of registrations follow the same procedure as registrations.
The subtag reviewer decides whether to allow a new registrant to
update a registration made by someone else; normally objections by
the original registrant would carry extra weight in such a decision.
Registrations are permanent and stable. When some registered subtag
should not be used any more, for instance because a corresponding ISO
639 code has been created, the registration should be amended by
adding a remark like "DEPRECATED: use instead" to the
"other relevant information" section.
Note: The purpose of the "published description" is intended as an
aid to people trying to verify whether a language is registered, or
Phillips & Davis Expires June 16, 2004 [Page 17]
Internet-Draft langtags December 2003
what language a particular subtag refers to. In most cases, reference
to an authoritative grammar or dictionary of that language will be
useful; in cases where no such work exists, other well known works
describing that language or in that language may be appropriate. The
subtag reviewer decides what constitutes "good enough" reference
material.
Phillips & Davis Expires June 16, 2004 [Page 18]
Internet-Draft langtags December 2003
4. Security Considerations
The only security issue that has been raised with language tags since
the publication of RFC 1766, which stated that "Security issues are
believed to be irrelevant to this memo", is a concern with language
ranges used in content negotiation - that they may be used to infer
the nationality of the sender, and thus identify potential targets
for surveillance.
This is a special case of the general problem that anything you send
is visible to the receiving party. It is useful to be aware that such
concerns can exist in some cases.
The evaluation of the exact magnitude of the threat, and any possible
countermeasures, is left to each application protocol.
Phillips & Davis Expires June 16, 2004 [Page 19]
Internet-Draft langtags December 2003
5. Character Set Considerations
Language tags may always be presented using the characters A-Z, a-z,
0-9, FULL STOP, PERCENT SIGN, and HYPHEN-MINUS, which are present in
most character sets, so presentation of language tags should not have
any character set issues.
The issue of deciding upon the rendering of a character set based on
the language tag is not addressed in this memo; however, it is
thought impossible to make such a decision correctly for all cases
unless means of switching language in the middle of a text are
defined (for example, a rendering engine that decides font based on
Japanese or Chinese language may produce sub-optimal output when a
mixed Japanese- Chinese text is encountered)
The extension subtags allow the use of non-ASCII characters via an
escaping mechanism. The non-ASCII characters are always percent
encoded using the UTF-8 encoding of the Unicode character set. Tags
may occasionally be rendered for human consumption using the
underlying character values rather than the UTF-8 escape sequence,
but this should be considered a presentation form and not the value
used for interchange.
Phillips & Davis Expires June 16, 2004 [Page 20]
Internet-Draft langtags December 2003
6. Changes from RFC3066
The main goals were to maintain backward compatibility (so that all
previous codes would remain valid); reduce the need for large numbers
of registrations; to provide a more formal structure to allow parsing
into subtags even where software does not have the latest
registrations; to provide stability in the face of potential
instability in ISO 639, 3166, and 15924 codes (*demonstrated
instability* in the case of ISO 3166); and to allow for external
extension mechanisms.
o Allows ISO15924 script code subtags and allows them to be used
generatively.
o Adds the concept of a variant subtag and allows variants to be
used generatively.
o Adds an extension mechanism which does not require registration to
use.
o Defines the private use tags in ISO639, ISO15924, and ISO3166 as
the mechanism for creating private use language, script, and
region subtags respectively
o Defines a syntax for private use variant subtags which can be used
without registration.
o Defines a process for handling reuse of values by ISO639,
ISO15924, and ISO3166 in the event that they register a previously
used value for a new purpose.
o Changes the IANA language tag registry to a language subtag
registry
Substantive changes between draft-01 and this version are:
Removed the year subtag
Changed from EQUALS SIGN to FULL STOP in the extension mechanism
Added an IANA Considerations section
Fixed the ABNF
Changed the name of the document.
Updated the introduction, by combining text suggested by Peter
Constable with the existing text and a few of our own revisions.
Phillips & Davis Expires June 16, 2004 [Page 21]
Internet-Draft langtags December 2003
Added a TOC.
Added the subsection Section 2.4.2 on matching language tags.
Revised the rules on choosing languages tags slightly in Section
2.3.
Added this section.
Phillips & Davis Expires June 16, 2004 [Page 22]
Internet-Draft langtags December 2003
References
[1] International Organization for Standardization, "ISO 639-2:1998
- Codes for the representation of names of languages -- Part 2:
Alpha-3 code - edition 1", August 1988.
[2] ISO TC46/WG3 and M. Everson, Ed., "ISO 15924:2003 (E/F) - Codes
for the representation of names of scripts", March 2003.
[3] International Organization for Standardization, "Code for the
representation of names of languages, 1st edition", ISO
Standard 639, 1988.
[4] International Organization for Standardization, "Codes for the
representation of names of countries, 3rd edition", ISO
Standard 3166, August 1988.
[5] Hardcastle-Kille, S., "Mapping between X.400(1988) / ISO 10021
and RFC 822", RFC 1327, May 1992.
[6] Borenstein, N. and N. Freed, "MIME (Multipurpose Internet Mail
Extensions) Part One: Mechanisms for Specifying and Describing
the Format of Internet Message Bodies", RFC 1521, September
1993.
[7] Alvestrand, H., "Tags for the Identification of Languages", RFC
1766, March 1995.
[8] Hovey, R. and S. Bradner, "The Organizations Involved in the
IETF Standards Process", BCP 11, RFC 2028, October 1996.
[9] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[10] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", RFC 2234, November 1997.
[11] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform
Resource Identifiers (URI): Generic Syntax", RFC 2396, August
1998.
[12] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
Considerations Section in RFCs", BCP 26, RFC 2434, October
1998.
[13] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L.,
Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol --
HTTP/1.1", RFC 2616, June 1999.
Phillips & Davis Expires June 16, 2004 [Page 23]
Internet-Draft langtags December 2003
[14] Carpenter, B., Baker, F. and M. Roberts, "Memorandum of
Understanding Concerning the Technical Work of the Internet
Assigned Numbers Authority", RFC 2860, June 2000.
[15] Alvestrand, H., "Tags for the Identification of Languages", BCP
47, RFC 3066, January 2001.
[16] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD
63, RFC 3629, November 2003.
[17]
Authors' Addresses
Addison Phillips (editor)
webMethods, Inc.
432 Lakeside Drive
Sunnyvale, CA 94088
US
EMail: aphillips@webmethods.com
Mark Davis
IBM
EMail: mark.davis@us.ibm.com
Phillips & Davis Expires June 16, 2004 [Page 24]
Internet-Draft langtags December 2003
Appendix A. Acknowledgements
Any list of contributors is bound to be incomplete; please regard the
following as only a selection from the group of people who have
contributed to make this document what it is today.
The contributors to RFC 3066 and RFC 1766, the precursors of this
document, made enormous contributions directly or indirectly to this
document and are generally responsible for the success of language
tags.
The following people (in alphabetical order) contributed to this
document or to RFCs 1766 and 3066:
Glenn Adams, Harald Tveit Alvestrand, Tim Berners-Lee, Marc Blanchet,
Nathaniel Borenstein, Eric Brunner, Sean M. Burke, John Clews, Jim
Conklin, Peter Constable, John Cowan, Mark Crispin, Dave Crocker,
Martin Duerst, Michael Everson, Ned Freed, Tim Goodwin, Dirk-Willem
van Gulik, Marion Gunn, Paul Hoffman, Olle Jarnefors, Kent Karlsson,
John Klensin, Alain LaBonte, Eric Mader, Keith Moore, Chris Newman,
Masataka Ohta, George Rhoten, Markus Scherer, Keld Jorn Simonsen,
Otto Stolz, Tex Texin, Rhys Weatherley, Misha Wolf, Francois Yergeau
and many, many others.
Very special thanks must go to Harald Tveit Alvestrand, who
originated RFCs 1766 and 3066, and without whom this document would
not have been possible. Special thanks must go to Michael Everson,
who has served as language tag reviewer for almost the complete
period since the publication of RFC 1766.
Phillips & Davis Expires June 16, 2004 [Page 25]
Internet-Draft langtags December 2003
Appendix B. Examples of Language Tags (Informative)
Simple language code:
de (German)
fr (French)
ja (Japanese)
Language code plus Script code :
zh-Hant (Traditional Chinese)
en-Latn (English written in Latin script)
sr-Cyrl (Serbian written with Cyrillic script)
Language-Script-Region:
zh-Hans-CN (Simplified Chinese for the PRC)
sr-Latn-CS1 (Serbian, Latin script, Serbia and Montenegro)
Language-Script-Region-Variant:
en-Latn-US-boont (Boontling dialect of English)
Language-Region:
de-DE (German for Germany)
zh-SG (Chinese for Singapore)
cs-CS (Czech for Czechoslovakia)
sr-CS1 (Serbian for Serbia and Montenegro, IANA registered
variant, see 7a in Section 2.3
Other Mixtures:
zh-CN (Chinese for the PRC)
en-boont (Boontling dialect of English)
Extension mechanism:
Phillips & Davis Expires June 16, 2004 [Page 26]
Internet-Draft langtags December 2003
de-CH-x-collation.phonebook
az-Arab-x-SIL.AZE-dialect.derbend
Phillips & Davis Expires June 16, 2004 [Page 27]
Internet-Draft langtags December 2003
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP-11. Copies of
claims of rights made available for publication and any assurances of
licenses to be made available, or the result of an attempt made to
obtain a general license or permission for the use of such
proprietary rights by implementors or users of this specification can
be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assignees.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
Phillips & Davis Expires June 16, 2004 [Page 28]
Internet-Draft langtags December 2003
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Phillips & Davis Expires June 16, 2004 [Page 29]