| Internet-Draft | Centralization and Internet Standards | February 2022 |
| Nottingham | Expires 13 August 2022 | [Page] |
Despite being designed and operated as a decentralized network-of-networks, the Internet is continuously subjected to forces that encourage centralization.¶
This document offers a definition of centralization, explains why it is undesirable, identifies different types of centralization, catalogues limitations of common approaches to controlling it, and explores what Internet standards efforts can do to address it.¶
This note is to be removed before publishing as an RFC.¶
Status information for this document may be found at https://datatracker.ietf.org/doc/draft-nottingham-avoiding-internet-centralization/.¶
Source for this draft and an issue tracker can be found at https://github.com/mnot/avoiding-internet-centralization.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 13 August 2022.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The Internet has succeeded in no small part because of its purposeful avoidance of any single controlling entity. While this approach may reflect a desire to prevent a single technical failure from having wide impact [RAND]. it has also enabled the Internet's rapid adoption and broad spread, because internetworking did not require networks to get permission from or cede control to another entity -- thereby accommodating a spectrum of requirements and positioning the Internet as a public good.¶
While avoiding centralization is a widely shared goal for the Internet, achieving it has proven difficult. Many successful protocols and applications on the Internet today work in a centralized fashion -- to the point where some proprietary, centralized services have become so well-known that they are commonly mistaken for the Internet itself. Even when protocols incorporate techniques intended to prevent centralization, economic and social factors can drive users to prefer centralized solutions built with or on top of supposedly decentralized technology.¶
These difficulties call into question what role architectural regulation -- in particular, open standards bodies such as the IETF -- should play in preventing, mitigating, and controlling Internet centralization.¶
This document discusses aspects of centralization that relate to Internet standards efforts. Section 2 provides a definition of centralization. Section 3 explains why centralization of the Internet's functions is undesirable. Section 4 surveys the different kinds of centralization that might surface on the Internet. Section 5 then catalogues high-level approaches to mitigating centralization and discusses their limitations. Finally, Section 6 considers the role that Internet standards play in avoiding centralization and mitigating its effects.¶
Engineers who design and standardize Internet protocols are the primary audience for this document. However, designers of proprietary protocols can benefit from considering aspects of centralization, especially if they intend their protocol to be considered for eventual standardisation. Likewise, policymakers can use this document to help identify and remedy inappropriately centralized protocols and applications.¶
This document defines "centralization" as the ability of a single entity (e.g., a person, company, or government) -- or a small group of them -- to exclusively observe, capture, control, or extract rent from the operation or use of a Internet function.¶
Here, "Internet function" is defined broadly. It might be an enabling protocol already defined by standards, such as IP [RFC791], BGP [RFC4271], TCP [RFC793], or HTTP [HTTP]. It might also be a proposal for a new enabling protocol, or an extension to an existing one.¶
However, the Internet's functions are not limited to standards-defined protocols. User-visible applications built on top of standard protocols are also vulnerable to centralization -- for example, social networking, file sharing, financial services, and news dissemination. Likewise, the supply of underlying networking equipment, hardware, operating systems, and software can exhibit centralization that affects the Internet.¶
Centralization risk is strongest when it affects the entire Internet. However, it can also be present when a substantial portion of the Internet's users lack options for a function. For example, if there is only one provider for a function in a region or legal jurisdiction, that function is effectively centralized for those users.¶
"Decentralization" is the process of identifying centralization risk in the functions of a protocol or application, followed by the application of techniques used to prevent or mitigate centralization.¶
Decentralization does not require that a function need be so widely distributed that other important factors are sacrificed. Because the same network effects that cause centralization can also deliver benefits (such as improvements in efficiency, resiliency, latency, and availability; see Section 6.4 for further discussion), the appropriate amount of decentralization for a function might vary, with the optimal balance being determined by many factors. A function that is only available through a relatively small number of providers can still be effectively decentralized (see, for example, the Domain Name System [RFC1035]).¶
Therefore, discussions of centralization and architectural efforts at decentralization need to be made on a case-by-base basis, depending on the function in question, surrounding circumstances, and other regulatory mechanisms.¶
Note that it is important to distinguish centralization from anti-competitive concerns (also known as "anti-trust"). While there are many interactions between them and making the Internet more competitive may be a motivation for avoiding centralization, only courts are authoritative in determining what is and is not anti-competitive in a defined market, not standards bodies and other technical fora.¶
Centralization is undesirable because it is counter to the Internet's nature, because it violates the end users' expectations, and because of the many negative effects it can have on the network's operation and evolution.¶
First, the Internet's very nature is incompatible with centralization of its functions. As a "large, heterogeneous collection of interconnected systems" [BCP95] the Internet is often characterised as a "network of networks". These networks relate as peers who agree to facilitate communication, rather than having a relationship of subservience to others' requirements or coercion by them. This focus on independence of action carries through the way the network is architected -- for example, in the concept of an "autonomous system".¶
Second, as the Internet's first duty is to the end user [RFC8890], allowing such power to be concentrated into few hands is counter to the IETF's mission of creating an Internet that "will help us to build a better human society." [BCP95] When a third party has unavoidable access to communications, the "informational and positional advantages" [INTERMEDIARY-INFLUENCE] gained can be used to observe behavior (the "panopticon effect") and shape or even deny behaviour (the "chokepoint effect") -- which can be used by those parties (or the states that have authority over them) for coercive ends. [WEAPONIZED-INTERDEPENDENCE]¶
Finally, concentration of power has deleterious effects on the Internet itself, including:¶
See also [TECH-SUCCESS-FACTORS] for further exploration of how centralization can affect the Internet.¶
To summarize, centralization would allow the Internet (or some part of it) to be captured, effectively turning it into a "walled garden" that cannot meet both architectural design goals and users' expectations, and endangering its ongoing viability at the same time.¶
Centralization of the Internet is not uniform; it presents in a variety of ways, depending on its relationship to the function in question and underlying causes. The subsections below suggest a classification system for Internet centralization.¶
Creation of a fixed role for a specific party is the most straightforward kind of centralization. For example, most proprietary messaging, videoconferencing, chat, and similar protocols operate in this fashion.¶
While some argue that such protocols are simpler to design, more amenable to evolution, and more likely to meet user needs [MOXIE], this approach most often reflects commercial goals -- in particular, a strong desire to capture the protocols' financial benefits by "locking in" users to a proprietary service.¶
Directly centralised protocols and applications are not considered to be part of the Internet per se; instead, they are more properly characterized as proprietary protocols that are built on top of the Internet. As such, the Internet architecture and associated standards do not regulate them, beyond the constraints that the underlying protocols (e.g., TCP, IP, HTTP) impose.¶
Some protocols introduce centralization risk that is unavoidable by nature.¶
For example, when there is a need for a single, globally coordinated "source of truth", that function is by nature centralized -- such as in the Domain Name System (DNS), which allows human-friendly naming to be converted into network addresses in a globally consistent fashion.¶
IP addresses allocation is another example of a necessary function having centralization risk. Internet routing requires addresses to be allocated uniquely, but if a single government or company captured the addressing function, the entire Internet would be at risk of abuse by that entity.¶
Similarly, the need for coordination in the Web's trust model brings centralization risk, because of the Certificate Authority's role in communication between clients and servers.¶
Protocols that need to solve the "rendezvous problem" to coordinate communication between two parties that are not in direct contact also suffer from this kind of centralization risk. For example, chat protocols need to coordinate communication between two parties that wish to talk; while the actual communication can be direct between them (so long as the protocol facilitates that), the endpoints' mutual discovery typically requires a third party.¶
Internet protocols often attempt to mitigate necessary centralization risk using measures such as federation (see Section 5.1) and multi-stakeholder administration (see Section 5.2).¶
Protocols that successfully mitigate necessary centralization are often reused, to avoid the considerable cost and risk of re-implementing those mitigations. For example, if a protocol requires a coordinated, global naming function, reusing the Domain Name System is usually preferable to establishing a new system.¶
Even when a protocol avoids direct centralization and does not exhibit any necessary centralization, it might become centralized in practice when external factors influence its deployment. Factors that encourage use of a central function, despite the absence of such a requirement in the protocol itself, can cause indirect centralization. Such factors might be economic, social, or legal.¶
Often, the factors driving indirect centralization are related to the network effects that are so often seen on the Internet. While in theory every node on the Internet is equal, in practice some nodes are much more connected than others: for example, just a few sites drive much of the traffic on the Web. While expected and observed in many kinds of networks,[SCALE-FREE] network effects award asymmetric power to nodes that act as intermediaries to communication.¶
Left unchecked, these factors can cause a potentially decentralized application to become directly centralised, because the central function has leverage to "lock in" users. For example, social networking is an application that is currently supplied by a few directly centralized, proprietary platforms despite standardization efforts (see, e.g., [W3C.CR-activitystreams-core-20161215]), because of the powerful network effects associated.¶
By its nature, indirect centralization is difficult to avoid in protocol design, and federated protocols are particularly vulnerable to it (see Section 5.1).¶
Most Internet protocols and applications depend on other, "lower-layer" protocols. The features, deployment, and operation of these dependencies can surface centralization risk into functions and applications build "on top" of them.¶
For example, the network between endpoints can introduce centralization risk to application-layer protocols, because it is necessary for communication and therefore has power over it. A network might block access to, slow down, or change the content of various application protocols or specific services for financial, political, operational, or criminal reasons, thereby creating pressure to use other services, which can result in centralization of them.¶
Inherited centralization risk is only present when users cannot use an alternative means of accessing the desired service. For example, users often have flexibility in choice of Internet access, so they could just "route around" a network that affects their chosen service. However, such choices are often not available at the moment, and the Internet's topology means that a choke point upstream could still affect their Internet access.¶
When deployed at scale, encryption can be an effective technique to control many inherited centralization risks. By reducing the number of parties who have access to content of communication, the ability of lower-layer protocols and intermediaries at those layers to interfere with or observe is prevented. Even while they may still prevent communication, encryption makes it more difficult to discriminate the target from other traffic.¶
Note that the prohibitive effect on inherited centralization is most pronounced when most (if not all) traffic is encrypted -- providing yet more motivation for that goal (see also [RFC7258]).¶
The complement to inherited centralization is platform centralization -- where a function does not directly define a central role, but could facilitate centralization in the applications it supports.¶
For example, HTTP [HTTP] is not considered a centralized protocol; interoperable servers are relatively easy to instantiate, and multiple clients are available. It can be used without central coordination beyond that provided by DNS, as discussed above.¶
However, applications built on top of HTTP (as well as the rest of the "Web Platform") often exhibit centralization. As such, HTTP is an example of a platform for centralization -- while the protocol itself is not centralized, it facilitates the creation of centralized services and applications.¶
Like indirect centralization, platform centralization is difficult to prevent with protocol design. Because of the layered nature of the Internet, most protocols allow considerable flexibility in how they are used, often in a way that it becomes attractive to form a dependency on one party's operation. Notably, this can happen even if the protocol does not accommodate intermediation explicitly.¶
Over time, various techniques have been developed to decentralize protocols and applications. While use of these approaches can result in a function which is less centralized or less amenable to some kinds of centralization, they are not adequate to avoid centralization completely. They are also not indicators of whether a protocol is centralized without further analysis.¶
A widely known technique for managing centralization in Internet protocols is federation -- designing them in such a way that new instances of any intermediary or otherwise centralized function are relatively easy to create, and they can maintain interoperability and connectivity with other instances.¶
For example, SMTP [RFC5321] is the basis of the e-mail suite of protocols, which has two functions that are necessarily centralized:¶
E-mail reuses DNS to help mitigate the first risk. To mitigate the second, it defines an intermediary role for routing users' messages, the Message Transfer Agent (MTA). By allowing anyone to deploy an MTA and defining rules for interconnecting them, the protocol's users avoid a requirement for a single central router.¶
Users can (and often do) choose to delegate that role to someone else, or run their own MTA. However, running your own mail server has become difficult, because of the likelihood of a small MTA being classified as a spam source. Because large MTA operators are widely known and have greater impact if their operation is affected, they are less likely to be classified as such, indirectly centralizing the protocol's operation (see Section 4.3).¶
Another example of a federated Internet protocol is XMPP [RFC6120], supporting "instant messaging" and similar functionality. Like e-mail, it reuses DNS for naming and requires federation to facilitate rendezvous of users from different systems.¶
While some deployments of XMPP do support truly federated messaging (i.e., a person using service A can interoperably chat with someone using service B), many of the largest do not. Because federation is voluntary, some operators captured their users into a single service, rather than provide the benefits of global interoperability.¶
The examples above illustrate that federation can be a useful technique to avoid direct centralization and manage necessary centralization, but on its own does not avoid indirect and platform centralization. If a single entity can capture the value provided by a protocol, they may use the protocol as a platform to get a "winner take all" outcome -- a significant risk with many Internet protocols, since network effects often promote such outcomes. Likewise, external factors (such as spam control) might naturally "tilt the table" towards a few operators.¶
The risks associated with a necessarily centralized function (see Section 4.2) are sometimes mitigated by delegating that function's administration to a multi-stakeholder body.¶
A multi-stakeholder body is an institution that includes representatives of the different kinds of parties that are affected by the system's operation ("stakeholders") in an attempt to make well-reasoned, broadly agreed-to, and authoritative decisions.¶
The most relevant example of this technique is the administration of the DNS, which as a "single source of truth" exhibits necessary centralization in its naming function, as well as the operation of the system overall. To mitigate operational centralization, multiple root servers that are administered by separate operators (themselves diverse in geography) and a selection of corporate entities, non-profits, and government bodies from many jurisdictions and affiliations carry this task out. The name space itself is regulated by the Internet Corporation for Assigned Names and Numbers (ICANN), which is defined as a globally multi-stakeholder body with representation from end users, governments, operators, and others.¶
Another example of multi-stakeholderism is the standardization of Internet protocols themselves. Because a specification effectively controls implementation behavior, the standardization process can be seen as a single point of control. As a result, Internet standards bodies like the IETF allow open participation and contribution, make decisions in an open and accountable way, have a well-defined process for making (and when necessary, appealing) decisions, considering the views of different stakeholder groups [RFC8890].¶
Yet another example is the administration of the Web's trust model, implemented by Web browsers as relying parties and Certificate Authorities as trust anchors. To assure that all parties meet the operational and security requirements necessary to provide the desired properties, the CA/Browser Forum was established as an oversight body that involves both of those parties as stakeholders.¶
A major downside of this approach is that setup and ongoing operation of multi-stakeholder bodies is not trivial. Additionally, their legitimacy cannot be assumed, and may be difficult to establish and maintain (see, e.g., [LEGITIMACY-MULTI]). This concern is especially relevant if the function being coordinated is broad, complex, and/or contentious.¶
Increasingly, distributed consensus technologies, such as the blockchain, are touted as a solution to centralization issues. A complete survey of this rapidly changing area is beyond the scope of this document, but at a high level, we can generalise about their properties.¶
These techniques attempt to avoid centralization risk by distributing intermediary or otherwise potentially centralized functions to members of a large pool of protocol participants. Proper performance of a function is typically guaranteed using cryptographic techniques (often, an append-only transaction ledger). A particular task's assignment to a node for handling usually cannot be predicted or controlled.¶
Sybil attacks (where enough participants coordinate their activity to affect the protocol's operation) are a major concern for these protocols. Diversity in the pool of participants is encouraged using indirect techniques such as proof-of-work (where each participant has to demonstrate significant consumption of resources) or proof-of-stake (where each participant has some other incentive to execute correctly).¶
Use of these techniques can create barriers to direct and inherited centralization. However, depending upon the application in question, indirect and platform centralization can still be possible.¶
Furthermore, distributed consensus technologies have several potential shortcomings that may make them inappropriate -- or at least difficult to use -- for many Internet applications, because their use conflicts with other important goals:¶
It is also important to recognise that a protocol or an application can use distributed consensus for some functions, but still have centralization risk elsewhere. Even when distributed consensus is used exclusively for all functions (which is uncommon, because of the associated costs), some coordination is still necessary -- whether that be through governance of the function itself, creation of shared implementations, or documentation of shared wire protocols. That represents centralization risk, just at a different layer (inherited or platform).¶
These potential shortcomings do not rule out the use of distributed consensus technologies for every use case. They do, however, caution against relying upon these technologies to avoid centralization uncritically.¶
Centralization is driven by powerful forces -- both economic and social -- as well as the network effects that come with Internet scale. Because permissionless innovation is a core value for the Internet, and yet much of the centralization seen on the Internet is performed by proprietary platforms that take advantage of this nature, the controls available to standards efforts on their own are very limited.¶
While standards bodies on their own cannot prevent centralization, there are meaningful steps that can be taken to prevent some functions from exhibiting some forms of centralization. There are also valuable contributions that standards efforts can make to other relevant forms of regulation.¶
Some types of centralization risk are relatively easy to manage in standards efforts. For example, a directly centralized protocol, were it to be proposed, would be rejected out of hand by the IETF. There is a growing body of knowledge and experience with necessary centralization, and a strong inclination to reuse existing infrastructure where possible. As discussed above, encryption is often a way to manage inherited centralization. These responses are appropriate ways for Internet standards to manage centralization risk.¶
However, preventing indirect and platform centralization is much more difficult in standards efforts. Because we have no "protocol police", it's not possible to demand that someone stop building a proprietary service using a purportedly federated protocol. We also cannot stop someone from building centralized services "on top" of standard protocols without abandoning architectural goals like permissionless innovation.¶
Therefore, committing significant resources to scrutinizing protocols for latent centralization risk -- especially for indirect and platform risks -- is unlikely to be effective in preventing Internet centralization. Almost all existing Internet protocols -- including IP, TCP, HTTP, and DNS -- suffer some form of indirect or platform centralization. Refusing to standardize a newer protocol because it faces similar risks would not be equitable, proportionate, or effective.¶
When we find centralization risk, we should consider its relationship with other goals, such as privacy. While there is rarely a pure tradeoff between two abstract goals such as these, when it surfaces attention should be paid to how effective architectural regulation (such as a standards effort) is in achieving each goal. In this example, a technical mechanism might be much more effective at improving privacy, whereas centralization might be better controlled by other regulators -- leading to the conclusion that the standards effort should focus on privacy.¶
Standards efforts should particularly focus on creating specifications for functions that are currently only satisfied by proprietary, centralized applications and protocols. For example, if social networking is thought to be a centralized function, this might mean creating specifications that enable decentralized social networking, perhaps using some or all of the techniques described in Section 5.¶
Keen readers will point out that social networking is effectively centralized despite the existence of such standards (see, e.g., [W3C.CR-activitystreams-core-20161215]), because the IETF and W3C create voluntary standards, not mandatory regulations.¶
However, architecture is not the only form of regulation; legal mechanisms combined with changing norms and the resulting market forces have their own regulatory effects [NEW-CHICAGO]. While for much of its lifetime the Internet has only been subject to limited legal regulation, that period appears to be ending.¶
It is far from certain that a legal mandate for interoperability based upon Internet standards will eventuate, but it is increasingly discussed as a remedy for competition issues (see, e.g., [OECD]). It is also uncertain that legally mandated interoperability will fully address centralization risks. However, if such specifications are not available from the Internet community, they may be created elsewhere without reference to the Internet's architectural goals.¶
Even absent a legal mandate, changes in norms and the market -- because of increasing knowledge and distrust of centralized functions -- can create demand for such specifications and the corresponding implementations.¶
To minimize centralization risk, standards-defined functions should have an explicit goal of broad, diverse implementation and deployment, so that users have as many choices as possible.¶
Section 2.1 of [RFC5218] explores some factors in protocol design that encourage this outcome.¶
This goal can also be furthered by ensuring that the cost of switching to a different implementation or deployment is as low as possible to facilitate subsequent substitution. This implies that the standard is functionally complete and specified precisely enough to result in meaningful interoperability.¶
These goals are sometimes in tension. For example, if a standard is extremely complex, it may discourage implementation diversity because the cost of a complete implementation is too high (consider: Web browsers). On the other hand, if the specification is too simple, it may not offer enough functionality to be complete, and the resulting proprietary extensions may make switching difficult (see Section 6.5).¶
Furthermore, if a new protocol is perceived as completely commoditized (so that no implementation can differentiate itself, and there is no barrier to switching), it may have difficulty achieving broad implementation -- at least by commercial entities.¶
Balancing these factors is difficult, but is often helped by community building and good design -- in particular, appropriate use of layering.¶
Some functions might see substantial benefits if intermediation -- i.e., adding a new party to communication to perform a function -- is introduced. When used well, intermediation can help improve:¶
However, introducing an intermediary role adds indirect and platform centralization risk to Internet protocols, because it brings opportunities for control and observation. While (as discussed above) standards efforts have a very limited capability to prevent or control these types of centralization, constraints on intermediary functions can prevent at least the most egregious outcomes.¶
As a result, intermediaries should only be interposed as a result of the positive action of at least one endpoint, and should have their ability to observe or control communication limited to what is necessary to perform their intended function.¶
For example, early deployments of HTTP allowed intermediaries to be interposed by the network without knowledge of the endpoints, and those intermediaries could see and change the full content of traffic by default -- even when they are only intended to perform basic functions such as caching. Because of the introduction of HTTPS and the CONNECT method (see Section 9.3.6 of [HTTP]), combined with efforts to encourage its adoption, those intermediaries are now required to be explicitly interposed by one endpoint.¶
See [I-D.thomson-tmi] for more guidance on protocol intermediation.¶
The term "intermediary" is also used (often in legal and regulatory contexts) more broadly than it has been in protocol design; for example, an auction Web site intermediates between buyers and sellers is considered an intermediary, even though it is not formally an intermediary in HTTP (see Section 3.7 of [HTTP]). Protocol designers can address the centralization risk associated with this kind of intermediation by standardising the function, rather than restricting the capabilities of the underlying protocols; see Section 6.2.¶
An important feature of Internet protocols is their ability to evolve, so that they can meet new requirements and adapt to new conditions without requiring a "flag day" to upgrade implementations. Typically, protocols accommodate evolution through extension mechanisms, which allow optional features to be added over time in an interoperable fashion.¶
However, protocol extensions can also increase the risk of platform centralization if a powerful entity can change the target for meaningful interoperability by adding proprietary extensions to a standard protocol. This is especially true when the core standard does not itself provide sufficient utility on its own.¶
For example, the SOAP protocol's [SOAP] extreme flexibility and failure to provide significant standalone value allowed vendors to require use of their preferred extensions, favouring those who had more market power.¶
Therefore, standards efforts should focus on providing concrete utility to the majority of their users as published, rather than being a "framework" where interoperability is not immediately available. Internet protocols should not make every aspect of their operation extensible; extension points should be reasoned, appropriate boundaries for flexibility and control. When a protocol defines extension points, they should not allow an extension to declare itself to be mandatory-to-interoperate, as that pattern invites abuse.¶
This document does not have direct security impact on Internet protocols. However, failure to consider centralization risks might cause a myriad of security issues.¶
This document benefits from discussions with Brian Trammell during our shared time on the Internet Architecture Board.¶