INTERNET-DRAFT H. Berkowitz Chesaapeake Computer Consultants Expiration Date: August 1998 March 1998 To Be Multihomed: Requirements & Definitions draft-berkowitz-multirqmt-01.txt 1. Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). 2. Abstract As organizations find their Internet connectivity increasingly critical to their mission, they seek ways of making that connectivity more robust. The term ''multi-homing'' often is used to describe means of fault-tolerant connection. Unfortunately, this term covers a variety of mechanisms, including naming/directory services, routing, and physical connectivity. The ''home'' may be identified by a DNS name, an IP address, or an IP address and transport-layer port identifier. Any of these identifiers may be translated in the path between source and destination. This memorandum presents a systematic way to define the requirement for resilience, and a taxonomy for describing mechanisms to achieve it. Its intended audience is primarily people concerned with planning and performing multihoming deployments, rather than protocol developers. It examines both requirements and applications, with the emphasis on the former. 3. Introduction As the Internet becomes more ubiquitous, more and more enterprises connect to it. Some of those enterprises, such as Web software vendors, have no effective business if their connectivity fails. Other enterprises do not have mission-critical Internet applications, but become so dependent on routine email, news, web, and similar access that a loss of connectivity becomes a crisis. As this Internet dependence becomes more critical, prudent management suggests there be no single point of failure that can break all Internet connectivity. The term "multihoming" has come into vogue to describe various means of enterprise-to-service provider connectivity that avoid a single point of failure. Multihoming also can describe connectivity between Internet Service Providers and "upstream" Network Service Providers. Several terms have become overloaded to the point of confusion, including multihoming, virtual private networks, and load balancing. This document attempts to bring some order to the definition of multihoming. It partially overlaps definitions of virtual private networks [Ferguson & Huston]. If we take the word "multihoming" in the broadest context, it implies there are multiple ways to reach a "home" destination. This "home" may be identified by a name, an IP address, or a combination of IP address and TCP/UDP port. In this document, TCP/UDP ports will be referred to as TU ports. There are other motivations for complex connectivity from enterprises to the Internet. Mergers and acquisitions, where the joined enterprises each had their own Internet access, often mean complex connectivity, at least for a transition period. Consolidation of separate divisional networks also creates this situation. A frequent case arises when a large enterprise decides that Internet access should be available corporate-wide, but their research labs have had Internet access for years -- and it works, as opposed to the new corporate connection that at best is untried. Many discussions of multihoming focus on the details of implementation, using such techniques as the Border Gateway Protocol (BGP) [RFC number of the Applicability Statement], multiple DNS entries for a server, etc. This document suggests that it is wise to look systematically at the requirements before selecting a means of resilient connectivity. One implementation technique is not appropriate for all requirements. There are special issues in implementing solutions in the general Internet, because poor implementations can jeopardize the proper function of global routing or DNS. An incorrect BGP route advertisement injected into the global routing system is a problem whether it originates in an ISP or in an enterprise. 4. Goals Requirements tend to be driven by one or more of several major goals for server availability and performance. Availability goals are realized with resiliency mechanisms, to avoid user-perceived failures from single failures in servers, routing systems, or media. Performance goals are realized by mechanisms that distribute the workload among multiple machines such that the load is distributed in an useful manner. Like multi-homing, the terms load-balancing and load-sharing have many definitions. In defining requirements, the servers themselves may either share or balance the load, there may be load-sharing or load-balancing routing paths to them, or the routed traffic may be carried over load-shared or load-balanced media. 4.1 Analyzing Application Requirements Several questions need to be answered in the process of refining goals: -- the administrative model and administrative awareness of endpoints -- availability requirements -- the security model -- addressing requirements -- scope of multihoming 4.1.1 Administrative Model A key question is: are endpoints predefined in the multihoming process, or will either the client or server end be arbitrary? The servers of interest may be inside the enterprise, or outside it. If they are outside, their names or addresses may or may not be preconfigured into multihoming mechanisms. In this document, intranet clients and servers are inside the enterprise and intendedprimarily for enterprise use. The existence of both can be preconfigured. Intranet clients have access only to machines on the intranet. Multinet clients servers are inside the enterprise, but there is pre-authorized access by known external partners. Internet servers are operated by the enterprise but intended to be accessible to the general Internet. Internet clients have general Internet access that may be mediated by a firewall. The client administrator will know the prior identity of clients, but not of servers. The server administrator will know the prior identity of servers, but not of clients. 4.1.2 Availability Requirements There are major implications between defining a requirement for high availability of initial access, and making the connection stay up once access has been achieved. The latter tends to require transport layer awareness. In the terminology of RFC1775, "To be 'on' the Internet," servers described here have "full" or a subset of "client" access. Client servers may not directly respond to specific IP packet from an arbitrary host, but a system such as a firewall MUST respond for them unless a security policy precludes that. Some valid security policies, for example, suppress the response of ICMP Destination Administratively Prohibited responses, because that would reveal there is an information resource being protected. RFC1775 defines full access as " a permanent (full-time) Internet attachment running TCP/IP, primarily appropriate for allowing the Internet community to access application servers, operated by Internet service providers. Machines with Full access are directly visible to others attached to the Internet, such as through the Internet Protocol's ICMP Echo (ping) facility. The core of the Internet comprises those machines with Full access." This definition is extended here to allow full firewalls or screening routers always to be present. If a proxy or address translation service exists between the real machine and the Internet, if this service is available on a full-time basis, and consistently responds to requests sent to a DNS name of the server, it is considered to be full-time. In this discussion, we generalize the definition beyond machines primarily appropriate for the Internet community as a whole, to include in-house and authorized partner machines that use the Internet for connectivity. Both the cases described in 4.2.3 and 4.2.4 have been termed "Virtual Private Networks." 4.1.3 Security Model Security requirements can include various cryptographic schemes, as well as mechanisms to hinder denial of service attacks. The requirements analyst must determine whether cryptography is needed, and, if so, whether cryptographic trust must be between end hosts or between end hosts and a trusted gateway. Such gateways can be routers or multiported application servers. 4.1.4. Addressing Refinements and Issues It is arguable that addressing used to support multihoming is a routing deployment issue, beyond the scope of this document. Rationales for including it here is that addressing MAY affect application behavior. There also may be administrative requirements for addressing, such as a service provider that contracts to run a multinet may require addresses to be registered, possibly from the provider's address space. If the enterprise runs applications that embed network layer addresses in higher-level data fields, solutions that employ address translation, at the packet or virtual connection level, MAY NOT work. Use of such applications inherently is a requirement for the eventual multihoming solution. Consideration also needs to be given to application caches in addition to those of DNS. Firewall proxy servers are a good example where multiple addresses associated with a given destination may not be supported. RFC1918 internal, static network address translation (NAT) to outside RFC1918 internal, dynamic port address translation (PAT) to outside Registered internal, Provider Assigned (PA) Registered internal, Provider Independent (PI) 4.1.5 Scope of multihoming Multihoming may be defined between an end host and a router or application gateway, on an end-to-end basis possibly involving virtual servers, among routers, or among elements in a transmission system. Different multihoming scopes may support the same application requirement. 4.2. Application Goals These goals need to be agreed to by the people or organization responsible for the applications. Not to reach fairly formal agreement here can lead to problems of inappropriate expectations. The term "service level agreement" often refers to expectations of performance, such as throughput or response time. Ideas here extend the performance-based model to include availability. 4.2.1 Specific server availability The first goal involves well-defined applications that run on specific servers visible to the Internet at large. This will be termed "endpoint multihoming", emphasizing the need for resilience of connectivity to well-defined endpoints. Solutions here often involve DNS mechanisms. There are both availability and performance goals here. Availability goals arise when there are multiple routing paths that can reach the server, protecting it from single routing failures. Other availability goals involve replicated servers, so that the client will reach a server regardless of single server failures. Performance goals include balancing client requests over multiple servers, so that one or more servers do not become overloaded and provide poor service. Requests can be distributed among servers in a round-robin fashion, or more sophisticated distribution mechanisms can be employed. Such mechanisms can consider actual real-time workload on the server, routing metric from the client to the server, known server capacity, etc. >From an application standpoint, this is either a many-to-one topology, many clients to one server, or a many-to-many topology when multiple servers are involved. It can be worthwhile to consider a many-to-few case, when the few are multiple instances of a server function, which may appear as a single server to the general Internet. The idea of many-to-few topology allows for a local optimization of inter-server communications, without affecting the global many-to-one model. Addresses on interfaces that connect to the general Internet need to be unique in the global Internet routing system, although they may be translated, at the network address or port level, from public to internal space.. 4.2.2 General Internet connectivity from the enterprise The second is high availability of general Internet connectivity for arbitrary enterprise users to the outside. This will be called "internetwork multihoming". Solutions here tend to involve routing mechanisms. This can be viewed as a few-to-many application topology. Addresses on interfaces that connect to the general Internet need to be unique in the global Internet routing system, although they may be translated, at the network address or port level, from internal private address to public space. 4.2.3 Use of Internet services to interconnect "intranet" enterprise campuses The third involves the growing number of situations where Internet services are used to interconnect parts of an enterprise. This is "intranetwork multihoming". This will usually involve dedicated or virtual circuits, or some sort of tunneling mechanisms. This case may be termed a "virtual private network." The "multihoming" aspect of this case is associated with high availability to the connectivity network that underlies the tunneling system. In this case, addresses only need to be unique within the enterprise. 4.2.4 Use of Internet services to connect to "multinet" partners A fourth category involves use of the Internet to connect with strategic partners. True, this does deal with endpoints, but the emphasis is different than the first case. In the first case, the emphasis is on connectivity from arbitrary points outside the enterprise to points within it. This second case deals with pairs of well-known endpoints. These endpoints may be linked with dedicated or virtual circuits defined at the physical or data link layer. Tunneling or other virtual private networks may be relevant here as well. There will be coordination issues that do not exist for the third case, where all resources are under common control. Addresses need to be unique in the different enterprises, but do not need to be unique in the global Internet. 5. Planning and Budgeting In each of these scenarios, organization managers need to assign some economic cost to outages. Typically, there will be an incident cost and an incremental cost based on the length or scope of the connectivity loss. Ideally, this cost is then weighted by the probability of outage. A weighted exposure cost results when the outage cost is multiplied by the probability of the outage. Resiliency measures modify the probability, but increase the cost of operation. Operational costs obviously include the costs of redundant mechanisms (i.e., the addititional multihomed paths), but also the incremental costs of personnel to administer the more complex mechanisms -- their training and salaries. 6. Issues 6.1 Performance vs. Robustness: the Cache Conundrum Goals of many forms of "multi-homing" conflict with goals of improving local performance. For example, DNS queries normally are cached in DNS servers, and in the requesting host. From the performance standpoint, this is a perfectly reasonable thing to do, reducing the need to send out queries. >From the multihoming standpoint, it is far less desirable, as application-level multihoming may be based on rapid changes of the DNS master files. The binding of a given IP address to a DNS name can change rapidly. 6.2 Service Level Agreements Enterprise networks, especially mainframe-based, are accustomed to building and enforcing service level agreements for application performance. A key to being able to do this is total control of the end-to-end communications path. In the current Internet, the enterprise(s) at one or both ends control their local environments, and have contractual control over connections to their direct service providers. If service level control is a requirement, and both ends of the path are not under control (i.e., cases 1 and 2), the general Internet cannot now provide service level guarantees. The need for control should be reexamined, and, if it still exists, the underlying structure will need to be dedicated resources at the network layer or below. A network service provider may be able to engineer this so that some facilities are shared to reduce cost, but the sharing is planned and controlled. 6.3 Symmetry One of the reasons service level agreements are not enforceable in the general Internet is the reality that global routing cannot be guaranteed to be symmetrical. Symmetrical routing assumes the path to a destination is simply reversed to return a response from that destination. Both legs of a symmetrical path are assumed to have the same performance characteristics. Global Internet routing is not necessarily optimized for best end-to-end routing, but for efficient handling in the Autonomous Systems along the path. Many service providers use "closest exit" routing, where they will go to the closest exit point from their perspective to get to the next hop AS. The return path, however, is not necessarily of a mirror image of the path from the original source to the destination. Closest exit routing is, in fact, a "feature" rather than a "bug" in some multihoming schemes [Peterson] [Friedman]. Especially when the enterprise network has multiple points of attachment to the Internet, either to a single ISP AS or to multiple ISPs, it becomes likely that the response to a given packet will not come back at the same entry point in which it left the enterprise. This is probably not avoidable, and troubleshooting procedures and traffic engineering have to consider this characteristic of multi-exit routing. 6.4 Security ISPs may be reluctant to let user routing advertisements or DNS zone information flow directly into their routing or naming systems. Users should understand that BGP is not intended to be a plug-and-play mechanism; manual configuration often is considered an important part of maintaining integrity. Supplemental mechanisms may be used for additional control, such as registering policies in a registry [RPS, RA documents] or egress/ingress filtering [Ferguson draft] Challenges may arise when client security mechanisms interact with fault tolerance mechanisms associated with servers. For example, if a server address changes to that of a backup server, a stateful packet screening firewall might not accept a valid return. Similarly, unless servers back one another up in a full mirroring mode, if one end of a TCP-based application connection fails, the user will need to reconnect. As long as another server is ready to accept that connection, there may not be major user impact, and the goal of high availability is realized. High availability and user transparent high availability are not synonymous. 6.5 Load Balancing vs. Load Sharing These terms are often interchanged, but they really mean different things. Load balancing is deterministic, and at a finer level of control than load sharing, which is statistical. Load balancing is generally not something that can be realized in general Internet routing, other than in special and local cases between adjacent AS. A degree of load sharing is achievable in routing, but it may introduce significant resource demands and operational complexity. Paul Ferguson defines load-balancing as "a true "50/50" sharing of equal paths. This can be done by either (a) round robin per- packet transmission, (b) binding pipes a the lower layers such that bits are either 'bit-striped' across all parallel paths (like the etherchannel stuff), or binding pipes so that SAR functions are done in a method such as multilink PPP. These are fundamentally the same. "Load-sharing is quite different. It simply implies that no link is sitting idle -- that at least all links get utilized in some fashion. Usually in closest exit routing. The equity of utilization may be massively skewed. It may also resemble something along the lines of 60/40, which is reasonable." 6.6 Application Compatibility Some deployment mechanisms involve network address, or network address and TCP/UDP port, translation (NAT and NAPT). If the application protocols embed IP addresses in their protocol fields, NAT or NAPT may cause protocol failures. Translation mechanisms for such cases may require knowledge of the application protocol, as typified by application proxies in firewalls, or in application gateways with multiple interfaces. 7. Multihoming Deployment Technologies A basic way to tell which technology(ies) is applicable is to ask oneself whether the functional requirement is defined in terms of multihoming to specific hosts, or to specific networks. If the former, some type of application or transport technology is needed, because only these technologies have awareness of specific hosts. A given multihoming implementation may draw on several of these technologies. For example, the Cisco Distributed Director does DNS name-level redirection based in part on routing metrics. 7.1 Application/Name Based Techologies in this category may involve referring a client request to different instances of the endpoint represented by a single name. Another aspect of application/name multihoming may work at the level of IP addressing, but specifically is constrained to endpoint (i.e., server) activities that redirect the client request to a different endpoint. Application-level firewall proxy services can provide this functionality, although their application protocol modification emphasizes security while a multihoming application service emphasizes availability and quality of service. 7.2 Transport Based Transport based technologies are based on maintaining tunnels through an underlying network or transmission system. The tunnels may be true end to end, connecting a client host with a server host, or may interconnect between proxy servers or other gateways. 7.3 Network Based Network based approaches to multihoming are router-based. They involve having alternate next-hops, or complete routes, to alternate destinations. 7.4 Data Link Based Data link layer methods may involve coordinated use of multiple physical paths, as in multilink PPP or X.25. If the underlying WAN service has a virtual circuit mechanism, as in frame relay or ATM, the service provider may have multihomed paths provided as part of the service. Such functions blur between data link and physical layers. Other data link methods may manipulate MAC addresses to provide virtual server functions. 7.5 Physically-based Physical multihoming strategies can use diverse media, often of different types such as a wire backed up with a wireless data link. They also involve transmission media which have internal redundancy, such as SONET. 8. Application/Name Multihoming While many people look at the multihoming problem as one of routing, the goal is often multihoming to endpoints. Finding an endpoint usually begins in DNS. Once an endpoint address is found, some application protocols, notably HTTP and FTP, may redirect the request to a different endpoint. The basic idea here is that arbitrary clients will first request access to a resource by its DNS name, and certain DNS servers will resolve the same name to different addresses based on conditions of which DNS is aware, or using some statistical load-distribution mechanism. There are some general DNS issues here. DNS was not really designed to do this. A key issue is that of DNS cacheing. Cacheing and frequent changes in name resolution are opposite goals. Traditional DNS schemes emphasize performance over resiliency. 8.1 DNS Caching DNS standards do provide the capability for short cache lifetimes, which in principle support name based multihoming. "The meaning of the TTL field is a time limit on how long an RR can be kept in a cache. This limit does not apply to authoritative data in zones; it is also timed out, but by the refreshing policies for the zone. The TTL is assigned by the administrator for the zone where the data originates. While short TTLs can be used to minimize caching, and a zero TTL prohibits caching, the realities of Internet performance suggest that these times should be on the order of days for the typical host. If a change can be anticipated, the TTL can be reduced prior to the change to minimize inconsistency during the change, and then increased back to its former value following the change. [RFC1034] " Several real-world factors limit the utility of simply shortening the cache time. Widely used BIND, the most widely used DNS implementation, does not accept cache lifetimes less than 5 minutes. Dynamic DNS may be a long-term solution here. In the short term, setting very short TTL values may be help in some cases, but is not likely to help elow a granularity of 5 minutes. Remember that the name normally is resolved when an application session first is established, and the decisions are made over a longer time base than per-packet routing decisions. 8.2 DNS Multiple Hosts and Round Robin Response The DNS protocol allows it to return multiple host addresses in respondse to a single query. At the first level of DNS-based multihoming, this can provide additional reliability. A DNS server knows three IP addresses for the server function identified by server.example.com, 10.0.1.1, 10.0.2.1, and 10.0.3.1. A simple response to a query for server.example.com returns all three addresses. Assume the response provides server addresses in the order 10.0.1.1, 10.0.2.1, and 10.0.3.1. Whether this will provide multihoming now depends on the DNS client. Not all host client implementations will, if the first address returned (i.e., 10.0.1.1) does not respond, try the additional addresses. In this example, 10.0.2.1 might be operating perfectly, A variant suggested by Kent England is to have the addresses returned in the DNS response come from the CIDR blocks of different ISPs that provide connectivity to the server function [England]. This approach combines aspects of name and network multihoming. Again, this will work when intelligent clients try every IP address returned until a server responds. 8.3 Replication One approach [Peterson] uses DNS as the main method, but makes assumptions about the underlying routing. The two assumptions it makes, which appears generally valid for the Internet, are that connectivity providers use closest exit routing, and once a packet reaches a provider network, that provider knows best how to reach the specific server inside that network. One implementation of this approach is Cisco's DistributedDirector product. It involves communication between DNS servers and routers in the multihomed domain. When a DNS request is received by a DistributedDirector DNS server, the DD server selects the IP address to be returned based on information on routing cost from the client entry point to closest server, on administrative weight, or other generally routing-associated factors. 8.4 Cache Servers and Application Multihoming The cache server is the primary home for servicing the client request, but the true server backs it up. 9. Transport Multihoming Transport layer functions are conceptually end-to-end. There are two broad classes of transport multihoming function, those maintained by the endpoints and those that involve intermediate translation devices. 9.1 End-to-end Tunnel Maintenance Basic point-to-point tunneling mechanisms include GRE, PPTP and L2TP. DVMRP is a special case. Choices here will depend in part on the security policy and the administrative model by which multihoming is provided. GRE, for example, does not itself provide encryption, while PPTP and L2TP do. "The differences between PPTP and L2TP are more of where one wishes the PPP session to terminate, and one of control. It really depends on who you are, and where you are, in the scheme of the control process. If your desire is to control where the PPP session terminates (as an ISP might wish to control), then L2TP might be the right approach. On the other hand, however, if you are a subscriber, and you wish to control where the PPP session terminates (to, say, a PPTP server somewhere across the cloud), then PPTP might be the right approach -- and it would be transparent to the service provider). It really depends on what problem one is trying to solve, and if you are in the business of trying to create "services." [Ferguson-1998-2] 9.2 Tunneling with Translation Various proxy and address translation mechanisms can play a role in multihoming. They can be divided into several levels of topological constraints: -- all servers are colocated in a single address domain "behind" the translator -- servers are in different parts of a single address domain. These parts are connected by tunnels. -- the servers are at arbitrary network addresses, but the translator knows how to reach them. Application-aware proxies can have even more knowledge of application load. A variant of NAT, called load sharing NAT (LS-NAT), offers a load sharing mechanism at the transport/network level rather than the DNS level [Srishuresh]. When considering LSNAT-style load sharing, remember that it emphasizes providing a pool of servers capable of servicing requests. In its "local" form, it does not easily provide mechanisms for increasing reliability by mapping the user request to geographically distributed servers. More advanced variants can combine with DNS- and routing-aware mechanisms to increase reliability as well as performance. The LSNAT function is visible globally as a server address. It is actually a virtual server. When a client request arrives at the LSNAT, the LSNAT translates the destination address, transparently to the client, and passes it to a server in the LSNAT's server pool. LSNATs, in their basic form, do have topological restriction. It has been suggested that all request/response traffic in a single session must go from the real client, to one specific LSNAT, to the server. It is conceivable that multiple routers could be used, but they would need to be tightly synchronized. LSNAT builds on NAPT, but adds intelligence to the port mapping function. Also, general NAPT is oriented toward outgoing requests from the stub domain to the outside, while LSNAT emphasizes incoming requests to a virtual server address. As currently conceived, LSNATs operate at the TCP/UDP transport level, so they could not easily change server hosts during a session. There are potential workarounds to this, including using a multicast address as the server pool destination, with coordination between the servers as to which actually answers the request. For highly fault-tolerant applications, more than one server conceivably could answer the NAT request, and the LSNAT decide which to pass to the client. Typically, if servers are identical, it would be the first response received by the server pool side of the LSNAT. This general topology restriction suggests LSNAT functionality belongs on a single NAT-capable border router, with the server pool inside the stub domain. A LSNAT violates the Internet end-to-end model in the same way that basic NAT does. There is no requirement that the addressing in the stub domain be private, only that all access to the servers go through the NAT. In basic LSNAT, an arbitrary external client attempts to establish a session with what it believes to be a server. In reality, it is attempting to establish a session with the virtual server address of the "outside" interface of the LSNAT. The LSNAT, based on internal criteria, redirects the external request to a specific internal server in a server pool. Unique connections are established from the LSNAT to the pool server for each request, translating addresses and ports as needed. 9.2.2. Load Shared Network Address and Port Translation (LS-NAPT) Adding the complexity of port as well as address translation gives additional flexibility. In particular, adding port translation removes many topological limitations between the real servers and the NAT. In a LS-NAT router, inbound sessions identified by the tuple From a typical server room, analog and digital signals physically flow to a wiring closet, where they join a riser cable. Depending on the specific building, the closet and riser may be the responsibility of the enterprise or ISP, the building management, or a telecommunications carrier. The riser cable joins with other riser cables in a cable vault, from which a cable leaves the building and goes to the end switching office of the local telecommunications provider. Most buildings have a single cable vault, possibly with multiple cables following a single physical route back to the end office. A single error by construction excavators can cut multiple cables on a single path. A failure in carrier systems can isolate a single end office. Highly robust systems have physical connectivity to two or more POPs reached through two or more end offices. Alternatives here can become creative. On a campus, it can be feasible to use some type of existing ductwork to run additional cables to another building that has a physically diverse path to the end office. Direct wire burial, fiber optic cables run in the air between buildings, etc., are all possible. In a non-campus environment, it is possible, in many urban areas, to find alternate means of running physical media to other buildings with alternate paths to end offices. Electrical power utilities may have empty ducts which they will lease, and through which privately owned fiber can be run. 11.2 Provider Core As demonstrated by a rash of fiber cuts in early 1997, carriers lease bandwidth from one another, so a cut to one carrier-owned facility may affect connectivity in several carriers. This reality makes some traditional diverse media strategies questionable. Many organizations consciously obtain WAN connectivity from multiple carriers, with the notion that a failure in carrier will not affect another. This is not a valid assumption. If the goal is to obtain diversity/resiliency among WAN circuits, it may be best to deal with a single service provider. The contract with this provider should require physical diversity among facilities, so the provider's engineering staff will be aware of requirements not to put multiple circuits into the same physical facility, owned by the carrier or leased from other carriers. 12. Security Considerations 13. Acknowledgments 14. References [RFC1775] D. Crocker. To Be "On" the Internet. March 1995. [RFC1930] Hawkinson, J., and T. Bates. Guidelines for creation, selection, and registration of an Autonomous System (AS). March 1996. (Also BCP0006) [RFC1034] Mockapetris, P.V. Domain names - concepts and facilities. Nov-01-1987. [RFC1998] An Application of the BGP Community Attribute in Multi-home Routing. E. Chen & T. Bates. August 1996 [RFC2071] Ferguson, P., and H. Berkowitz, "Network Renumbering Overview: Why would I want it and what is it anyway?", RFC 2071, January 1997. [RFC2050] Hubbard, K., Kosters, M., Conrad, D., Karrenberg, D., and J. Postel, "INTERNET REGISTRY IP ALLOCATION GUIDELINES", BCP 12, RFC 2050, November 1996. [RFC1631] Egevang,, K., and P. Francis, "The IP Network Address Translator (NAT)", RFC 1631, May 1994. [Srishuresh] Srishuresh, P., and D. Gan, "Load Sharing using IP Network Address Translation (LSNAT)", work in progress, ftp://ftp.ietf.org/internet-drafts/draft-srisuresh-lsnat-02.txt [RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., de Groot, G-J., and E. Lear, "Address Allocation for Private Internets", RFC 1918, February 1996. [RFC1900] Carpenter, B., and Y. Rekhter, "Renumbering Needs Work", RFC 1900, February 1996. [RFC2280] Routing Policy Specification Language (RPSL). C. Alaettinoglu, T. Bates, E. Gerich, D. Karrenberg, D. Meyer, M. Terpstra, C. Villamizar. January 1998. [RFC1812] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, June 1995. [Peterson] Peterson, A. "Dynamic Selection of Geographically Distributed Servers," presentation at NANOG October 1997 meeting, notes at http://www.academ.com/nanog/october1997/dynamic-selection.html [Freedman] Freedman, A. "Dynamic Selection of Geographically Distributed Servers," presentation at NANOG October 1997 meeting, notes at http://www.academ.com/nanog/october1997/dynamic-selection.html [Ferguson-1998-1] Ferguson, P. "Re: Comments on "What is a VPN?"" Message to IETF VPN mailing list, 08 Mar 1998 19:52:29 15. Author's Address Howard C. Berkowitz Chesapeake Computer Consultants PO Box 6897 Arlington VA 22206 Phone: +1 703 998 5819 EMail: hcb@clark.net