Internet-Draft Leslie L. Daigle Category: Informational Thinking Cat Enterprises Expires: July 28, 2000 Thommy Eklof Ericsson January 28, 2000 Networking Multiple DAG servers: Meshes draft-daigle-dag-mesh-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on July 28, 2000. Abstract The Common Indexing Protocol ([CIP1]) is designed to facilitate the creation not only of query referral indexes, but also of meshes of (loosely) affiliated referral indexes. The purpose of such a mesh of servers is to implement some kind of distributed sharing of indexing and/or searching tasks across different servers. So far, the TISDAG project ([TISDAG], [DAGEXP]) has focused on creating a single referral index; the obvious next step is to integrate that into a larger set of interoperating services. 1. Introduction 1.1 Overview of mesh possibilities Daigle & Eklof [Page 1] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Two different possibilities are possible for extending the TISDAG service to a mesh model (or some combination of both). First, it should be possible to create a mesh of DAG-based services. Or, it might be interesting to use the mesh architecture to incorporate access to other types of services (e.g., the Norwegian Directory of Directories). In either case, the basic principle for establishing a mesh is that interoperating services should exchange index objects, according to the architecture of the mesh (e.g., hierarchical, or graph-like, preferrably without loops!). As is outlined in the CIP documentation ([CIP1]), many possibilities exist for mechanisms for creating indexes over multiple referral servers -- for example, WDSP index objects could be passed along untouched, or a referral index server's contents could be aggregated into a new index object, generating referrals back to that server. The proposal is that the mesh should be constructed using index objects aggregated over participating services' servers. That is, referrals will be generated to other recognized services, not their individual participants. This can be done as a hierarchy or a level mesh one-layer deep, but the important reason for not simply passing forward index objects (unaggregated) is that individual services may support different ranges of access protocols, have particular security requirements, etc. Referrals should be directed to a CAP or CAPs -- either the standard ones used by the DAG system, or new ones established to support particular semantics of remote systems (e.g., other query types, etc). Within a given DAG system, referrals to these remote servers will look just like any other referral, although a particular SAP or SAPs may be established to provide query fulfillment (again, to enable translations between variations of service, to allow secure access if the relationship between the services is restricted, etc). In the following scenarios of mesh traversal, the assumption is that the primary service in discussion (Country A in Scenario 1, Country B in Scenario 2) is a DAG-based service. The scenarios are presented in the light of interoperating DAG services, but in most cases it would be equally applicable if the remote service was provided by some other service architecture. Again, the key element for establishing a mesh of any sort is the exchange of the CIP index object, not internal system architecture. 1.1.1 Scenario 1: Top Down Suppose 2 countries tie their services together. A user makes a query in Country A. A certain number of hits are made against the index objects of A's WDSPs. There is also a hit in the aggregate index of Country B. There are 3 possible cases under which this must be handled: Daigle & Eklof [Page 2] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Case 1: Country A and Country B are running services that are essentially the same -- in terms of protocols, queries, and schema that are supported. In this case, one referral should be generated per protocol supported by Country B's service. The referral can be passed back as far as the client, if its protocol supports referrals. Alternatively, the CAP may chain the referral through an appropriate SAP, in the usual fashion. In other words, the CAPs of Country B's service act as WDSPs to Country A's service. Daigle & Eklof [Page 3] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Consider the following illustration (only relevant CAPs, SAPs, etc, are shown; others suppressed for lack of room): +-----------------+ (1) |-----+ Country A | +-------+ ------>|Prot1| DAG | |A-WSDP1| <------| CAP | +-----| | Prot1 | (2) |-----+ |Prot1| +-------+ | | SAP | ----+ | +-----| +-------+ (3)| | +-------+ | |A-WDSP2| | | | RI-A | | | Prot1 | | +-----------------+ +-------+ | | +-------+ | |A-WDSP3| | | Prot2 | +----------------+ +-------+ | [...] | | +-----------------+ | |-----+ Country B | +-------+ +-------->|Prot1| DAG | |B-WSDP1| | CAP | +-----| | Prot2 | |-----+ |Prot1| +-------+ | | SAP | | +-----| +-------+ | +-------+ | |B-WDSP2| | | RI-B | | | Prot1 | +-----------------+ +-------+ [...] where Prot[i] is some particular query protocol RI-A has an index over all A-WDSP[i] and RI-B RI-B has an index over all B-WDSP[i] (1) is the query to the Country A DAG system, which yields a referral based on the index object from RI-B (2) is that referral (3) is the resolution of that referral, which the client takes to the Country B DAG system directly (to find out which, if any, B-WDSP[i] have relevant information) Case 2: Country A and Country B are running services that address the same service type (e.g., whitepages), but are not using an identical Daigle & Eklof [Page 4] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 collection of protocols, allowed queries, or schema. The index object that Country B sent to Country A's DAG service must be constructed in terms of Country A's service, in order for appropriate hits to be generated against the index object (i.e. for referrals to Country B's service). However, to resolve the referral, it will be necessary to do some further protocol/schema/query mapping. This can be done by a special SAP established within Country A's service, that maps Country A's service into the published service of Country B. Country A may then elect to support only one of Country B's access protocols, and the designated SAP will always contact one type of CAP at Country B. Alternatively, Country B can establish a particular CAP that does the mapping from Country A's service into something that is most appropriate against the internal structure of its service. In this case, Country A's referral will be to a special CAP in Country B's service (which, again, will look like a WDSP to the Country A service); in fact, the referral may be handled directly by the client software. The difference between the two possible approaches lies in the responsibility of managing the relationship between the 2 service types. On the one hand, Country A could handle it if it knows its service as well as the published access to Country B. On the other, Country B could be responsible for establishing a CAP for every country that may want to connect to it. The latter can, in some cases, be justified by the amount of internal optimization that can be done, and because it reduces the overhead for Country A's service (can pass the referral directly back to the client software). Daigle & Eklof [Page 5] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Consider the following illustration (only relevant CAPs, SAPs, etc, are shown; others suppressed for lack of room): +-----------------+ (1) |-----+ Country A | +-------+ ------>|Prot1| DAG | |A-WSDP1| <------| CAP | +-----| | Prot1 | (2) |-----+ |Prot1| +-------+ | | SAP | ----+ | +-----| +-------+ (3)| | +-------+ | |A-WDSP2| | | | RI-A | | | Prot1 | | +-----------------+ +-------+ | | +-------+ | |A-WDSP3| | | Prot2 | +----------------+ +-------+ | [...] | | +-----------------+ | |-----+ Country B | +-------+ | |Prot3| DAG | |B-WSDP1| | | CAP | +-----| | Prot3 | | |-----+ |Prot3| +-------+ | |---------+ | SAP | | |Country A| +-----| +-------->|CAP:Prot1| | |---------+ | +-------+ | +-------+ | |B-WDSP2| | | RI-B | | | Prot3 | +-----------------+ +-------+ [...] where Prot[i] is some particular query protocol RI-A has an index over all A-WDSP[i] and RI-B RI-B has an index over all B-WDSP[i] (1) is the query to the Country A DAG system, which yields a referral based on the index object from RI-B (2) is that referral (3) is the resolution of that referral, which the client takes to the Country B DAG system directly, but to a CAP that is specifically designed to accommodate protocols from Country A's service, and map it (and schema) into Country B's service. Likely, all Country B referrals will be chained for the Country A client Daigle & Eklof [Page 6] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Case 3: The third possibility is, in fact, a refinement of the first. If Country A and Country B are running services that are every way identical except for the data (WDSPs covered), then it may make sense to NOT aggregate Country B's WDSP index objects, but to copy them to Country A's server. Then, Country A's CAPs might be given access to the SAPs of Country B in order to carry out chaining directly at the remote service (instead of implicating Country A's SAPs and Country B's CAPs, as in the first example above). The answer does not come from technology -- it depends entirely on the nature of the relationship that can be established between Country A and Country B's services. 1.1.2 Scenario 2: Working Up The above scenario implicitly assumes that Country A's server had received index objects from Country B's server. This will be the case if Country A's server is higher in the levels of a hierarchy of services (established by agreements between the service operators), or if the network is comprised of servers that share their index objects with all others, for example. In the latter case, searching at any one of the servers in the service yields the full range of results -- referrals will be made to any other server that might have data that fulfills the user's query. The sharing of the index objects is a mechanism to allow each server to manage local data, while enabling distributed load-sharing on the basic query handling. However, if a hierarchical, or at least not-completely-connected model is used for the server network, queries carried out at a level other than the top of the hierarchy, or in one particular branch of the hierarchy, will not actually be matched against all index objects. Therefore, there may be other servers to which the query should be directed if the full space needs to be searched. Suppose, for example, that in the above example Country B is in fact lower in the hierarchy than Country A. A user sending a query to Country B's service may be content to limit the scope of the query to that country's information (this is true in enough real-life situations that this hierarchical relationship becomes an effective mechanism for scoping queries and avoiding having to flood the entire network with every single query or keep full copies of all data in every server). Still in theoretical stages, the DAG/IP provides control constructs to allow DAG components to act according to the topology of the mesh. A CAP might use the "polled-by" system command to establish what other servers in the mesh exist in higher levels (and therefore would be worth contacting if the scope of the search is to be increased). Daigle & Eklof [Page 7] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 In the example above, a CAP in Country B's system could determine that Country A's service was polling Country B, and therefore make it a logical target for expanding the scope of the query. More experience (primarily with server mesh topologies) is necessary before it will be clear how to best make use of these capabilities: . should the CAP always broaden the scope? only if there are no local referrals? under user direction? . should the CAP use a local SAP to contact the remote service's CAP? . is it better to completely connect the mesh of servers, or produce some kind of hierarchy? . etc 2. Other considerations Depending on the context in which a mesh is established (e.g., between national white pages services, or different units of a corporate organization, etc), it may be useful to allow individual WDSPs to indicate whether they are willing to have their data included in a DAG system's aggregated index object (i.e., allowing the DAG system to receive referrals from other systems in the mesh). 3. Authors' Addresses Leslie L. Daigle Thinking Cat Enterprises Email: leslie@thinkingcat.com Thommy Eklof Ericsson S-126 25 STOCKHOLM Sweden Email: thommy.eklof@ericsson.com 4. References Request For Comments (RFC) and Internet Draft documents are available from numerous mirror sites. [CIP1] J. Allen, M. Mealling, "The Architecture of the Common Indexing Protocol (CIP)", RFC 2651, August 1999. [TISDAG] L. Daigle, R. Hedberg "Technical Infrastructure for Daigle & Eklof [Page 8] Internet-Draft draft-daigle-dag-mesh-00.txt January 2000 Swedish Directory Access Gateways (TISDAG)," RFC XXXX, January 2000 [DAGEXP] T. Eklof, L. Daigle, "Wide Area Directory Deployment Experiences", Internet Draft (work in progress), September 1999 draft-eklof-dag-experiences-02.txt [NDD] R. Hedberg, H. Alvestrand, "Technical Specifica tion, The Norwegian Directory of Directories (NDD)," Internet Draft (work in progress), May 1999 Daigle & Eklof [Page 9]