INTERNET DRAFT Vishal Goenka Novell, Inc. Expires in six months from 1 June 1999 Intended Category: Standards Track Complex Directory Lookup using Java Based LDAP Query Extension 1. Status of this memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt To view the list Internet-Draft Shadow Directories, see http://www.ietf.org/shadow.html. 2. Abstract LDAP provides simple primitives to construct search filters for directory lookups. Complex LDAP search queries such as those involving joins or functional evaluation of simple query results require client side processing of the intermediate search results. The network overhead of retrieving intermediate results can be substantial for large databases. This document defines an LDAP extension for supporting server side evaluation of complex search queries. Goenka INTERNET DRAFT [Page 1] June 1999 2.1 Specification Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Motivation "Directory" is at the heart of many powerful and distributed applications today. With the widespread use of directory for variety of applications, the lookup queries would no longer be simple exact/partial matches of a subset of object attributes. Search queries involving joins or complex functional evaluation of simple query results would be quite common. An example of a search involving joins is : "Get me all the names and phone numbers of all printer administrators", which currently would require one query to get the administrator attribute of all printer objects and then separate queries to get the phone number for each administrator. A query based on functional evaluation would be one, where the search criteria is based on a function, f(x) of a simple query result x, rather than on x itself. An example is a X.509 Certificate parsing function, which returns a particular attribute (say, key usage) of a BER encoded X.509 Certificate. Assuming that a user's public key certificates are stored in the directory as simple attributes (byte array) rather than as objects, a search for the Root CA (Certification Authority) of a user's data encryption public key certificate would require retrieving all the certificates of the user, parsing them at the client side to find the one corresponding to the data encryption public key, and issuing another set of search queries to find each certificate's signer (CA), until the Root CA's self signed certificates are found. LDAP [1] is the most widely used protocol to access a directory. LDAP search queries can be simple enough to be built from rather simple primitives or could be quite complex to require special logic for evaluation. An LDAP client would resolve a complex query by fetching many directory entries from the server and evaluating each of them at the client side, issuing further queries based on intermediate results till the final desired result is found. Unless the initial search space can be reduced at the server end, using some exact/partial matches, this would be prohibitively expensive for large directory databases. This document outlines an efficient alternative for such enhanced directory search operations. Goenka INTERNET DRAFT [Page 2] June 1999 4 Introduction LDAP provides simple primitives to construct search filters. Specific implementations of LDAP Server may provide custom matching rules (called extensibleMatch), but their usefulness is limited because they are server specific and cannot be updated dynamically. To be useful to a large variety of directory based applications, the search mechanism should allow LDAP clients to specify application specific filters based on complex logic, (such as X509 Certificate parsing logic to retrieve specific attributes of a BER encoded certificate, or a logic to determine a prime number) in a secure manner. The variety of processing logic required to construct complex search filters can only be adequately specified using a programming/scripting language. The choice for the right language is a difficult one and subject to debate. We follow an elimination process on the following non-exhaustive list of requirements of a suitable language. 1. Wide acceptance as a programming language. 2. Availability on wide range of server and client platforms. 3. Availability of a rich and standard set of useful libraries to facilitate writing complex search queries. 4. Security features to address security implications of running client code on the server. 5. Good integration with the programming language used to write LDAP client applications, so that the search query can be adequately constructed by the client application. Scripting languages such as Perl, JavaScript, Visual Basic Script etc. do not meet several of the above requirements. In particular, requirements of availability on a wide variety of platforms, suitability for general purpose application programming, support for security features and availability of a rich set of standard libraries and protocol implementations. Among programming languages, Java is the only language that meets all the above requirements with its write-once-run-anywhere feature and technologies for secure object migration, namely, serialization and reflection, and security features designed for remote execution in web browsers across the Internet. With wider acceptance of Java as the Internet Programming Language, many business application are being written in Java, including directory based applications. A rich set of standard libraries is available on all Java platforms as part of Java Development Kit (JDK) or standard extensions to it. Non-standard Goenka INTERNET DRAFT [Page 3] June 1999 libraries are easy to integrate since classes can be dynamically retrieved and loaded as required. 4.1 Proposal Overview The key idea is to define an LDAP extension for sending complex search queries as serialized java objects and the required class definitions in a java archive (jar), which may optionally be signed. The serialized java object contains state information such as query parameters, while the class definition contains the evaluation logic. The server instantiates the java object in a Java Virtual Machine (JVM) and initializes it with an LDAP connection (over a socket) to itself with the same associated privileges as the client that originated the search. The java object MUST implement a known interface (discussed later)to enable the LDAP server invoke its initialization routine. The java object executing at the server end acts as a "client-proxy" and makes local LDAP search queries to the server. It evaluates the search results using the embedded custom logic and may perform several simple directory lookups to arrive at the final result. The client-proxy passes only the final result back to the LDAP server, which returns them back to the originating client. The client-proxy interacts with the LDAP server using LDAP calls over the LDAP connection that the server initialized it with. This allow the client-proxy to perform additional intermediate queries without requiring any API change to the LDAP server. The server enforces the client's access restrictions by granting the LDAP connection with the client-proxy, the same privileges as the originating client connection. The client-proxy SHOULD NOT be granted java permissions to open any new socket or do any file I/O on the host machine for security reasons. The JVM running the client-proxy code MAY execute on a different physical machine than the LDAP server, since the client-proxy communicates only with the LDAP server and over an LDAP socket that it is bootstrapped with. This flexibility allows various options for deployment of JVM to run client-proxies, including a dedicated pool of Java Application Servers, possibly connected to the host running the LDAP server over a high speed LAN. Java applications that use LDAP can also reuse a good chunk of their existing search filter code by sending them across to LDAP servers that support the proposed extension, and executing it locally otherwise (the default choice). 4.2 Goals and Requirements The requirements of this specification are : Goenka INTERNET DRAFT [Page 4] June 1999 - to provide a uniform and extendible interface and data format for securely executing client-proxy code on the server and communicating the results back to the client - to provide a simple way of indicating time-based resource requirements for executing client code on the server side - to provide a light-weight way of communicating the client proxy code to the server, including all java objects and required class definitions The goals of this document are : - to show the way in which the notion of a client-proxy is used to extend the capability of LDAP search operations - to describe the data structure of the LDAP extended operation for communicating java objects encapsulating search queries - to indicate an appropriate design for implementing the extension handler at the LDAP server, requiring minimum changes to the server code - to address the security concerns of executing client code at server side 4. Security Considerations Secure data exchange is one of the foremost issues associated with any client-server communication. The privacy and/or integrity of data exchanges is not available to LDAP, unless security at the transport layer (e.g., Secure Socket Layer) or at network layer (e.g., IPSEC or VPN) is used. Thus the mechanism used to protect all data exchanges between the LDAP client and the server, applies to the proposed extension as well, for all known attacks in this category (e.g., server spoofing, data sniffing, man-in-the-middle attack, session hijacking). There are two key aspects of security associated with the execution of client-proxy at the server end. The first one concerns the security of data and resources at the server, that hosts the client-proxy. This aspect of security is analogous to the security considerations of running a downloaded Java Applet in a client's web browser. The Java Security Architecture allows association of permissions with java code, based on where the code came from (known as code source) and who signed it. LDAP server implementations MAY define their own permission policies, and thereby achieve a sandbox environment for the execution of the client-proxy. Goenka INTERNET DRAFT [Page 5] June 1999 LDAP servers MAY require that the jar files containing the client-proxy be digitally signed in order to place trust on the non-malicious nature of the code. The second aspect is the fairness of execution. Its possible for the client-code (malicious, bug-ridden or otherwise) to take more than its fair share of server processing time. The client could use the lease model [6] to negotiate an approximate processing time at the server end. The server would use the client authentication information, along with current load to grant a lease for appropriate time. The java thread executing the client-proxy could be killed if the lease expires, unless the lease is renewed. The details of the leasing model are discussed in the following section. 5. Lease of Server Resources The lease model is used to negotiate the resource requirements at the server-side. The server resources to be considered for lease are execution time, memory usage, I/O resources (e.g., reading from or writing to files) and network resources such as sockets. - Execution Time -- The client indicates the estimated and maximum values of execution time (estimatedDuration and maxDuration) that might be required by the client-proxy. The server should either completely deny the lease or grant for atleast estimatedDuration up to a maximum of maxDuration. If the leased duration is elapsed and the client-proxy is still executing, the server can either extend the lease (depending upon server load conditions) or terminate the client-proxy thread. - Memory Usage -- The client indicates the estimated and maximum values of runtime memory (estimatedMemory and maxMemory) that might be required by the client-proxy. The server should either completely deny the lease or grant for atleast estimatedMemory up to a maximum of maxMemory. Memory requests beyond the granted lease MAY fail, unless the server relaxes or extends the lease, and the client-proxy code SHOULD be prepared to handle such situations. Server implementations MAY choose a relaxed memory lease policy, if a strict policy is difficult to implement. - I/O and Network Resources -- It is not expected of the client-proxy to use the I/O or network resources at the server end. An LDAP socket would be provided to the client-proxy by the bootstrap code at the server to communicate with the LDAP server locally. Java Permissions (as defined by RequestedPermissions) are to be used to Goenka INTERNET DRAFT [Page 6] June 1999 restrict or selectively allow usage of specific I/O or network resources. The lease response contains the lease granted to the client. Server implementations MAY implement their own algorithms to decide the lease grant for time-duration and memory usage depending upon the client's rights and server load conditions. 5.1 Lease Renewal A client-proxy might need to execute for a much longer duration to complete, though it might be able to generate partial results in much shorter time. A server might be unable to grant lease for the entire requested duration (maxDuration) at first negotiation, but might be fine with renewing the grant for another chunk of time, depending upon load conditions. If the client-proxy hasn't finished executing completely till the granted lease expires, the server can choose to renew the lease up to a maximum of maxDuration minus the previous grant. Once a granted lease expires, the server must notify the client about either the lease renewal or the termination of the client-proxy. (The lease grant on resources other than execution time would have to be handled in an analogous way.) 6. Architecture for JVM activation and client-proxy execution The current architecture assumes no change to the existing LDAP server other than an extension handler for "Java based Search Query Extension". The extension handler parses the extended operation request, and determines whether the requirements to invoke a JVM for client-proxy activation are met. The basic requirements include : - Lease Grant : Does the server have enough resources to grant the requested lease for server resources? - Java LDAP implementation : Does the server have an implementation of LDAP in Java, conforming to the interface that the client-proxy is written to? There are a few possible standards emerging, namely, JNDI (Java Naming and Directory Interface) with LDAP binding, and java LDAP API [3]. It is expected that the server may already have the java class files corresponding to standard LDAP interfaces, in which case these classes need not be sent as part of the query. For clients which are written to proprietary java LDAP interfaces, the implementation for the same must be supplied in a jar, along with the extended request. Goenka INTERNET DRAFT [Page 7] June 1999 - Java Permissions : Does the server policy allow granting the requested java permissions to the client-proxy code, possibly based on the client's authentication credentials. An appropriate response is sent to the client, whether or not the requirements for activation are met. If the requirements are met, the server launches a JVM (or communicates with an existing JVM), and schedules the activation and execution of the client-proxy. The java extension handler thread creates a class loader, to read the required classes from the jar accompanying the request. Appropriate permissions are associated with the protection domain defined by these classes. Next, it opens a socket with the LDAP server, and associates the permissions of the requesting client with the socket, using the internal APIs of the LDAP server. Finally, the java object encapsulating the query is de-serialized, using the readObject() method of the object's class. The LDAP socket is passed as an argument to the startup function (run(socket)) and a new thread starts executing the client-proxy. The client-proxy code does a simple bind to the LDAP server over the given LDAP socket, and makes further LDAP search queries to the server. Separate methods are provided to return (partial) results, and to query the progress of the client-proxy. The client-proxy MUST be allowed client's privileges using simple LDAP Bind. The interface definition for the client-proxy is as follows: public interface org.ietf.ldap.client.LDAPClientProxy implements java.io.Serializable { /** * The startup function, which queries the LDAP server using * the given LDAP Socket. The partial results are stored * internally such that they can be retrieved at any time, * using getPartialResults(). */ void run (java.net.Socket socket); /** * Returns the (partial) results (if available). The results * are formatted as a byte [], which is interpreted by the * client. null is returned, if there are no partial results * available. */ synchronized byte [] getPartialResults(); /** * Returns the progress in terms of percentage (0-100). This * helps the monitor thread to gauge the progress, and take Goenka INTERNET DRAFT [Page 8] June 1999 * the lease renewal decisions. The main thread (executing * run()) has the responsibility to increment the progress * appropriately. */ synchronized int progress(); } The client-proxy thread executes through the run() method, incrementing the progress variable as well as generating partial results (formatted as byte[]) during its execution. A monitor thread MAY be started by the java extension handler to monitor the progress of activity, as well as conformance to lease agreement. Periodically, the monitor thread reads the partial results (if any) and communicates them back to the LDAP server (after adding other fields of the response) for returning as part results to the client. The frequency at which partial results should be sent can be controlled by the client-proxy by way of generating the results. client-proxy code SHOULD catch all java exceptions that might occur during the execution of the run() method, and translate exceptions into results, to be returned back to the client. Server implementations SHOULD return stack- trace for any thrown exceptions back to the client, with appropriate error code. 7. ASN.1 Definition of Search Query Extension Comments on the design and utility of these definitions are specifically invited from all reviewers. The ASN.1 definition of extended operation as per LDAP (v3) specification [2] is as follows : ExtendedRequest ::= [APPLICATION 23] SEQUENCE { requestName [0] LDAPOID, requestValue [1] OCTET STRING OPTIONAL } ExtendedResponse ::= [APPLICATION 24] SEQUENCE { COMPONENTS OF LDAPResult, responseName [10] LDAPOID OPTIONAL, response [11] OCTET STRING OPTIONAL } where, LDAPOID is a dotted-decimal representation of the OBJECT IDENTIFIER corresponding to the request/response, and LDAPResult corresponds to the final status (success/failure indications) of the protocol operation request. In the following section, the ASN.1 definition of request and response values are presented. requestValue ::= SEQUENCE { leaseRequest LEASERequest, Goenka INTERNET DRAFT [Page 9] June 1999 ldap_sdk ENUMERATED { JNDI (0), LDAP_JavaAPI (1), -- 2 to 5 reserved -- Proprietary (6) } interface ClassName, serializedObject OCTET STRING, classes JavaARchive, classURL [0] LDAP_URL OPTIONAL, permissions [1] RequiredPermissions OPTIONAL } LEASERequest ::= SEQUENCE { estimatedDuration INTEGER (0 .. maxInt), -- in seconds -- maxDuration INTEGER (0 .. maxInt), estimatedMemory INTEGER (0 .. maxInt), -- in KB -- maxMemory INTEGER (0 .. maxInt) } RequiredPermissions ::= SEQUENCE { permType ClassName, permName OCTET STRING, critical BOOLEAN DEFAULT TRUE, actions [0] OCTET STRING OPTIONAL } ClassName ::= OCTET STRING JavaARchive ::= OCTET STRING - estimatedDuration -- duration in seconds, for which the client-proxy must execute to generate any useful results. - maxDuration -- the duration in seconds, which is the maximum time the client-proxy would need to execute completely. - estimatedMemory -- the memory requirement in KB, which the client-proxy must have to execute. - maxMemory -- the maximum amount of memory that the client-proxy would ever need. - ldap_sdk -- the java LDAP API SDK to which the serializedObject (client-proxy) is written. There are two possible standards emerging, namely, JNDI (Java Naming and Directory Interface) with LDAP binding, and java LDAP API [3]. It is expected that the server may already have the class files corresponding to standard APIs, in which case these classes need not be sent as part of the query. For clients which are written to proprietary java LDAP implementation, the client MUST send the entire LDAP implementation in a (optionally signed) jar, as part of the query, at the expense of the overhead involved. ASN.1 OIDs might be a better alternative for specifying the ldap_sdk than the current method via Goenka INTERNET DRAFT [Page 10] June 1999 enumeration. Comments. - interface -- the full name {package.class} of the class/interface corresponding to the serialized object. This is required by the server in order to appropriately deserialize the object and invoke its methods. - serializedObject -- the serialized java object which encapsulates the state and methods to execute the search query on the server. There is only one root object (implementing the given interface), which would be the entry point for execution. The root object may have references to other objects (deep copy of non-transient references) which are preserved during serialization. - classes -- the supporting class definitions (java byte code) for all the classes required to de-serialize and execute the client-proxy. This may include java implementations of LDAP, if the server doesn't support the ldap_sdk to which the client-proxy is written. The classes are packaged in a JAR (java archive), in the standard JAR format. - classURL -- specifies (optionally) the URL reference for the class definitions that might be required for executing the client-proxy. This would include class definitions for proprietary java LDAP implementations. The Server MAY NOT support downloading classes from a URL, in which case an appropriate error is returned. - permissions -- these correspond to java permissions that might be required by a client-proxy to execute. By default, the client-proxy may have a minimal set of permissions grant (similar to a restricted applet). Any specific permission must be approved by the server. This model assumes the jdk1.2 security architecture with respect to permissions. The permission requests have the following structure : - permType -- the java permission class governing the required permission. e.g., java.lang.RuntimePermission, java.io.FilePermission etc. - permName -- the name of the permission or first argument to the appropriate permission class, such as host name (for java.net.SocketPermission), createClassLoader (for java.lang.RuntimePermission) - critical -- determines whether this permission is a must for the client-proxy to execute. All critical permissions must be granted by the server, in order to execute the client-proxy. - actions -- second argument that some permissions take. e.g., for java.io.FilePermission the allowed actions are read, write, Goenka INTERNET DRAFT [Page 11] June 1999 execute, delete. Some permissions do not take a third argument (such as java.lang.RuntimePermission), and hence this is an optional argument. The ASN.1 definition for the extended response value follows : response ::= SEQUENCE { COMPONENTS OF LDAPExtendedResult, leaseGrant LEASEGrant, ldap_sdk_supported [0] ENUMERATED { JNDI (0), LDAP_JavaAPI (1), -- 2 to 5 reserved -- } OPTIONAL resultValue [1] OCTET STRING OPTIONAL resultDone BOOLEAN DEFAULT FALSE } LEASEGrant ::= SEQUENCE { duration INTEGER (0 .. maxInt), -- in seconds -- memory INTEGER (0 .. maxInt) } -- in KB -- LDAPExtendedResult ::= SEQUENCE { resultCode ENUMERATED { success (0), leaseExpired (1), resourceUseExceeded (2), securityViolation (3), noActivity (4), permissionDenied (5), dataError (6), serverBusy (7), leaseRenewed (8), leaseGranted (9), URL_ReferenceNotSupported (10), -- 11-30 unused -- other (31) } operationAbort BOOLEAN DEFAULT TRUE, description OCTET STRING OPTIONAL } - LDAPExtendedResult -- these return codes are specific to this extension and are in addition to the more general LDAPResult codes. They are used to communicate errors specific to the java execution of the client-proxy. The above enumerated list is clearly non-exhaustive, and have obvious meanings. If an error results in the abortion of client-proxy, the operationAbort flag is set true. description provides additional textual information about the error, such as the stack trace in case an exception is thrown in the run( socket) method. - LEASEGrant -- the duration (in seconds) and the memory (in KB) for which the lease is granted by the server. In case of a Goenka INTERNET DRAFT [Page 12] June 1999 lease renewal, these values are in addition to the values or previous lease grants. - ldap_sdk_supported -- the LDAP APIs supported by the server. A server may use this to communicate all SDKs that it supports. If the ldap_sdk that the client-proxy is written to, is not supported by the server, the client MUST send a new extended request, along with the class files for the ldap_sdk. - resultValue -- the result value is generated by the client-proxy, to be consumed only by the client, and hence can have any proprietary format, including but not limited to existing LDAP response formats, or even java object(s) etc. - resultDone -- when false, denotes that this is a partial result, and more partial results would follow. When true, denotes that this is the last and final part of the result. This allows the client- proxy to generate partial results, as appropriate, and communicate them to the client. 8. Implementation Considerations To make use of this proposal, the LDAP server should be capable of invoking a JVM, and should implement the proposed LDAP extension. Java LDAP APIs should be standardized and the client should be written to them. Until standard Java APIs for LDAP are available, clients MUST send all the java classes required for LDAP communication over a given socket. The LDAP clients should be enabled to send a search query encapsulated as a serialized java object using the LDAP extension API. Java Security Architecture version 1.2 is assumed on the server end for associating specific permissions with client-proxy code. 9. Acknowledgements Thanks are due to Brian Jarvis and Vipul Modi for their contributions to this work. 10. References [1] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access Protocol", RFC 2251. [2] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access Protocol (v3)", Internet Draft draft-ietf-asid-ldapv3-protocol-04.txt, March 1997. Goenka INTERNET DRAFT [Page 13] June 1999 [3] R. Weltman, T. Howes, M. Smith, Christine Ho, "The Java LDAP Application Program Interface", Internet Draft draft-ietf-ldapext-ldap-java-api-02.txt, July 1998. [4] T. Howes, "A String Representation of LDAP Search Filters," RFC 1960, June 1996. [5] David Boreham, Chris Weider, "LDAP Extensions for Scrolling View Browsing of Search Results", Internet Draft, draft-ietf-ldapext-ldapv3-vlv-01.txt, March, 1998 [6] Sun Microsystems, Inc. "Distributed Leasing Specification", Whitepaper (JINI), lease.pdf, Revision 1.0 Beta, July, 1998 11. Author's Address Vishal Goenka Novell, Inc. 122 East, 1700 South Provo, UT 84606 USA Phone: +1 801 861 4573 Fax: +1 801 861 2522 Email: vgoenka@novell.com