Network Working Group                                        B. Fiorelli
Internet-Draft                                                      RIPE
Intended status: Informational                              June 9, 2011
Expires: December 11, 2011


                       The RIPE Database REST API
                      draft-fiorelli-weirds-rws-00

Abstract

   This document describes the RIPE Database REST API.

   The purpose of this document is to facilitate discussion and serve as
   input into a standards process in this area, currently being
   discussed on the WHOIS-based Extensible Internet Registration Data
   Service (WEIRDS) mailing list
   (https://www.ietf.org/mailman/listinfo/weirds).

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 11, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of



Fiorelli                Expires December 11, 2011               [Page 1]


Internet-Draft                  RIPE-RWS                       June 2011


   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
     1.1.  Why REST? . . . . . . . . . . . . . . . . . . . . . . . . . 3
   2.  Some features of the Query Services . . . . . . . . . . . . . . 5
   3.  Some sample invocation of the query services  . . . . . . . . . 6
   4.  Some details on the CRUD Services . . . . . . . . . . . . . . . 6
   5.  Internationalization Considerations . . . . . . . . . . . . . . 7
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 7
   7.  Security Considerations . . . . . . . . . . . . . . . . . . . . 8
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 8
     8.1.  Normative References  . . . . . . . . . . . . . . . . . . . 8
     8.2.  Informative References  . . . . . . . . . . . . . . . . . . 8
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 8

































Fiorelli                Expires December 11, 2011               [Page 2]


Internet-Draft                  RIPE-RWS                       June 2011


1.  Introduction

   This document describes a pilot service for querying and displaying
   data in the RIPE database.  The service is implemented using the HTTP
   protocol [RFC2616] and conforms to the architectural constraints of
   REST [REST].  In this document, we explain our reasons for new
   interfaces (and limits of existing ones) as well as high level
   functionalities of the API.  We did not include technical details of
   our implementation, which can be found as a separate document and is
   available at:
   http://labs.ripe.net/Members/bfiorell/api-documentation.

1.1.  Why REST?

   The RIPE Database has a large number of power users: organisations
   and individuals that make or maintain software which depends on the
   RIPE Database.  The way these specialized clients interact with the
   RIPE Database is dictated by historical and technical aspects of RPSL
   and the Whois protocol.

   RPSL ([RFC2622]) is the routing policy specification language, but
   for historical and convenience reasons it actually specifies much
   more than routing policies, there are many RFCs related to the RPSL
   (see [RFC2622], [RFC2650], [RFC4012]) and in practice the language
   describes more than 20 different types of RPSL objects, it is more a
   formalisation of the way policy records have been stored and
   exchanged in the existing Whois systems since to the late 80's
   instead of a high level domain language specification.

   The result is that the policy specification language and its
   extensions are tightly coupled with the way policies are stored in
   the existing Whois systems and vice versa, so any new system that is
   implementing the language ends up reproducing a Whois system.

   The qualities of RPSL and Whois are well known.  In this section we
   highlight their limits by describing the way a client would interact
   with the two Whois interfaces of query and update.

   First scenario, a client needs to extract sensible information from
   one or more Internet Registries like the RIPE Database, the steps
   are:

      prepare one or more queries and invoke a command line Whois client
      program or open a TCP socket connection to one or more Whois
      servers,

      parse and post-process the RPSL responses, facing many intricacies
      like continuation lines, end of line comments, comma separated



Fiorelli                Expires December 11, 2011               [Page 3]


Internet-Draft                  RIPE-RWS                       June 2011


      values, attribute values that can reference to different types of
      RPSL objects depending on various conditions, etc.,

      if object references need to be resolved, extract referenced
      primary keys, execute other queries, parse again RPSL to identify
      the right referenced objects; this step can be recursive,

      merge results from multiple queries, maybe queries from multiple
      internet registries that may adopt different RPSL flavours,

      extract subsets of objects, by applying various criteria,

      extract subsets of attributes from set of objects, by applying
      various criteria.

   The other standard scenarios is that of update:

      in case of update or delete, fetch the object, this operation may
      require several of the steps described in the previous workflow,

      in case of a new object, build RPSL text, maybe starting
      converting business data into RPSL text,

      decorate the object with authorisation credentials, adding
      password hashes to the RPSL text or signing the RPSL text with a
      PGP key,

      send the text data via mail to the Mail Update interface or via
      HTTP post to the Syncupdates interface,

      in case of error, parse a human readable text response if specific
      error conditions must be handled in a special way.

   As shown above, such processes are modeled on a human centered
   workflow, they are not the optimal to build new services, extend
   existing ones or build tools on top of them.

   Concretely, this means clients have to handle too much complexity,
   many clients have problems at handling all the complexity of RPSL,
   many are too expensive to be extended and difficult to maintain.

   RPSL and Whois are still invaluable but it's evident that there is a
   need for simpler interfaces and machine ready data formats in order
   to simplify the development of services and tools and increase the
   value of Registries by exposing new domain specific interfaces.

   The first step can be the identification of a layer of Service
   Provider Interfaces and a data schema simple enough to be agnostic of



Fiorelli                Expires December 11, 2011               [Page 4]


Internet-Draft                  RIPE-RWS                       June 2011


   the underlying Registry implementation.

   The SPI would sit on top of a Registry Database allowing the
   composition of domain specific Services but also making real-time
   interoperation between Registries and services on multiple
   authoritative Databases possible to achieve.

   With these forces and this vision the RIPE Database Group has been
   prototyping the RIPE Whois RESTful Query and CRUD API and the related
   XML schemas.

   The choice of REST has been quite obvious, HTTP is nowadays the most
   accessible protocol and it represents a good architectural blueprint
   for stateless services, with its different HTTP methods, the resource
   locator protocol, the HTTPS, etc.

   For the representation schema we have designed a very relaxed
   attribute oriented XML Schema.  We only apply a structural validation
   via XSD, after some testing we decided to remove any form of
   attribute or type validation in order to reuse the same schema on
   different RPSL flavors.

   The services support representational styles JSON, HTML and plain
   text, all derived via XSL transformation, with the latter two to
   showcase the transformation powers of XML and also to demo resource
   navigation via lookup references on any HTML browser.

   Content negotiation can be done using HTTP headers or appending a
   file extension to the request URL.

   The query services can be used on any RPSL based Whois server or
   mirror.

   For example in one request it is possible to execute the same Lookup
   or Search request on all the Regional Internet Registries and return
   all the responses as a unique set of resources.

   Similarly can be done for the CRUD interfaces, building adapter
   modules for the different update mechanisms provided by different
   Registries given that they adopt the same set of resource types.

   Actually we implemented an adapter for the Whois Syncupdates
   interface in the form of an HTTP proxy.


2.  Some features of the Query Services





Fiorelli                Expires December 11, 2011               [Page 5]


Internet-Draft                  RIPE-RWS                       June 2011


      Single resource Lookup Service, given a primary key a type and a
      registry it always return one and only one object.  It can also be
      used to identify resources by URL bookmark.

      Resolution of referenced resources.  Given a resource, all the
      attribute values that represent references to other resource
      contain an xlink anchor that can be followed to navigate and
      browse networks of resources.

      HATEOAS, the client can navigate through any network of resources
      via xlinks without requiring any stateful information to be stored
      on the servers, this comes as a benefit of the two previous
      features.

      Normalisation of comma separated values, when they represent
      references to multiple resources.

      Normalisation of continuation lines, end of line comments and
      other RPSL intricacies.  This is still a work in progress and
      still not complete because is difficult to test all the side
      effects of this operation.  The only real solution would be to
      deprecate continuation lines in RPSL and split them into multiple
      attribute value pairs.

      All the Query Services are available also on HTTPS.


3.  Some sample invocation of the query services

   A Lookup Service URL for an organization object:
   http://apps.db.ripe.net/whois/lookup/ripe/organisation/ORG-BME1-RIPE

   A Lookup Service URL for a route object:
   http://apps.db.ripe.net/whois/lookup/ripe/route/193.6.23.0/24/AS2547

   A Search Service URL on multiple registries: http://apps.db.ripe.net/
   whois/
   search.xml?flags=r&source=ripe&source=apnic&source=afrinic&
   query-string=AS2547


4.  Some details on the CRUD Services

   The CRUD Services are an attempt to split the single overloaded
   generic update interface provided by Mail Updates and Syncupdates
   into separate low level interfaces, each defining a simpler contract,
   designed for programmatic use rather than for human interaction.




Fiorelli                Expires December 11, 2011               [Page 6]


Internet-Draft                  RIPE-RWS                       June 2011


   For example the existing update interfaces always require clients to
   provide the entire RPSL resource in exactly the same text-format as
   it is stored in the Whois database, even if the requested operation
   is just a deletion.  A client will have to fetch the RPSL object from
   the RIPE Database doing an appropriate query, than extract ony the
   text blob that represent the resource to be deleted and finally it
   will have to execute a request to the update service providing the
   entire text and adding delete attributes and authorisation
   credentials.

   The new CRUD Services on the other hand provide a Delete interface
   that only requires a primary key, a type, a registry identifier and
   one or more passwords.  It is the equivalent of a lookup but is
   executed with the HTTP Delete method.  It is exposed only on HTTPS.

   The client will just have to send a request like: HTTP DELETE: https:
   //lab.db.ripe.net/whois/delete/test/person/pp16-test?password=123 :

   The server will just respond with an HTTP Status code indicating
   success or failure, in case of failure different status codes will
   indicate different error conditions.

   On top of Create, Update and Delete methods we also prototyped some
   attribute modification services that in one only request can
   implement complex update workflow.

   For example with one only HTTP request will be possible to execute
   commands like:

      replace all the attributes of a given type with a new set of
      attribute,

      remove all the attributes that have value matching the given
      regular expression,

      add a new set of attributes after the Nth attribute of type X.


5.  Internationalization Considerations

   TBA


6.  IANA Considerations

   TBA





Fiorelli                Expires December 11, 2011               [Page 7]


Internet-Draft                  RIPE-RWS                       June 2011


7.  Security Considerations

   TBA.


8.  References

8.1.  Normative References

   [REST]     Fielding, R. and R. Taylor, "Principled Design of the
              Modern Web Architecture", ACM Transactions on Internet
              Technology Vol. 2, No. 2, May 2002.

   [RFC2616]  Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
              Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
              Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.

   [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
              Resource Identifier (URI): Generic Syntax", STD 66,
              RFC 3986, January 2005.

   [RFC3987]  Duerst, M. and M. Suignard, "Internationalized Resource
              Identifiers (IRIs)", RFC 3987, January 2005.

8.2.  Informative References

   [RFC2622]  Alaettinoglu, C., Villamizar, C., Gerich, E., Kessens, D.,
              Meyer, D., Bates, T., Karrenberg, D., and M. Terpstra,
              "Routing Policy Specification Language (RPSL)", RFC 2622,
              June 1999.

   [RFC2650]  Meyer, D., Schmitz, J., Orange, C., Prior, M., and C.
              Alaettinoglu, "Using RPSL in Practice", RFC 2650,
              August 1999.

   [RFC3707]  Newton, A., "Cross Registry Internet Service Protocol
              (CRISP) Requirements", RFC 3707, February 2004.

   [RFC3912]  Daigle, L., "WHOIS Protocol Specification", RFC 3912,
              September 2004.

   [RFC4012]  Blunk, L., Damas, J., Parent, F., and A. Robachevsky,
              "Routing Policy Specification Language next generation
              (RPSLng)", RFC 4012, March 2005.







Fiorelli                Expires December 11, 2011               [Page 8]


Internet-Draft                  RIPE-RWS                       June 2011


Author's Address

   Benedetto Fiorelli
   RIPE NCC
   P.O. Box 10096
   Amsterdam  1001EB
   The Netherlands

   Phone: +31.20.535.4444
   Email: bfiorell@ripe.net









































Fiorelli                Expires December 11, 2011               [Page 9]