| # Licensed to the Apache Software Foundation (ASF) under one or more |
| # contributor license agreements. See the NOTICE file distributed with |
| # this work for additional information regarding copyright ownership. |
| # The ASF licenses this file to You under the Apache License, Version 2.0 |
| # (the "License"); you may not use this file except in compliance with |
| # the License. You may obtain a copy of the License at |
| # |
| # http://www.apache.org/licenses/LICENSE-2.0 |
| # |
| # Unless required by applicable law or agreed to in writing, software |
| # distributed under the License is distributed on an "AS IS" BASIS, |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| # See the License for the specific language governing permissions and |
| # limitations under the License. |
| |
| A REST HTTP gateway for ZooKeeper |
| ================================= |
| |
| Specification Version: 2 |
| |
| ZooKeeper is meant to enable distributed coordination and also store |
| system configuration and other relatively small amounts of information |
| that must be stored in a persistent and consistent manner. The |
| information stored in ZooKeeper is meant to be highly available to a |
| large number of nodes in a distributed-computing cluster. |
| |
| ZooKeeper offers a client-side library that supports rich semantics |
| that include strict ordering guarantees on operations, the creation of |
| ephemeral znodes, and the ability to watch for changes to state. |
| However, where clients need simple "CRUD" (create, read, update, |
| delete) operations, the ZooKeeper libraries can be cumbersome, both to |
| the programmers who have to use them (who are increasingly used to |
| REST-style APIs), and to the operators who have to deploy and update |
| them (for whom deploying and updating client libraries can be very |
| painful). |
| |
| It turns out that most languages comes with client libraries for HTTP |
| that are easy and familiar to program against, and deployed as part of |
| the language runtime. Thus, for simple CRUD clients, an HTTP gateway |
| would be a less cumbersome interface than the ZooKeeper library. |
| |
| This document describes a gatway for using HTTP to interact with a |
| ZooKeeper repository. |
| |
| Binding ZooKeeper to HTTP |
| ------------------------- |
| |
| Encoding |
| -------- |
| |
| UTF-8 unless otherwise noted |
| |
| Paths |
| ----- |
| |
| A ZooKeeper paths are mapped to IRIs and URIs as follows. ZK paths |
| are converted to IRIs by simply percent-encoding any characters in the |
| ZK path that are not allowed in IRI paths. ZK paths are converted to |
| URIs by mapping them first to IRIs, then converting to URIs in the |
| standard way. |
| |
| Going from URIs and IRIs is the reverse of the above but for one |
| difference: any "." and ".." segments in an IRI or URI must be folded |
| before conversion. (Fortunately, ZK does not allow "." and ".." |
| segments in its paths.) |
| |
| ZK and IRIs recommend the same practices when it comes to Unicode |
| normalization: ultimately, normalization is left to application |
| designers, but both recommend that application designers use NFC as a |
| best practice. |
| |
| Root |
| ---- |
| |
| The following examples assume that the ZooKeeper znode heirarchy is |
| bound to the root of the HTTP servers namespace. This may not be the |
| case in practice however, the gateway may bind to some prefix, for |
| example the URL for accessing /a/b/c may be: |
| |
| http://localhost/zookeeper/znodes/v1/a/b/c |
| |
| This is perfectly valid. Users of the REST service should be aware of |
| this fact and code their clients to support any root (in this case |
| "/zookeeper" on the server localhost). |
| |
| |
| Basics: GET, PUT, HEAD, and DELETE |
| ---------------------------------- |
| |
| HTTP's GET, PUT, HEAD, and DELETE operations map naturally to |
| ZooKeeper's "get," "set," "exists," and "delete" operations. |
| |
| ZooKeeper znodes have a version number that changes each time the |
| znode's value is updated. This number is returned by "get," "set," and |
| "exists" operations. The "set" and "delete" operations optionally take |
| a version number. If one is supplied, then "set" or "delete" will fail |
| if the current version of the znode doesn't match the version-number |
| supplied in the call. This mechanism supports atomic read-modify-write |
| cycles. Set/delete requests may include an optional parameter |
| "version" which defaults to no version check. |
| |
| |
| Getting ZooKeeper children |
| -------------------------- |
| |
| We overload the GET method to return the children of a ZooKeeper. In |
| particular, the GET method takes an optional parameter "view" which |
| could be set to one of type values, either "data" or "children". The |
| default is "data". Thus, to get the children of a znode named |
| "/a/b/c", then the GET request should start: |
| |
| GET /znodes/v1/a/b/c?view=children HTTP/1.1 |
| |
| If the requested view is "data", then the data of a znode is returned |
| as described in the previous section. If the requested view is |
| "children", then a list of children is returned in either an XML |
| document, or in a JSON object. (The default is JSON, but this can be |
| controlled changed by setting the Accept header.) |
| |
| |
| Creating a ZooKeeper session |
| ---------------------------- |
| |
| In order to be able to create ephemeral nodes you first need to start |
| a new session. |
| |
| POST /sessions/v1?op=create&expire=<SECONDS> HTTP/1.1 |
| |
| If the session creation is successful, then a 201 code will be returned. |
| |
| A session is just an UUID that you can pass around as a parameter and |
| the REST server will foward your request on the attached persistent |
| connection. |
| |
| Keeping a session alive |
| ----------------------- |
| |
| To keep a session alive you must send hearbeat requests: |
| |
| PUT /sessions/v1/<SESSION-UUID> HTTP/1.1 |
| |
| Closing a ZooKeeper session |
| --------------------------- |
| |
| You can close a connection by sending a DELETE request. |
| |
| DELETE /sessions/v1/<SESSION-UUID> HTTP/1.1 |
| |
| If you don't close a session it will automatically expire after |
| the amount of time you specified on creation. |
| |
| Creating a ZooKeeper znode |
| -------------------------- |
| |
| We use the POST method to create a ZooKeeper znode. For example, to |
| create a znode named "c" under a parent named "/a/b", then the POST |
| request should start: |
| |
| POST /znodes/v1/a/b?op=create&name=c HTTP/1.1 |
| |
| If the creation is successful, then a 201 code will be returned. If |
| it fails, then a number of different codes might be returned |
| (documented in a later subsection). |
| |
| ZooKeeper's create operation has a flag that tells the server to |
| append a sequence-number to the client-supplied znode-name in order to |
| make the znode-name unique. If you set this flag and ask to create a |
| znode named "/a/b/c", and a znode named "/a/b" already exists, then |
| "create" will create a znode named "/a/b/c-#" instead, where "#" is and |
| integer required to generate a unique name in for format %10d. |
| |
| To obtain this behavior, an additional "sequence=true" parameter |
| should be added to the parameters of the POST. (Note that "sequence" |
| is an optional parameter, that defaults to "false"; this default may |
| be provided explicitly if desired.) |
| |
| On success the actual path of the created znode will be returned. |
| |
| If you want to create an ephemeral node you need to specify an |
| additional "ephemeral=true" parameter. (Note that "ephemeral" is an optional |
| parameter, that defaults to "false") |
| |
| (Note: ZooKeeper also allows the client to set ACLs for the |
| newly-created znode. This feature is not currently supported by the |
| HTTP gateway to ZooKeeper.) |
| |
| |
| Content types and negotiation |
| ----------------------------- |
| |
| ZooKeeper REST gateway implementations may support three content-types |
| for request and response messages: |
| |
| * application/octet-stream |
| |
| HEAD - returns nothing (note below: status = 204) |
| GET - returns the znode data as an octet-stream |
| PUT - send binary data, returns nothing |
| POST - send binary data, returns the name of the znode |
| DELETE - returns nothing |
| |
| For PUT and HEAD some other content-type (i.e. JSON or XML) must be |
| used to access the Stat information of a znode. |
| |
| * application/json, application/javascript & application/xml |
| |
| HEAD - returns nothing |
| GET - returns a STAT or CHILD structure |
| PUT - send binary data, returns a STAT structure (sans data field) |
| POST - send binary data, returns a PATH structure |
| DELETE - returns nothing |
| |
| (structures defined below) |
| |
| Results returning DATA may include an optional "dataformat" |
| parameter which has two possible values; base64 (default) or |
| utf8. This allows the caller to control the format of returned data |
| and may simplify usage -- for example cat'ing results to the command |
| line with something like curl, or accessing a url through a browser. |
| Care should be exercised however, if utf8 is used on non character |
| data errors may result. |
| |
| "application/javascript" requests may include an optional "callback" |
| parameter. The response is wrapped in a callback method of your |
| choice. e.g. appending &callback=foo to your request will result in |
| a response body of: foo(...). Callbacks may only contain |
| alphanumeric characters and underscores. |
| |
| PATH |
| path : string |
| uri: string |
| |
| path is the full path to the znode as seen by ZooKeeper |
| |
| uri is the full URI of the znode as seen by the REST server, does not |
| include any query parameters (i.e. it's the path to the REST resource) |
| |
| SESSION |
| id : string UUID |
| uri : string |
| |
| CHILD |
| PATH |
| child_uri_template: string |
| children : [ string* ] |
| |
| The children list of strings contains only the name of the child |
| znodes, not the full path. |
| |
| child_uri_template is a template for URI of child znodes as seen by the |
| REST server. e.g. "http://localhost:9998/znodes/v1/foo/{child}", where |
| foo is the parent node, and {child} can be substituted with the name |
| of each child in the children array in order to access that resource. |
| This template is provided to simplify child access. |
| |
| STAT |
| PATH |
| encoding : value of "base64" or "utf8" |
| data : base64 or utf8 encoded string |
| stat : |
| czxid : number |
| mzxid : number |
| ctime : number |
| mtime : number |
| version : number |
| cversion : number |
| aversion : number |
| ephemeralOwner : number |
| datalength : number |
| numChildren : number |
| pzxid : number |
| |
| |
| Error Codes |
| ----------- |
| |
| The ZooKeeper gateway uses HTTP response codes as follows: |
| |
| * 200 (Success) - ZOK for "get" "set" "delete", "yes" case of "exists" (json/xml) |
| * 201 (Created) - ZOK for "create" |
| * 204 (No Content) - ZOK for "yes" case of "exists" (octet) |
| * 400 (Bad Request) - ZINVALIDACL, ZBADARGUMENTS, version param not a number |
| * 401 (Unauthorized) - ZAUTHFAILED |
| * 404 (Not Found) - ZOK for "no" case of "exists;" ZNONODE for "get," "set," and "delete" |
| * 409 (Conflict) - ZNODEEXISTS, ZNONODE for "create," ZNOTEMPTY, |
| * 412 (Precondition Failed) - ZBADVERSION |
| * 415 (Unsupported Media Type) - if content-type of PUT or POST is not "application/octet-stream" |
| * 500 (Internal Server Error) - Failure in gateway code |
| * 501 (Not Implemented) - HTTP method other than GET, PUT, HEAD, DELETE |
| * 502 (Bad Gateway) - All other ZooKeeper error codes |
| * 503 (Service Unavailable) - ZSESSIONEXPIRED, ZCONNECTIONLOSS, (gateway will try to reestablish the connection, but will not hold the request waiting...) |
| * 504 (Gateway Timeout) - ZOPERATIONTIMEOUT, or ZooKeeper does not return in a timely manner |
| |
| Note that these are the codes used by the HTTP-to-Gateway software |
| itself. Depending on how this software is configured into a Web |
| server, the resulting Web Server might behave differently, e.g., it |
| might do redirection, check other headers, etc. |
| |
| Error Messages |
| -------------- |
| |
| Error messages are returned to the caller, format is dependent on the |
| format requested in the call. |
| |
| * application/octet-stream |
| |
| A string containing the error message. It should include the request |
| and information detailing the reason for the error. |
| |
| * application/json |
| |
| { "request":"GET /a/b/c", "message":"Node doesn't exist" } |
| |
| * application/xml |
| |
| <?xml version="1.0" encoding="UTF-8"?> |
| <error> |
| <request>GET /a/b/c</request> |
| <message>Node doesn't exist</message> |
| </error> |
| |
| |
| Binding ZooKeeper to an HTTP server |
| ----------------------------------- |
| |
| It might be sage to assume that everyone is happy to run an Apache |
| server, and thus write a "mod_zookeeper" for Apache that works only |
| for the Apache Web Server. However, different operational |
| environments prefer different Web Servers, and it would be nice to |
| support more than one Web server. |
| |
| Issues: |
| |
| * Configuration. |
| |
| * Defining a root: Need to provide a URL alias and associate it |
| with a server. Need to be able to map different aliases to |
| different servers (implemented via multiple ZK connections). |
| |
| * Sharing connection across multiple processes. |
| |
| * Asynchronous. |
| |
| * Adaptors. |
| |
| * Code re-use. |
| |
| |
| Authentication -- TBD, not currently supported |
| |
| ...the config file should contain authentication material for the gateway |
| |
| ...the config file should contain an ACL list to be passed along to "create" |
| |
| ...would we ever want to authenticate each request to ZooKeeper?... |