├── .gitignore ├── Makefile ├── conf.py ├── forms.rst ├── images ├── concepts.png ├── concepts.svg ├── scope.png └── scope.svg ├── index.rst ├── intro.rst ├── methods.rst ├── misc.rst ├── relationships.rst ├── resources.rst ├── scope.rst └── urls.rst /.gitignore: -------------------------------------------------------------------------------- 1 | _build 2 | _static 3 | _templates 4 | .*.swp 5 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | 15 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest 16 | 17 | help: 18 | @echo "Please use \`make ' where is one of" 19 | @echo " html to make standalone HTML files" 20 | @echo " dirhtml to make HTML files named index.html in directories" 21 | @echo " singlehtml to make a single large HTML file" 22 | @echo " pickle to make pickle files" 23 | @echo " json to make JSON files" 24 | @echo " htmlhelp to make HTML files and a HTML help project" 25 | @echo " qthelp to make HTML files and a qthelp project" 26 | @echo " devhelp to make HTML files and a Devhelp project" 27 | @echo " epub to make an epub" 28 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 29 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 30 | @echo " text to make text files" 31 | @echo " man to make manual pages" 32 | @echo " changes to make an overview of all changed/added/deprecated items" 33 | @echo " linkcheck to check all external links for integrity" 34 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 35 | 36 | clean: 37 | -rm -rf $(BUILDDIR)/* 38 | 39 | html: 40 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 41 | @echo 42 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 43 | 44 | dirhtml: 45 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 46 | @echo 47 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 48 | 49 | singlehtml: 50 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 51 | @echo 52 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 53 | 54 | pickle: 55 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 56 | @echo 57 | @echo "Build finished; now you can process the pickle files." 58 | 59 | json: 60 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 61 | @echo 62 | @echo "Build finished; now you can process the JSON files." 63 | 64 | htmlhelp: 65 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 66 | @echo 67 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 68 | ".hhp project file in $(BUILDDIR)/htmlhelp." 69 | 70 | qthelp: 71 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 72 | @echo 73 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 74 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 75 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Real-worldREST.qhcp" 76 | @echo "To view the help file:" 77 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Real-worldREST.qhc" 78 | 79 | devhelp: 80 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 81 | @echo 82 | @echo "Build finished." 83 | @echo "To view the help file:" 84 | @echo "# mkdir -p $$HOME/.local/share/devhelp/Real-worldREST" 85 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Real-worldREST" 86 | @echo "# devhelp" 87 | 88 | epub: 89 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 90 | @echo 91 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 92 | 93 | latex: 94 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 95 | @echo 96 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 97 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 98 | "(use \`make latexpdf' here to do that automatically)." 99 | 100 | latexpdf: 101 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 102 | @echo "Running LaTeX files through pdflatex..." 103 | make -C $(BUILDDIR)/latex all-pdf 104 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 105 | 106 | text: 107 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 108 | @echo 109 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 110 | 111 | man: 112 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 113 | @echo 114 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 115 | 116 | changes: 117 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 118 | @echo 119 | @echo "The overview file is in $(BUILDDIR)/changes." 120 | 121 | linkcheck: 122 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 123 | @echo 124 | @echo "Link check complete; look for any errors in the above output " \ 125 | "or in $(BUILDDIR)/linkcheck/output.txt." 126 | 127 | doctest: 128 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 129 | @echo "Testing of doctests in the sources finished, look at the " \ 130 | "results in $(BUILDDIR)/doctest/output.txt." 131 | -------------------------------------------------------------------------------- /conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # RESTful API Deisgn build configuration file, created by 4 | # sphinx-quickstart on Mon Jul 25 09:21:34 2011. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | #sys.path.insert(0, os.path.abspath('.')) 20 | 21 | # -- General configuration ----------------------------------------------------- 22 | 23 | # If your documentation needs a minimal Sphinx version, state it here. 24 | #needs_sphinx = '1.0' 25 | 26 | # Add any Sphinx extension module names here, as strings. They can be extensions 27 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 28 | extensions = [] 29 | 30 | # Add any paths that contain templates here, relative to this directory. 31 | templates_path = ['_templates'] 32 | 33 | # The suffix of source filenames. 34 | source_suffix = '.rst' 35 | 36 | # The encoding of source files. 37 | #source_encoding = 'utf-8-sig' 38 | 39 | # The master toctree document. 40 | master_doc = 'index' 41 | 42 | # General information about the project. 43 | project = u'RESTful API Design' 44 | copyright = u'2011, Geert Jansen' 45 | 46 | # The version info for the project you're documenting, acts as replacement for 47 | # |version| and |release|, also used in various other places throughout the 48 | # built documents. 49 | # 50 | # The short X.Y version. 51 | version = '1.0' 52 | # The full version, including alpha/beta/rc tags. 53 | release = '1.0' 54 | 55 | # The language for content autogenerated by Sphinx. Refer to documentation 56 | # for a list of supported languages. 57 | #language = None 58 | 59 | # There are two options for replacing |today|: either, you set today to some 60 | # non-false value, then it is used: 61 | #today = '' 62 | # Else, today_fmt is used as the format for a strftime call. 63 | #today_fmt = '%B %d, %Y' 64 | 65 | # List of patterns, relative to source directory, that match files and 66 | # directories to ignore when looking for source files. 67 | exclude_patterns = ['_build'] 68 | 69 | # The reST default role (used for this markup: `text`) to use for all documents. 70 | #default_role = None 71 | 72 | # If true, '()' will be appended to :func: etc. cross-reference text. 73 | #add_function_parentheses = True 74 | 75 | # If true, the current module name will be prepended to all description 76 | # unit titles (such as .. function::). 77 | #add_module_names = True 78 | 79 | # If true, sectionauthor and moduleauthor directives will be shown in the 80 | # output. They are ignored by default. 81 | #show_authors = False 82 | 83 | # The name of the Pygments (syntax highlighting) style to use. 84 | pygments_style = 'sphinx' 85 | highlight_language = 'guess' 86 | 87 | # A list of ignored prefixes for module index sorting. 88 | #modindex_common_prefix = [] 89 | 90 | 91 | # -- Options for HTML output --------------------------------------------------- 92 | 93 | # The theme to use for HTML and HTML Help pages. See the documentation for 94 | # a list of builtin themes. 95 | html_theme = 'agogo' 96 | 97 | # Theme options are theme-specific and customize the look and feel of a theme 98 | # further. For a list of options available for each theme, see the 99 | # documentation. 100 | #html_theme_options = {} 101 | 102 | # Add any paths that contain custom themes here, relative to this directory. 103 | #html_theme_path = [] 104 | 105 | # The name for this set of Sphinx documents. If None, it defaults to 106 | # " v documentation". 107 | #html_title = 'Thoughts on RESTful API Design' 108 | html_title = '' 109 | 110 | # A shorter title for the navigation bar. Default is the same as html_title. 111 | #html_short_title = None 112 | 113 | # The name of an image file (relative to this directory) to place at the top 114 | # of the sidebar. 115 | #html_logo = None 116 | 117 | # The name of an image file (within the static path) to use as favicon of the 118 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 119 | # pixels large. 120 | #html_favicon = None 121 | 122 | # Add any paths that contain custom static files (such as style sheets) here, 123 | # relative to this directory. They are copied after the builtin static files, 124 | # so a file named "default.css" will overwrite the builtin "default.css". 125 | html_static_path = ['_static'] 126 | 127 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 128 | # using the given strftime format. 129 | #html_last_updated_fmt = '%b %d, %Y' 130 | 131 | # If true, SmartyPants will be used to convert quotes and dashes to 132 | # typographically correct entities. 133 | #html_use_smartypants = True 134 | 135 | # Custom sidebar templates, maps document names to template names. 136 | #html_sidebars = {} 137 | 138 | # Additional templates that should be rendered to pages, maps page names to 139 | # template names. 140 | #html_additional_pages = {} 141 | 142 | # If false, no module index is generated. 143 | #html_domain_indices = True 144 | 145 | # If false, no index is generated. 146 | #html_use_index = True 147 | 148 | # If true, the index is split into individual pages for each letter. 149 | #html_split_index = False 150 | 151 | # If true, links to the reST sources are added to the pages. 152 | #html_show_sourcelink = True 153 | 154 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 155 | #html_show_sphinx = True 156 | 157 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 158 | #html_show_copyright = True 159 | 160 | # If true, an OpenSearch description file will be output, and all pages will 161 | # contain a tag referring to it. The value of this option must be the 162 | # base URL from which the finished HTML is served. 163 | #html_use_opensearch = '' 164 | 165 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 166 | #html_file_suffix = None 167 | 168 | # Output file base name for HTML help builder. 169 | htmlhelp_basename = 'restful-api-design' 170 | 171 | 172 | # -- Options for LaTeX output -------------------------------------------------- 173 | 174 | # The paper size ('letter' or 'a4'). 175 | #latex_paper_size = 'letter' 176 | 177 | # The font size ('10pt', '11pt' or '12pt'). 178 | #latex_font_size = '10pt' 179 | 180 | # Grouping the document tree into LaTeX files. List of tuples 181 | # (source start file, target name, title, author, documentclass [howto/manual]). 182 | latex_documents = [ 183 | ('index', 'restful-api-design.tex', u'Thoughts on RESTful API design', 184 | u'Geert Jansen', 'manual'), 185 | ] 186 | 187 | # The name of an image file (relative to this directory) to place at the top of 188 | # the title page. 189 | #latex_logo = None 190 | 191 | # For "manual" documents, if this is true, then toplevel headings are parts, 192 | # not chapters. 193 | #latex_use_parts = False 194 | 195 | # If true, show page references after internal links. 196 | #latex_show_pagerefs = False 197 | 198 | # If true, show URL addresses after external links. 199 | #latex_show_urls = False 200 | 201 | # Additional stuff for the LaTeX preamble. 202 | #latex_preamble = '' 203 | 204 | # Documents to append as an appendix to all manuals. 205 | #latex_appendices = [] 206 | 207 | # If false, no module index is generated. 208 | #latex_domain_indices = True 209 | 210 | 211 | # -- Options for manual page output -------------------------------------------- 212 | 213 | # One entry per manual page. List of tuples 214 | # (source start file, name, description, authors, manual section). 215 | man_pages = [ 216 | ('index', 'restful-api-design', u'Thoughts on RESTful API design', 217 | [u'Geert Jansen'], 1) 218 | ] 219 | -------------------------------------------------------------------------------- /forms.rst: -------------------------------------------------------------------------------- 1 | ===== 2 | Forms 3 | ===== 4 | 5 | In this section, a solution is proposed to a usability issue that is present 6 | in a lot of RESTful APIs today. This issue is that, without referring to 7 | external documentation, an API user does not know what data to provide to 8 | operations that take input. For example, the POST call is used to create a new 9 | resource in a collection, and an API user can find out with the OPTIONS call 10 | that a certain collection supports POST. However, he does not know what data 11 | is required to create the new resource. So far, most RESTful APIs require the 12 | user to refer to the API documentation to get that information. This violates 13 | an important RESTful principle that APIs should be self-descriptive. 14 | 15 | As far as I can see, this usability issue impacts two important use cases: 16 | 17 | 1. A person is browsing the API, trying to use and learn it as he goes. The 18 | person can either be using a web browser, or a command-line tool like 19 | "curl". 20 | 2. The development of a command-line or graphical interface to a RESTful API. 21 | Without a proper description of required input types, knowledge about 22 | these has to be encoded explicitly in the client. This has the disadvantage 23 | that new features are not automatically exposed, and that there are strict 24 | requirements around not making backwards incompatible changes. 25 | 26 | When we look at the web, this issue doesn't exist. Millions of users interact 27 | with all kinds of IT systems over the web at any given moment, and none of 28 | them has to resort to API documentation to understand how to do that. 29 | Interactions with IT systems on the web are all self-explanatory using 30 | hypertext, and input is guided by forms. 31 | 32 | In fact, forms are what I see as the solution here. But before I go into that, 33 | let's look into two classes of solutions that have been proposed to 34 | address this issue, and why I think they actually do not solve it. The first 35 | is using a type definition language like XMLSchema to describe the input 36 | types, the second is using some kind of service description language like 37 | WADL. 38 | 39 | Type Definition 40 | =============== 41 | 42 | As alluded to in :doc:`resources`, some RESTful APIs use a type definition 43 | language to provide information about how to construct request entities. In my 44 | view this is a bad approach, and has the following issues: 45 | 46 | 1. It creates a tight coupling between servers and clients. 47 | 2. It still does not allow a plain web browser as an HTTP client. 48 | 3. It does not help in the automatic construction of CLIs. 49 | 50 | Often, XMLSchema is used as the type definition language. This brings in a few 51 | more issues: 52 | 53 | 1. XMLSchema is a horribly complicated specification on one side, but on the 54 | other side it can't properly solve the very common requirement of an 55 | unordered set of key:value pairs (Note that does not solve this. 56 | Its limitations make it almost useless.). 57 | 2. Type definitions in XMLSchema are context-free. This means that you 58 | cannot just define one type, you need to define one type for PUT, one for 59 | POST, etc. And since one XML element can only have one type, this means 60 | you're PUTing a different element that you'd POST or GET. This breaks that 61 | "homogeneous collection" principle of REST. 62 | 63 | Service Definition 64 | ================== 65 | 66 | `WADL `_ is an approach to define a 67 | service description language for RESTful APIs. It tries to solve the problem 68 | by meticulously defining entry points and parameters to those. The problem I 69 | have with WADL, is that it feels very non-RESTful. I can't see a lot of 70 | difference between a method on a WADL resource, and an RPC entry point. 71 | 72 | It would be possible to construct a more RESTful service description language. 73 | It would focus on mapping the RESTful concepts of resource, collection, 74 | relationships, and links. It would probably need a type definition language as 75 | well to define the constraints on types for each method (again, PUT may have 76 | different constraints than POST). There have been discussions in the RHEV-M 77 | project on this. 78 | 79 | In the end though, this approach also feels wrong. What I think is needed is 80 | something that works like the web, where everything is self descriptive, and 81 | guided only by the actual URL flow the user is going through, not by reference 82 | to some external description. 83 | 84 | Using Forms to Guide Input 85 | ========================== 86 | 87 | The right solution to guide client input in my view is to use forms. Note that 88 | I'm using the word "forms" here in a generic sense, as an entity that guides 89 | user input, and not necessarily equal to an HTML form. 90 | 91 | In my view, forms need to specify three pieces of information: 92 | 93 | 1. How to contact the target and format the input. 94 | 2. A list of all available input fields. 95 | 3. A list of constraints which which the input fields must comply. 96 | 97 | HTML forms provide for #1 and #2, but not for #3 above. #3 is typically 98 | achieved using client-side Javascript. Because Javascript is not normally 99 | available to clients that use the API, we need to define another way to do 100 | validation. The approach I propose here is to define a form definition 101 | language that captures the information #1 through #3. A form expressed in this 102 | language can be sent over to the client as-is, in case the client understands 103 | the form language. It can also be translated into an HTML form with Javascript 104 | in case the client is a web browser. 105 | 106 | Form Definition Language 107 | ------------------------ 108 | 109 | Our forms are defined as a special RESTful resource with type "form". This 110 | means they use the JSON data model, can be represented in JSON, YAML and XML 111 | according to the rules in :doc:`resources`. 112 | 113 | Forms have their own content type. This is listed in the table below: 114 | 115 | ==== ============================ 116 | Type Content-Type 117 | ==== ============================ 118 | Form | application/x-form+json 119 | | application/x-form+yaml 120 | | application/x-form+xml 121 | ==== ============================ 122 | 123 | Rather than providing a formal definition, I will introduce the form 124 | definition language by using an example. Below is an example of a form that 125 | guides the the creation of a virtual machine. In YAML format:: 126 | 127 | !form 128 | method: POST 129 | action: {url} 130 | type: vm 131 | fields: 132 | - name: name 133 | type: string 134 | regex: [a-zA-Z0-9]{5,32} 135 | - name: description 136 | type: string 137 | maxlen: 128 138 | - name: memory 139 | type: number 140 | min: 512 141 | max: 8192 142 | - name: restart 143 | type: boolean 144 | - name: priority 145 | type: number 146 | min: 0 147 | max: 100 148 | constraints: 149 | - sense: mandatory 150 | field: name 151 | - sense: optional 152 | field: description 153 | - sense: optional 154 | field: cpu.cores 155 | - sense: optional 156 | field: cpu.sockets 157 | - sense: optional 158 | exclusive: true 159 | constraints: 160 | - sense: mandatory 161 | field: highlyavailable 162 | - sense: optional 163 | field: priority 164 | 165 | As can be seen the form consists of 3 parts: form metadata, field definitions, 166 | and constraints. 167 | 168 | Form Metadata 169 | ------------- 170 | 171 | The form metadata is very simple. The following attributes are defined: 172 | 173 | ========= ============================================================ 174 | Attribute Description 175 | ========= ============================================================ 176 | method The HTTP method to use. Can be GET, POST, PUT or DELETE. 177 | url The URL to submit this form to. 178 | type The type of the resource to submit. 179 | ========= ============================================================ 180 | 181 | Fields 182 | ------ 183 | 184 | The list of available fields are specified using the "fields" attribute. This 185 | should be a list of field definitions. Each field definition has the following 186 | attributes: 187 | 188 | ========= =============================================================== 189 | Attribute Description 190 | ========= =============================================================== 191 | name The field name. Should be in dotted-name notation. 192 | type One of "string", "number" or "boolean" 193 | min Field value must be greater than or equal to this (numbers) 194 | max Field value must be less than or equal to this (numbers) 195 | minlen Minimum field length (strings) 196 | maxlen Maximum field length (strings) 197 | regex Field value needs to match this regular expression (strings) 198 | multiple Boolean that indicates if multiple values are accepted (array). 199 | ========= =============================================================== 200 | 201 | Constraints 202 | ----------- 203 | 204 | First we need to answer the question of what kind of constraints we want to 205 | express in our form definition language. I will start by mentioning that in my 206 | view, it is impossible to express each and every constraint on the client side. Some 207 | constraints for example require access to other data (e.g. when creating 208 | relationships), are computationally intensive, or even unknown to the API 209 | designer because they are undocumented for the application the API is written 210 | for. So in my view we need to find a good subset that is useful, without 211 | making it too complicated and without worrying about the fact that some 212 | constraints possibly cannot not be expressed. 213 | 214 | This leads me to define the following two kinds of constraints, which in my 215 | view are both useful, as well as sufficient for our purposes: 216 | 217 | 1. Constraints on individual data values. 218 | 2. Presence constraints on fields, i.e. whether a field is allowed or not 219 | allowed, and if allowed, whether it is mandatory. 220 | 221 | The constraints on data values are useful because CLIs and GUIs could use this 222 | information to help a user input data. For example, depending on the type, a 223 | GUI could render a certain field a checkbox, a text box, or a dropdown list. 224 | For brevity, constraints on individual data values are defined as part of the 225 | field definition, and were discussed in the previous section. 226 | 227 | Presence constraints are also useful, as they allow an API user to generate a 228 | concise synopsis on how to call a certain operation to be used e.g. in CLIs. 229 | Presence constraints specify which (combination of) input fields can and 230 | cannot exist. Each presence constraint has the following attributes: 231 | 232 | =========== ============================================================ 233 | Attribute Description 234 | =========== ============================================================ 235 | sense One of "mandatory" or "optional" 236 | field This constraint refers to a field. 237 | constraints This constraint is a group with nested constraints. 238 | exclusive This is an exclusive group (groups only). 239 | =========== ============================================================ 240 | 241 | Either "field" or "constraints" has to be specified, but not both. A 242 | constraint that has the field attribute set is called a simple constraint. A 243 | constraint that has the constraints attribute set, is called a group. 244 | 245 | Checking Constraints 246 | -------------------- 247 | 248 | Value constraints should be checked first, and should be checked only on 249 | non-null values. 250 | 251 | After value constraints, the presence constraints should to be checked. This 252 | is a bit more complicated because the constraints are not only used for making 253 | sure that all mandatory fields exist, but also that no non-optional fields are 254 | present. The following algorithm should be used: 255 | 256 | 1. Start with an empty list called "referenced" that will collect all 257 | referenced fields. 258 | 2. Walk over all constraints in order. If a constraint is a group, you need 259 | to recurse into it, depth first, passing it the "referenced" list. 260 | 3. For every simple constraint that validates, add the field name to 261 | "referenced". 262 | 4. In exclusive groups, matching stops at the first matching sub-constraint, 263 | in which case the group matches. In non-exclusive groups, matching stops at 264 | the first non-matching sub-constraint, in which case the group does not 265 | match. 266 | 5. When matching a group, you need to backtrack to the previous value of 267 | "referenced" in case the group does not match. 268 | 6. A constraint only fails if it is mandatory and it is a top-level 269 | constraint. If a constraint fails, processing may stop. 270 | 7. When you've walked through all constraints, it is an error if there are 271 | fields that have a non-null value but are not in the referenced list. 272 | 273 | Building the Request Entity 274 | --------------------------- 275 | 276 | After all constraints have been satisfied, a client should build a request 277 | entity that it will pass in the body of the POST, PUT or DELETE method. In 278 | case the form was requested in JSON, YAML or XML format, it is assumed that 279 | the client is not a web browser, and the following applies: 280 | 281 | First, a new resource is created of the type specified in the form metadata. 282 | Dotted field names should be interpreted as follows. Each dot creates a new 283 | object, and stores it under the name immediately left of the dot in its parent 284 | object. This means that the parent must be an object as well, which means it 285 | cannot correspond to a field definition with "multiple" set to "true" (which 286 | would make it a list). This resource is then represented in a format that the 287 | server supports, using the rules described in :doc:`resources`. 288 | 289 | If a client requested a "text/html" representation of the form, it is assumed 290 | that the client is a web browser, and we assume the form will be processed as 291 | a regular HTML form. In this case: 292 | 293 | * An HTML
should be generated, with an appropriate element for 294 | each field. 295 | * The HTML form's "method" attribute should be set to POST, unconditionally. In 296 | case the RESTful form's method is not POST, the server should include a 297 | hidden input element with the name "_method" to indicate to the server the 298 | original method. (HTML does not support PUT or DELETE in a form). 299 | * The form's "enctype" should be set to "multipart/form-data" or 300 | "application/x-www-form-urlencoded", as appropriate for the input elements. 301 | * A hidden field called "_type" is generated that contains the value of the 302 | "type" attribute in the form metadata. 303 | * The server may generate Javascript and include that in the HTML to check the 304 | value and presence constraints. 305 | 306 | Linking to Forms 307 | ================ 308 | 309 | Forms are associated with resources and collections by link objects. The name 310 | of the link object defines the meaning of the form. The following standard 311 | forms names are defined: 312 | 313 | ============ ========== ============================= 314 | Name Scope Description 315 | ============ ========== ============================= 316 | form/search collection Form to search for resources 317 | form/create collection Form to create a new resource 318 | form/update resource Form to update a resource 319 | form/delete resource Form to delete a resource 320 | ============ ========== ============================= 321 | -------------------------------------------------------------------------------- /images/concepts.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geertj/restful-api-design/b16658d0fcada422a8eb947e2609b72902042a5d/images/concepts.png -------------------------------------------------------------------------------- /images/concepts.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 21 | 23 | 41 | 43 | 44 | 46 | image/svg+xml 47 | 49 | 50 | 51 | 52 | 53 | 57 | 64 | 71 | Sub-collections andSub-resources 86 | 94 | 102 | 110 | 118 | 129 | 137 | Collection 148 | 156 | 164 | 172 | Resource 183 | 191 | Resource 202 | A Collection withResources 217 | A SingletonResource 232 | 240 | Collection 251 | 259 | Resource 270 | 278 | 286 | 294 | 302 | 310 | Sub- collection 321 | Sub-resources 336 | 344 | 355 | Resource Model 366 | 371 | 372 | 373 | -------------------------------------------------------------------------------- /images/scope.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geertj/restful-api-design/b16658d0fcada422a8eb947e2609b72902042a5d/images/scope.png -------------------------------------------------------------------------------- /images/scope.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 21 | 23 | 30 | 35 | 36 | 38 | 42 | 46 | 47 | 54 | 59 | 60 | 67 | 73 | 74 | 81 | 87 | 88 | 89 | 109 | 111 | 112 | 114 | image/svg+xml 115 | 117 | 118 | 119 | 120 | 121 | 126 | 134 | 142 | 150 | Application 162 | 170 | API 181 | to_resource()from_resource() 198 | Resource Model 210 | ApplicationData Model 227 | 235 | Client 246 | 251 | 256 | 268 | 283 | 288 | 300 | /entrypoint /collection1 /resource1 /resource2 /collection2 /resource1 /resrouce2 .... 339 | application/x-collectionapplication/x-resource 360 | 378 | HTTP Library 389 | 407 | REST Library(optional) 424 | 432 | Client App 443 | 450 | Server 461 | 468 | Client 479 | 487 | 495 | 500 | 505 | ApplicationInterface 522 | HTTP 534 | 535 | 536 | -------------------------------------------------------------------------------- /index.rst: -------------------------------------------------------------------------------- 1 | .. RESTful API Design master file, created by sphinx-quickstart on Mon 2 | Jul 25 09:21:34 2011. You can adapt this file completely to your 3 | liking, but it should at least contain the root `toctree` directive. 4 | 5 | ======================================================================= 6 | Thoughts on RESTful API Design 7 | ======================================================================= 8 | Lessons learnt from designing the Red Hat Enterprise Virtualization API 9 | ----------------------------------------------------------------------- 10 | 11 | Author: Geert Jansen 12 | 13 | Date: November 15th, 2012 14 | 15 | This work is licensed under a `Creative Commons Attribution 3.0 Unported 16 | License `_. 17 | 18 | The source of this document can be found `on github 19 | `_. 20 | 21 | .. toctree:: 22 | :maxdepth: 2 23 | 24 | intro 25 | scope 26 | resources 27 | urls 28 | methods 29 | relationships 30 | forms 31 | misc 32 | -------------------------------------------------------------------------------- /intro.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Introduction 3 | ============ 4 | 5 | This essay is an attempt to put down my thoughts on how to design a 6 | real-world yet beautiful RESTful API. It draws from the experience I have 7 | gained being involved in the design of the `RESTful API 8 | `_ for Red Hat's Enterprise 9 | Virtualization product, `twice `_. During 10 | the design phase of the API we had to solve many of the real-world problems 11 | described above, but we weren't willing to add non-RESTful or "RPC-like" 12 | interfaces to our API too easily. 13 | 14 | In my definition, a real-world RESTful API is an API that provides answers to 15 | questions that you won't find in introductory texts, but that inevitably 16 | surface in the real world, such as whether or not resources should be 17 | described formally, how to create useful and automatic command-line 18 | interfaces, how to do polling, asynchronous and other non-standard types of 19 | requests, and how to deal with operations that have no good RESTful mapping. 20 | 21 | A beautiful RESTful API on the other hand is one that does not deviate from 22 | the principles of RESTful architecture style too easily. One important design 23 | element for example that is not always addressed is the possibility for 24 | complete auto-discovery by the API user, so that the API can be used by a 25 | human with a web browser, without any reference to external documentation. I 26 | will discuss this issue in detail in :doc:`forms`. 27 | 28 | This essay does not attempt to explain the basics about REST, as there are 29 | many other texts about that. For a good description of REST, I would recommend 30 | reading at least `the Atom Publishing Protocol 31 | `_ and `this blog post 32 | `_ by 33 | Roy Fielding. 34 | 35 | While the views exposed in this essay are my own, they are heavily based on 36 | the public discussion on the `rhevm-api mailing list 37 | `_. My thanks goes to 38 | all people who have contributed and are still contributing to it, including 39 | Mark McLoughlin, Michael Pasternak, Ori Liel, Sander Hoentjen, Ewoud Kohl van 40 | Wijngaarden, Tomas V.V. Cox and everybody else I forgot. 41 | -------------------------------------------------------------------------------- /methods.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Methods 3 | ======= 4 | 5 | Standard Methods 6 | ================ 7 | 8 | We already discussed that resources are the fundamental concept in a RESTful 9 | API, and that each resource has its own unique URL. Methods can be executed on 10 | resources via their URL. 11 | 12 | The table below lists the standard methods that have a well-defined meaning for 13 | all resources and collections. 14 | 15 | ======= ========== ================================================== 16 | Method Scope Semantics 17 | ======= ========== ================================================== 18 | GET collection Retrieve all resources in a collection 19 | GET resource Retrieve a single resource 20 | HEAD collection Retrieve all resources in a collection (header only) 21 | HEAD resource Retrieve a single resource (header only) 22 | POST collection Create a new resource in a collection 23 | PUT resource Update a resource 24 | PATCH resource Update a resource 25 | DELETE resource Delete a resource 26 | OPTIONS any Return available HTTP methods and other options 27 | ======= ========== ================================================== 28 | 29 | Normally, not all resources and collections implement all methods. There are 30 | two ways to find out which methods are accepted by a resource or collection. 31 | 32 | 1. Use the OPTIONS method on the URL, and look at the "Allow" header that is 33 | returned. This header contains a comma-separated list of methods are are 34 | supported for the resource or collection. 35 | 2. Just issue the method you want to issue, but be prepared for a "405 Method 36 | Not Allowed" response that indicates the method is not accepted for this 37 | resource or collection. 38 | 39 | 40 | Actions 41 | ======= 42 | 43 | Sometimes, it is required to expose an operation in the API that inherently is 44 | non RESTful. One example of such an operation is where you want to introduce a 45 | state change for a resource, but there are multiple ways in which the same 46 | final state can be achieved, and those ways actually differ in a significant 47 | but non-observable side-effect. Some may say such transitions are bad API 48 | design, but not having to model all state can greatly simplify an API. A great 49 | example of this is the difference between a "power off" and a "shutdown" of a 50 | virtual machine. Both will lead to a vm resource in the "DOWN" state. 51 | However, these operations are quite different. 52 | 53 | As a solution to such non-RESTful operations, an "actions" sub-collection can 54 | be used on a resource. Actions are basically RPC-like messages to a resource 55 | to perform a certain operation. The "actions" sub-collection can be seen as a 56 | command queue to which new action can be POSTed, that are then executed by the 57 | API. Each action resource that is POSTed, should have a "type" attribute that 58 | indicates the type of action to be performed, and can have arbitrary other 59 | attributes that parameterize the operation. 60 | 61 | It should be noted that actions should only be used as an exception, when 62 | there's a good reason that an operation cannot be mapped to one of the 63 | standard RESTful methods. If an API has too many actions, then that's an 64 | indication that either it was designed with an RPC viewpoint rather than using 65 | RESTful principles, or that the API in question is naturally a better fit for 66 | an RPC type model. 67 | 68 | 69 | PATCH vs PUT 70 | ============ 71 | 72 | The HTTP RFC specifies that PUT must take a full new resource representation as 73 | the request entity. This means that if for example only certain attributes are 74 | provided, those should be removed (i.e. set to null). 75 | 76 | An additional method called PATCH `has been proposed recently 77 | `_. The semantics of this call are like PUT 78 | inthat it updates a resource, but unlike PUT, it applies a delta rather than 79 | replacing the entire resource. At the time of writing, PATCH was still a 80 | proposed standard waiting final approval. 81 | 82 | For simple resource representations, the difference is often not important, and 83 | many APIs simply implement PUT as a synonym for PATCH. This usually doesn't 84 | give any problems because it is not very common that you need to set an 85 | attribute to null, and if you need to, you can always explicitly include it. 86 | 87 | However for more complex representations, especially including lists, it 88 | becomes very important to be able to express accurately the changes you want to 89 | make. Therefore, it is my recommendation now to both provide PATCH and PUT, and 90 | make PATCH do an relative update and have PUT replace the entire resource. 91 | 92 | It is important to realize that the request entity to PATCH is of a different 93 | content-type than the entity that it is modifying. Instead of being a full 94 | resource, it is a resource that describes modifications to be made to a 95 | resource. For a JSON data model, which is what this essay is advocating, I 96 | believe that there are two sensible ways to define the patch format. 97 | 98 | 1. An informal approach where you accept a dict with a partial representation 99 | of the object. Only attributes that are present are updated. Attributes that 100 | are not present are left alone. This approach is simple, but it has the 101 | drawback that if the resource has a complex internal structure e.g. 102 | containing a big list of dicts, then that entire list of dicts need to be 103 | given in the entity. Effectively PATCH becomes similar to PUT again. 104 | 105 | 2. A more formal approach would be to accept a list of modifications. Each 106 | modification can be a dict specifying the JSON path of the node to modify, 107 | the modification ('add', 'remove', 'change') and the new value. 108 | 109 | 110 | Link Headers 111 | ============ 112 | 113 | An `Internet draft 114 | `_ exists 115 | defining the "Link:" header type. This header takes the "link" attributes of 116 | the response entity, and formats them as an HTTP header. It is argued that the 117 | Link header is useful because it allows a client to quickly get links from a 118 | response without having to parse the response entity, or even without 119 | retrieving the response entity at all using the HEAD HTTP method. 120 | 121 | In my view, the usefulness of this feature is dubious. First of all, it can 122 | increase response sizes quite significantly. Second, it can only be used when 123 | a resource is being returned; it does not make sense to be used with 124 | collections. Because I have not yet seen any good use of this header in the 125 | context of a RESTful API, I recommend not to implement Link headers. 126 | 127 | 128 | Asynchronous Requests 129 | ===================== 130 | 131 | Sometimes an action takes too long to be completed in the context of a single 132 | HTTP request. In that case, a "202 Accepted" status can be returned to the 133 | client. Such a response should only be returned for POST, PUT, PATCH or DELETE. 134 | 135 | The response entity of a 202 Accepted response should be a regular resource 136 | with only the information filled in that was available at the time the request 137 | was accepted. The resource should contain a "link" attribute that points to a 138 | status monitor that can be polled to get updated status information. 139 | 140 | When polling the status monitor, it should return a "response" object with 141 | information on the current status of the asynchronous request. If the request 142 | is still in progress, such a response could look like this (in YAML):: 143 | 144 | !response 145 | status: 202 Accepted 146 | progress: 50% 147 | 148 | If the call has finished, the response should include the same headers and 149 | response body had the request been fulfilled synchronously:: 150 | 151 | !response 152 | status: 201 Created 153 | headers: 154 | - name: content-type 155 | value: applicaton/x-resource+yaml 156 | response: !!str 157 | Response goes here 158 | 159 | After the response has been retrieved once with a status that is not equal to 160 | "202 Accepted", the API code may garbage collect it and therefore clients 161 | should not assume it will continue to be available. 162 | 163 | A client may request the server to modify its asynchronous behavior with the 164 | following "Expect" headers: 165 | 166 | * "Expect: 200-ok/201-created/204-no-content" disables all asynchronous 167 | functionality. The server may return a "417 Expectation Failed" 168 | if it is not willing to wait for an operation to complete. 169 | * "Expect: 202-accepted" explicitly request an asynchronous response. The 170 | server may return a "417 Expectation Failed" if it is not willing to perform 171 | the request asynchronously. 172 | 173 | If no expectation is provided, client must be prepared to accept a 202 174 | Accepted status for any request other than GET. 175 | 176 | 177 | Ranges / Pagination 178 | =================== 179 | 180 | When collections contain many resources, it is quite a common 181 | requirement for a client to retrieve only a subset of the available resources. 182 | This can be implemented using the Range header with a "resource" range unit: 183 | 184 | .. code-block:: none 185 | 186 | GET /api/collection 187 | Range: resources=100-199 188 | 189 | The above example would return resources 100 through 199 (inclusive). 190 | 191 | Note that it is the responsibility of the API implementer to ensure a proper 192 | and preferably meaningful ordering can be guaranteed for the resources. 193 | 194 | Servers should provide an "Accept-Ranges: resource" header to indicate to a 195 | client that they support resource-based range queries. This header should be 196 | provided in an OPTIONS response: 197 | 198 | .. code-block:: none 199 | 200 | OPTIONS /api/collection HTTP/1.1 201 | 202 | HTTP/1.1 200 OK 203 | Accept-Ranges: resources 204 | 205 | 206 | Notifications 207 | ============= 208 | 209 | Another common requirement is where a client wants to be notified immediately 210 | when some kind of event happens. 211 | 212 | Ideally, such a notification would be implemented using a call-out from the 213 | server to the client. However, there is no good portable standard to do this 214 | over HTTP, and it also breaks with network address translation and HTTP 215 | proxies. A second approach called busy-loop polling is horribly inefficient. 216 | 217 | In my view, the best approach is what is is called "long polling". In long 218 | polling, the client will retrieve a URL but the server will not generate a 219 | response yet. The client will wait for a configurable amount of time, until it 220 | will close the connection and reconnect. If the server becomes aware of an 221 | event that requires notification of clients, it can provide that event 222 | immediately to clients that are currently waiting. 223 | 224 | Long polling should be disabled by default, and can be enabled by a client 225 | using an Expect header. For example, a client could long poll for new 226 | resources in a collection using a combination of long-polling and a 227 | resource-based range query: 228 | 229 | .. code-block:: none 230 | 231 | GET /api/collection 232 | Range: 100- 233 | Expect: nonempty-response 234 | 235 | In this case, resource "100" would be the last resource that was read, and the 236 | call is requesting the API to return at least one resource with an ID > 100. 237 | 238 | Server implementers need to decide whether they want to implement long polling 239 | using one thread per waiting client, or one thread that uses multiplexed IO to 240 | wait for all clients. This is a trade-off to be made between ease of 241 | implementation and scalability (that said, threads are pretty cheap on modern 242 | operating systems). 243 | -------------------------------------------------------------------------------- /misc.rst: -------------------------------------------------------------------------------- 1 | ==================== 2 | Miscellaneous Topics 3 | ==================== 4 | 5 | Offline Help 6 | ============ 7 | 8 | A requirement that sometimes comes up is that to have an offline description 9 | of the API that can be used to generate offline help for a command-line 10 | interface (CLI) for that API. When connected to an API, all this information 11 | is of course available because of the self-descriptive nature of a RESTful 12 | API. But the request is to have that information offline as well, so that CLI 13 | users can get help without have to connect first. 14 | 15 | In my view it is a bad idea to offer this offline help. It introduces a 16 | tight coupling between a client and a server, which will break the RESTful 17 | model. A server cannot be independently upgraded anymore from a client, and 18 | clients become tied to a specific server version. Also I doubt the usefulness 19 | of this feature as having to connect first to get help does not seem like a 20 | big issue to me. 21 | 22 | That said, it is relatively straightforward to define a way that such an offline 23 | description can be facilitated. 24 | 25 | One server-side change is required for this. For each collection, the API 26 | should implement a placeholder resource with a special ID (let's say "_", but 27 | it doesn't matter as long as it cannot be a valid ID), and some fake data. 28 | When a special HTTP Expect header is set, only this placeholder resource is 29 | returned when querying a collection. 30 | 31 | Having the placeholder resources in place for every collection, the following 32 | procedure can be used by the client to retrieve all relevant metadata: 33 | 34 | 1. Retrieve the entry point of the API and store it in memory as a resource. 35 | 2. For every hyperlink in the entry point, fetch the target, and store it 36 | under the "target" property under the link object, recursively. When 37 | collections are retrieved, the special Expect header must be set. 38 | 3. For every hyperlink in the entry point, execute an OPTIONS call on the 39 | target, and store the resulting headers under the "options" property of the 40 | link object, recursively. 41 | 4. Serialize the resulting object to a file. 42 | 43 | The result of this is basically a recursive dump of the "GET"-able part of an 44 | API with one placeholder resource in every collection. It will contain all 45 | metadata including forms, options, available resource types and collections, 46 | in a format that is very similar to the real on-line data. It can be used as 47 | an off-line store to generate any help that you would be able to generate 48 | online as well. 49 | 50 | An API can store a version number on its entry point, so that clients can see 51 | when their offline cached copy expired and can therefore generate a new one. 52 | -------------------------------------------------------------------------------- /relationships.rst: -------------------------------------------------------------------------------- 1 | ============= 2 | Relationships 3 | ============= 4 | 5 | As we have seen in :doc:`resources`, the resource is the fundamental unit in 6 | RESTful API design. Resources model objects from the application data model. 7 | 8 | Resources do not exist in isolation, but have relationships to other 9 | resources. Sometimes these relationships exist between the mapped objects in 10 | the application data model as well, sometimes they are specific to the RESTful 11 | resources. 12 | 13 | One of the principles of the RESTful architecture style is that these 14 | relationships are expressed by hyperlinks to the representation of a resource. 15 | 16 | In our resource model, we interpret any object with an "href" attribute as a 17 | hyperlink. The value of the href attribute contains an absolute URL that can 18 | be retrieved with GET. Using GET on a such a URL is guaranteed to be 19 | side-effect free. 20 | 21 | Two types of hyperlinks are normally used: 22 | 23 | 1. Using "link" objects. This is a special object of type "link" with 24 | "href" and a "rel" attributes. The "rel" attribute value identifies the 25 | semantics of the relationship. The link object is borrowed from 26 | `HTML `_. 27 | 2. Using any object with a "href" attribute. The object type defines the 28 | semantics of the relationship. I'll call these "object links," not to be 29 | confused with the "link objects" above. 30 | 31 | Below is an example of a virtual machine representation in YAML that 32 | illustrates the two different ways in which a relationship can be expressed:: 33 | 34 | !vm 35 | id: 123 36 | href: /api/vms/123 37 | link: 38 | rel: collection/nics 39 | href: /api/vms/123/nics 40 | cluster: 41 | href: /api/clusters/456 42 | 43 | The "link" object is used with rel="collection/nics" to refer to a 44 | sub-collection to the VM holding the virtual network interfaces. The cluster 45 | link instead points to the cluster containing this VM by means of a "cluster" 46 | object. 47 | 48 | Note that the VM itself has a href attribute as well. This is called a "self" 49 | link, and it is a convention for a resource to provide such a link. That way, 50 | the object can later be easily re-retrieved or updated without keeping 51 | external state. 52 | 53 | There are no absolute rules when to use link objects, and when not. That said, 54 | we have been using the rules below in the rhevm-api project with good success. 55 | They seem to be a good trade-off between compactness (favoring object links) 56 | and the ability to understand links without understanding the specific 57 | resource model (favoring link objects). 58 | 59 | * Link objects are used to express structural relationships in the API. So for 60 | example, the top-level collections, singleton resources and sub-collections 61 | (including actions) are all referenced using link objects. 62 | * Object links are used to express semantic relationships from the 63 | application data model. In the example above, the vm to cluster link comes 64 | directly from the application data model and is therefore modeled as a link. 65 | 66 | Note however that sub-collections can express both a semantic relationship, 67 | as well as a structural relationship. See for example the "collection/nics" 68 | sub-collection to a VM in the example above. In such a case, our convention 69 | has been to use link objects. 70 | 71 | 72 | Standard Structural Relationships 73 | --------------------------------- 74 | 75 | It makes sense to define a few standard structural relationship names that can 76 | be relevant to many different APIs. In this way, client authors know what to 77 | expect and can write to some extent portable clients that work with multiple 78 | APIs. The table below contains a few of these relationship names. 79 | 80 | ================= ===================================== 81 | Relationship Semantics 82 | ================= ===================================== 83 | collection/{name} Link to a related collection {name}. 84 | resource/{name} Link to a related resource {name}. 85 | form/{name} Link to a related form {name}. 86 | ================= ===================================== 87 | 88 | Here, {name} is replaced with a term that has meaning for the specific API. 89 | For example, a collection of virtual machines could be linked from the entry 90 | point of a virtualization API using rel="collection/vms". 91 | 92 | 93 | Modeling Semantic Relationships 94 | ------------------------------- 95 | 96 | Semantic relationships can be modeled either by an object link, or by a 97 | sub-collection. I believe that choosing the right way to represent a 98 | sub-collection is important to get a consistent API experience for the client. 99 | I would advocate using the following rules: 100 | 101 | 1. In case of a 1:N relationship, where the target object is **existentially 102 | dependent** on the source object, I'd recommend to use a sub-collection. 103 | By "existentially dependent" I mean that a target object cannot exist 104 | without its source. In database parlance, this would be a FOREIGN KEY 105 | relationship with an ON DELETE CASCADE. An example of such a relationship 106 | are the NICs that are associated with a VM. 107 | 2. In case of a 1:N relationship, where there is data that is associated with 108 | the link, I'd recommend to use a sub-collection. Note that we are talking 109 | about data here that is neither part of the source object, nor the target 110 | object. The resources in the sub-collection can hold the extra data. In 111 | this case, the data in the sub-resource would therefore be a merge of the 112 | data from the mapped object from the application data model, and link data. 113 | 3. In case of any other 1:N relationship, I'd recommend to use an object link. 114 | 4. In case of a N:M relationship, I'd recommend to use a sub-collection. 115 | The sub-collection can be defined initially to support the lookup direction 116 | that you most often need to follow. If you need to query both sides of the 117 | relationship often, then two sub-collections can be defined, one on each 118 | linked object. 119 | -------------------------------------------------------------------------------- /resources.rst: -------------------------------------------------------------------------------- 1 | ========= 2 | Resources 3 | ========= 4 | 5 | The fundamental concept in any RESTful API is the *resource*. A resource is 6 | an object with a type, associated data, relationships to other resources, and 7 | a set of methods that operate on it. It is similar to an object 8 | instance in an object-oriented programming language, with the important 9 | difference that only a few standard methods are defined for the resource 10 | (corresponding to the standard HTTP GET, POST, PUT and DELETE methods), while 11 | an object instance typically has many methods. 12 | 13 | Resources can be grouped into *collections*. Each collection is homogeneous so 14 | that it contains only one type of resource, and unordered. Resources can also 15 | exist outside any collection. In this case, we refer to these resources as 16 | *singleton resources*. Collections are themselves resources as well. 17 | 18 | Collections can exist globally, at the top level of an API, but can also be 19 | contained inside a single resource. In the latter case, we refer to these 20 | collections as *sub-collections*. Sub-collections are usually used to express 21 | some kind of "contained in" relationship. We go into more detail on this in 22 | :doc:`relationships`. 23 | 24 | The diagram below illustrates the key concepts in a RESTful API. 25 | 26 | .. image:: images/concepts.png 27 | :width: 90% 28 | :align: center 29 | 30 | We call information that describes available resources types, their behavior, 31 | and their relationships the *resource model* of an API. The resource model can 32 | be viewed as the RESTful mapping of the application data model. 33 | 34 | Resource Data 35 | ============= 36 | 37 | Resources have data associated with them. The richness of data that can be 38 | associated with a resource is part of the *resource model* for an API. It 39 | defines for example the available data types and their behavior. 40 | 41 | Based on my experience, I have developed a strong conviction that the JSON 42 | data model has just the right "richness" so that it is an ideal data model for 43 | RESTful resources. I would recommend that everybody use it. 44 | 45 | In JSON, just three types of data exist: 46 | 47 | * scalar (number, string, boolean, null). 48 | * array 49 | * object 50 | 51 | Scalar types have just a single value. Arrays contain an ordered list of 52 | values of arbitrary type. Objects consist of a unordered set of key:value 53 | pairs (also called attributes, not to be confused with XML attributes), where 54 | the key is a string and the value can have an arbitrary type. For more 55 | detailed information on JSON, see the `JSON web site `_. 56 | 57 | Why the strong preference for JSON? In my view, JSON has the right balance 58 | between expressiveness, and broad availability. The three types of data 59 | (scalars, arrays and objects) are powerful enough to describe in a natural way 60 | virtually all the data that you might want to expose as resource, and at the 61 | same time these types are minimal enough so that almost any modern language 62 | has built-in support for them. 63 | 64 | XML would be the other obvious contender. Actually, in the final incarnation 65 | of the RHEV-M API, XML is used to describe resources, via an XMLSchema 66 | definition. With hindsight, I believe that the XML data model is a bad choice 67 | for a RESTful API. On one side, it is too rich, and on the other side, it 68 | lacks features. XML, as an SGML off-shoot, is in my view great for 69 | representing structured documents, but not for representing structured data. 70 | 71 | Features of XML that are too rich include: 72 | 73 | 1. Attributes vs elements. An XML element can have both attributes as well as 74 | sub-elements. A data item associated with a resource could be encoded in 75 | either one, and it would not be clear beforehand which one a client or a 76 | server should use. 77 | 2. Relevance of order. The order between child-elements is relevant in XML. It 78 | is not natural in my view for object attributes to have ordering. 79 | 80 | The limitations of the XML data model are: 81 | 82 | 1. Lack of types. Elements in XML documents do not have types, and in order to 83 | use types, one would have to use e.g. XMLSchema. XMLSchema unfortunately is 84 | a strong contender for the most convoluted specification ever written. 85 | 2. Lack of lists. XML cannot natively express lists. This can lead to issues 86 | whereby it is not clear whether a certain element is supposed to be a list 87 | or an object, and where that element ends up being both. 88 | 89 | Application Data 90 | ---------------- 91 | 92 | We define the data that can be associated with a resource in terms of the JSON 93 | data model, using the following mapping rules: 94 | 95 | 1. Resources are modeled as a JSON object. The type of the resource is 96 | stored under the special key:value pair "_type". 97 | 2. Data associated with a resource is modeled as key:value pairs on the 98 | JSON object. To prevent naming conflicts with internal key:value pairs, 99 | keys must not start with "_". 100 | 3. The values of key:value pairs use any of the native JSON data types of 101 | string, number, boolean, null, or arrays thereof. Values can also be 102 | objects, in which case they are modeling nested resources. 103 | 4. Collections are modeled as an array of objects. 104 | 105 | We will also refer to key:value pairs as attributes of the JSON object, and we 106 | will be sloppy and use that same term for data items associated with resources, 107 | too. This use of attributes is not to be confused with XML attributes. 108 | 109 | REST Metadata 110 | ------------- 111 | 112 | In addition to exposing application data, resources also include other 113 | information that is specific to the RESTful API. Such information includes 114 | URLs and relationships. 115 | 116 | The following table lists generic attributes that are defined and have a 117 | specific meaning on all resources. They should not be used for mapping 118 | application model attributes. 119 | 120 | ========= ====== ========================================== 121 | Attribute Type Meaning 122 | ========= ====== ========================================== 123 | id String Identifies the unique ID of a resource. 124 | href String Identifies the URL of the current resource. 125 | link Object Identifies a relationship for a resource. 126 | This attribute is itself an object and has 127 | "rel" "href" attributes. 128 | ========= ====== ========================================== 129 | 130 | Other Data 131 | ---------- 132 | 133 | Apart from application data, and REST metadata, sometimes other data is 134 | required as well. This is usually "RPC like" data where a setting is needed 135 | for an operation, and where that setting will not end up being part of the 136 | resource itself. 137 | 138 | One example that I can give here is where a resource creation needs a 139 | reference to another resource that is used during the creation, but where that 140 | other resource will not become part of the resource itself. 141 | 142 | It is the responsibility of the API code to merge the application data 143 | together with the REST metadata and the other data into a single resource, 144 | resolving possible naming conflicts that may arise. 145 | 146 | Representations 147 | =============== 148 | 149 | We have defined resources, and defined the data associated with them in terms 150 | of the JSON data model. However, these resources are still abstract entities. 151 | Before they can be communicated to a client over an HTTP connection, they need 152 | to be serialized to a textual representation. This representation can then be 153 | included as an entity in an HTTP message body. 154 | 155 | The following representations are common for resources. The table also lists 156 | the appropriate content-type to use: 157 | 158 | ==== =========================================================== 159 | Type Content-Type 160 | ==== =========================================================== 161 | JSON | application/x-resource+json 162 | | application/x-collection+json 163 | YAML | application/x-resource+yaml 164 | | application/x-collection+yaml 165 | XML | application/x-resource+xml 166 | | application/x-collection+xml 167 | HTML text/html 168 | ==== =========================================================== 169 | 170 | Note: all these content types use the "x-" experimental prefix that is allowed 171 | by `RFC2046 `_. 172 | 173 | JSON Format 174 | ----------- 175 | 176 | Formatting a resource to JSON is trivial because the data model of a 177 | resource is defined in terms of the JSON model. Below we give an example of a 178 | JSON serialization of a virtual machine:: 179 | 180 | { 181 | "_type": "vm", 182 | "name": "A virtual machine", 183 | "memory": 1024, 184 | "cpu": { 185 | "cores": 4, 186 | "speed": 3600 187 | }, 188 | "boot": { 189 | "devices": ["cdrom", "harddisk"] 190 | } 191 | } 192 | 193 | 194 | YAML Format 195 | ----------- 196 | 197 | Formatting to YAML is only slightly different than representing a resource in 198 | JSON. The resource type that is stored under the "_type" key/value pair is 199 | serialized as a YAML "!type" annotation instead. The same virtual machine as 200 | above, now in YAML format:: 201 | 202 | !vm 203 | name: A virtual machine 204 | memory: 1024 205 | cpu: 206 | cores: 4 207 | speed: 3600 208 | boot: 209 | devices: 210 | - cdrom 211 | - harddisk 212 | 213 | XML Format 214 | ---------- 215 | 216 | XML is the most complex representation format due to both its complexity as 217 | well as its limitations. I recommend the following rules: 218 | 219 | * Resources are mapped to XML elements with a tag name equal to the resource 220 | type. 221 | * Attributes of resources are mapped to XML child elements with the tag name 222 | equal to the attribute name. 223 | * Scalar values are stored as text nodes. A special "type" attribute on the 224 | containing element should be used to refer to an `XML Schema Part 2 225 | `_ type definition. 226 | * Lists should be stored as a single container element with child elements for 227 | each list item. The tag of the container element should be the English 228 | plural of the attribute name. The item tag should be the English singular of 229 | the attribute name. Lists should have the "xd:list" type annotation. 230 | 231 | The same VM again, now in XML:: 232 | 233 | 234 | My VM 235 | 1024 236 | 237 | 4 238 | 3600 239 | 240 | 241 | 242 | cdrom 243 | harddisk 244 | 245 | 246 | 247 | 248 | 249 | HTML Format 250 | ----------- 251 | 252 | The exact format of a HTML response can be API dependent. HTML is for human 253 | consumption, and the only requirement is therefore that it be easy to 254 | understand. A simple implementation may choose the following representation: 255 | 256 | * For a collection, a with a column per attribute where each object 257 | is listed in a separate row. 258 | * For a resource, a
with two columns, one with all the attribute 259 | names, one with the corresponding attribute value. 260 | 261 | Content-Types 262 | ============= 263 | 264 | As can be seen above, I am advocating the use of a generic content types 265 | "application/x-resource+format" and "application/x-collection+format". In my 266 | view this represents the right middle ground between two extremes that are 267 | commonly found in RESTful APIs: 268 | 269 | Some RESTful APIs only use the "bare" XML, JSON or YAML content types. An 270 | example of such as API is the Red Hat Enterprise Virtualization API. In this 271 | case, the content type expresses nothing but the fact that an entity is in 272 | XML, JSON or YAML format. In my view, this is not sufficient. Resources and 273 | collections have some specific semantics around for example the use of "href" 274 | attributes, "link" attributes, and types. Therefore, these are a 275 | specialization of XML, JSON and YAML and should be defined as such. 276 | 277 | Other RESTful APIs define a content-type for every resource type that exists 278 | in the resource model. Examples of such APIs include for example VMware's 279 | `vSphere Director API `_. In 280 | my view, this is not proper either. Specifying detailed content types invites 281 | both the API implementer, as well as a client implementer to think about these 282 | types as having specific interfaces. In my view though, all resources should 283 | share the same basic interface, which is defined by the RESTful design 284 | principles and expressed by the "application/x-resource" content type. 285 | 286 | One reason that is sometimes given in favor of defining detailed content types 287 | is that this way, the content type can be associated with a specific 288 | definition in some type definition language (such as XMLSchema). This, 289 | supposedly, facilitates client auto-discovery because a client can know 290 | available attributes for a certain type. I go into a lot of detail on this 291 | topic in :doc:`forms` but the summary is that I do not agree with this 292 | argument. 293 | 294 | Selecting a Representation Format 295 | --------------------------------- 296 | 297 | Clients can express their preference for a certain representation format using 298 | the HTTP "Accept" header. The HTTP RFC defines an `elaborate set of rules 299 | `_ in which 300 | multiple formats can be requested, each with its own priority. In the 301 | following example, the client tells the API that it accepts only YAML input: 302 | 303 | .. code-block:: none 304 | 305 | GET /api/collection 306 | Accept: application/x-collection+yaml 307 | -------------------------------------------------------------------------------- /scope.rst: -------------------------------------------------------------------------------- 1 | =========================== 2 | The Job of the API Designer 3 | =========================== 4 | 5 | Before we dive fully into RESTful API design, it makes sense to look in a 6 | little more detail what the job of a RESTful API designer is. 7 | 8 | APIs don't exist in isolation. Instead, APIs expose functionality of an 9 | application or service that exists independently of the API. In my view, the 10 | job of the RESTful API designer is two-fold. 11 | 12 | 1. Understanding enough of the important details of the application for which 13 | an API is to be created, so that an informed decision can be made as to what 14 | functionality needs to be exposed, how it needs to be exposed, and what 15 | functionality can instead be left out. 16 | 2. Modeling this functionality in an API that addresses all use cases that 17 | come up in the real world, following the RESTful principles as closely as 18 | possible. 19 | 20 | There are three distinct components involved in RESTful API design: the 21 | application, the API code, and the client. The image above illustrates how 22 | these three components interact. 23 | 24 | .. image:: images/scope.png 25 | :width: 90% 26 | :align: center 27 | 28 | 29 | The Application 30 | =============== 31 | 32 | The application for which an API is to be provided exists independently of 33 | that API. Maybe the application has a GUI, and you have a requirement to add a 34 | programmatic interface to it. Or maybe the application was designed under the 35 | assumption that it would be accessed only via the API you are designing. 36 | 37 | Like any application, the application for which the API is to be created 38 | contains state. That state is dynamic, and will change due to various 39 | operations that are executed on it. This state, and the operations on it, need 40 | to be modeled and exposed, and will form the API you are designing. 41 | 42 | The easiest way to think about the application state, is to assume that it is 43 | is described by an *application data model*, which can be described by an 44 | *Entity-Relationship diagram*. The ER-Diagram lists the details of the 45 | *entities* (I will call them *objects*) in the application state, and the 46 | *relationships* between them. 47 | 48 | In some cases, it is very easy to create the ER diagram. For example, in case 49 | of a web application that stores all its state in a database, it can be 50 | trivially created from the schema of the database. In other cases where there 51 | is not such a precise definition, the job of the API designer becomes a little 52 | more difficult. In such a case it would actually make sense to create an ER 53 | diagram for the application in question. That is a useful exercise in its own 54 | right, as it will make you better understand the application you're designing the API 55 | for. But more importantly, it will also help you in designing a good 56 | RESTful API. We will talk more about that in a minute. Going forward, I will 57 | assume that an ER diagram is available. 58 | 59 | In addition to understanding the application data model, and the operations on 60 | it, you of course need an entry point into the application that allows you to 61 | access and change the application state. This "way in" is fully application 62 | dependent and could take many forms. We will call this "way in" the 63 | *application interface*. Formally, this application interface could be 64 | considered an API as well. The difference, though, is that this interface is 65 | usually not intended for external consumption or even fully documented. In 66 | order not to introduce any confusion, we will not refer to this interface as 67 | an API: that term will be reserved for the RESTful API that is being designed. 68 | 69 | 70 | The API Code 71 | ============ 72 | 73 | The job of the API code is to access the application state, as well as the 74 | operations on that state, via the application interface, and expose it as a 75 | RESTful API. In between the application interface and the RESTful API, there 76 | is a transformation step that adapts the application data model and makes it 77 | comply with the RESTful architecture style. 78 | 79 | The result of this transformation would be RESTful *resources*, operations on 80 | those resources, and relationships between the resources. All of these are 81 | described by what we call the RESTful *resource model*. 82 | 83 | Resources are the foundation behind any RESTful API, and we will go into a lot 84 | of detail on them in :doc:`resources`. For now, just think of resources as 85 | being very similar to the entities from the ER diagram (which is why I 86 | encouraged you to create an ER diagram for your application in case it didn't 87 | exist). 88 | 89 | Relationships between resources are expressed as hyperlinks in the 90 | representation of the resource. This is one of the `fundamental principles 91 | `_ of 92 | RESTful API design. Resources also respond to a very limited set of operations 93 | (usually just 4), which is a second principle of the RESTful architectural 94 | style. 95 | 96 | When transforming objects from the application data model to RESTful 97 | resources, you may find it useful to define two utility functions: 98 | 99 | .. function:: to_resource() 100 | 101 | This function is assumed to take an object from the application data model, 102 | and convert it into a resource. 103 | 104 | .. function:: from_resource() 105 | 106 | This function is assumed to take a resource, and translate it into 107 | an object in the application data model. 108 | 109 | We don't discuss these methods further, other than mentioning that these can 110 | be rather simple functions if the application data model is similar to the 111 | resource model you're exposing, and can be quite complicated if they are very 112 | different. 113 | 114 | 115 | The Client 116 | ========== 117 | 118 | The client consumes the RESTful API via the standard HTTP protocol. In theory, 119 | the service could be provided on top of other protocols as well. 120 | Since HTTP is so ubiquitous, however, it is not certain how valuable such a mapping to 121 | another protocol would be in the real world. Therefore, this essay limits 122 | itself to describing our RESTful protocol in terms of HTTP. 123 | 124 | Clients would typically use an HTTP library to access the RESTful API. HTTP 125 | has become a moderately complex protocol, and very good libraries exist for 126 | many target platforms / languages. It therefore makes a lot of sense to use 127 | one of those libraries. 128 | 129 | In some cases, it may make sense to use a generic REST library on top of an 130 | HTTP library. However, since there are so many different conventions in 131 | RESTful APIs, this library may actually be specific to the API that you are 132 | consuming. 133 | -------------------------------------------------------------------------------- /urls.rst: -------------------------------------------------------------------------------- 1 | ==== 2 | URLs 3 | ==== 4 | 5 | Entry Point 6 | =========== 7 | 8 | A RESTful API needs to have one and exactly one entry point. The URL of the 9 | entry point needs to be communicated to API clients so that they can find the 10 | API. 11 | 12 | Technically speaking, the entry point can be seen as a singleton resource that 13 | exists outside any collection. It is common for the entry point to contain some 14 | or all of the following information: 15 | 16 | * Information on API version, supported features, etc. 17 | * A list of top-level collections. 18 | * A list of singleton resources. 19 | * Any other information that the API designer deemed useful, for example a 20 | small summary of operating status, statistics, etc. 21 | 22 | URL Structure 23 | ============= 24 | 25 | Each collection and resource in the API has its own URL. URLs should never be 26 | constructed by an API client. Instead, the client should only follow links that 27 | are generated by the API itself. 28 | 29 | The recommended convention for URLs is to use alternate collection / resource 30 | path segments, relative to the API entry point. This is best described by 31 | example. The table below uses the ":name" URL variable style from Rail's 32 | "Routes" implementation. 33 | 34 | ============================== ============================================= 35 | URL Description 36 | ============================== ============================================= 37 | /api The API entry point 38 | /api/:coll A top-level collection named "coll" 39 | /api/:coll/:id The resource "id" inside collection "coll" 40 | /api/:coll/:id/:subcoll Sub-collection "subcoll" under resource "id" 41 | /api/:coll/:id/:subcoll/:subid The resource "subid" inside "subcoll" 42 | ============================== ============================================= 43 | 44 | Even though sub-collections may be arbitrarily nested, in my experience, you 45 | want to keep the depth limited to 2, if possible. Longer URLs are more 46 | difficult to work with when using simple command-line tools like `curl 47 | `_. 48 | 49 | Relative vs Absolute 50 | ==================== 51 | 52 | It is strongly recommended that the URLs generated by the API should be 53 | absolute URLs. 54 | 55 | The reason is mostly for ease of use by the client, so that it never has to 56 | find out the correct base URI for a resource, which is needed to interpret a 57 | relative URL. The `URL RFC `_ 58 | specifies the algorithm for determining a base URL, which is rather complex. 59 | One of the options for finding the base URL is to use the URL of the resource 60 | that was requested. Since a resource may appear under multiple URLs (for 61 | example, as part of a collection, or stand-alone), it would be a significant 62 | overhead to a client to remember where he retrieved a representation from. By 63 | using absolute URLs, this problem doesn't present itself. 64 | 65 | URL Templates 66 | ============= 67 | 68 | A `draft standard `_ 69 | for URL templates exists. URL templates can be useful in link attributes 70 | where the target URL takes query arguments. It is recommended though to make 71 | only conservative use of these. So far the only good use case I have come 72 | across is when searching in a collection. In that case, it seems fair that the 73 | search criteria can be specified as GET style query arguments appended to the 74 | collection URL. 75 | 76 | I would recommend to only use the "literal expansion" part of the URL 77 | templates spec, because the "expression expansion" part adds too much 78 | complexity to a client in my view, for very little benefit. 79 | 80 | Variants 81 | ======== 82 | 83 | Sometimes you need to work on a variant of a resource. In our RHEV-M API for 84 | example, some attributes on a virtual machine can be updated while a virtual 85 | machine is running. This amounts to a hot plug/unplug of the resource, which 86 | is a different operation altogether from changing the saved representation. A 87 | nice way to implement this is using ";variant" identifier in a URL. For 88 | example: 89 | 90 | ====================== ============================================= 91 | URL Description 92 | ====================== ============================================= 93 | /api/:coll/:id;saved Identifies the saved variant of a resource. 94 | /api/:coll/:id;current Identifies the current variant of a resource. 95 | ====================== ============================================= 96 | 97 | The use of the semicolon to provide options that are specific to a path 98 | segment is explicitly alowed in `RFC3986 99 | `_ The advantage over using a 100 | "?variant" query argument is that this format is specific to a path segment 101 | only, and would allow you to e.g. work on the saved variant of a sub-resource 102 | in a sub-collection of the current variant of another resource. 103 | --------------------------------------------------------------------------------