a micro web framework for Java & Groovy
25 |This is the main page for your Ratpack app.
30 |
70 |
71 | Request in JSON format is a body of POST request to **api/invoke** endpoint.
72 | Response with external API output is a body of this endpoint result.
73 |
74 | Request attributes:
75 |
76 | * mode=SYNC
77 | * method=GET|POST|PUT|PATCH|DELETE
78 | * format=JSON|XML|URLENC
79 |
80 |
81 | ## Invoke external REST API asynchronously
82 |
83 |
84 |
85 | Request in JSON format is a body of POST request to **api/invoke** endpoint.
86 | Response is acknowledgment only.
87 | External API is called in separate thread and its output could be retrieved by **api/invoke/{id}/response** endpoint.
88 |
89 | Request attributes:
90 |
91 | * mode=ASYNC
92 | * method=GET|POST|PUT|PATH|DELETE
93 | * format=JSON|XML|URLENC
94 |
95 | ## Get Request that initialized invocation of external API
96 |
97 |
98 |
99 | Get request for invocation defined by {id} UUID. Enrich request with links to itself and to its response.
100 |
101 | ## Get ack or final response of external API invocation
102 |
103 |
104 |
105 | If external API has finished its final response will return. Otherwise acknowledgement response will come.
106 | Enrich response with links to itself and to its request.
107 |
108 | # API specification
109 |
110 | API specification is inline with [Swagger 2.0 JSON API specification](https://github.com/swagger-api/swagger-spec).
111 |
112 | ## HTTP headers
113 |
114 | **GET**
115 |
116 | Accept: application/json
117 |
118 | **POST|PUT|PATCH**
119 |
120 | Content-Type: application/json
121 | Accept: application/json
122 |
123 | ## Endpoints
124 |
125 | ### /api-docs
126 |
127 | Get list of available APIs in Swagger 2.0 format.
128 |
129 | **Method:** GET
130 | **Accept:** application/json
131 | **HTTP return codes:**
132 |
133 | * 200 - OK
134 |
135 | ### /api
136 |
137 | Redirects to ```/api-docs```.
138 |
139 | **Method:** GET
140 | **Accept:** application/json
141 | **HTTP return codes:**
142 |
143 | * 200 - OK
144 |
145 | ### /api/invoke
146 |
147 | Invoke external API either synchronously or asynchronously.
148 | Use diverse HTTP methods and formats for API invocations.
149 |
150 | **Method:** POST
151 | **Content-Type:** application/json
152 | **Accept:** application/json
153 | **HTTP return codes:**
154 |
155 | * 200 - OK
156 |
157 | #### Input message format
158 |
159 | {
160 | "request": {
161 | "id": "Universal Unique Identifier (UUID)",
162 | "method": "GET|POST|PUT|PATCH|DELETE",
163 | "mode": "SYNC|ASYNC|EVENT",
164 | "format": "JSON|XML|URLENC",
165 | "url": "URI OF EXTERNAL ENDPOINT",
166 | "headers": JSON,
167 | "data": JSON
168 | }
169 | }
170 |
171 | where **request** attributes are:
172 |
173 | **method:**
174 |
175 | * HTTP method to be used for external API call
176 |
177 | **mode:**
178 |
179 | * mode=SYNC - call API synchronously, send request and wait for response
180 | * mode=ASYNC - call API asynchronously, send request and do not wait for response. Response might be avilable for caller as:
181 | * callback invocation
182 | * pull request
183 | * mode=EVENT - call API asynchronously without response, send request as notification
184 |
185 | **format:**
186 |
187 | * format=JSON sets header **Content-Type: application/json**
188 | * format=XML sets header **Content-Type: application/xml**
189 | * format=URLENC sets header **Content-Type: application/x-www-form-urlencoded**
190 |
191 | **Important:** URLENC format makes sense only for method=POST.
192 |
193 | **url:**
194 |
195 | * url of target API
196 | * If method==GET query parameters (after "?") are merged with simple attributes from **data** structure.
197 |
198 | **headers:**
199 |
200 | * list of HTTP request headers in the form of key-value pairs
201 |
202 | "headers": {
203 | "Authorization": "Bearer ACCESS_TOKEN"
204 | }
205 |
206 | **data:**
207 |
208 | * JSON, either with list of query parameters or request body content.
209 |
210 | #### Output message format
211 |
212 | {
213 | "response": {
214 | "success": "true|false",
215 | "errorCode": "0 if no error, else otherwise",
216 | "errorDescr": "Error description",
217 | "data": JSON WITH EXTERNAL API OUTPUT,
218 | "statusCode": "HTTP status code from external API invoke",
219 | "id": "Universal Unique Identifier (UUID)",
220 | "href": "http://localhost:5050/api/invoke/{id}/response",
221 | "links": {
222 | "request": {
223 | "href": "http://localhost:5050/api/invoke/{id}/request"
224 | }
225 | }
226 | }
227 | }
228 |
229 | ### /api/invoke/{id}/request
230 |
231 | Get request that started invocation given by {id}.
232 |
233 | **Method:** GET
234 | **Accept:** application/json
235 | **HTTP return codes:**
236 |
237 | * 200 - OK
238 |
239 | #### Output message format
240 |
241 | {
242 | "request": {
243 | "id": "Universal Unique Identifier (UUID)",
244 | "method": "GET|POST|PUT|PATCH|DELETE",
245 | "mode": "SYNC|ASYNC|EVENT",
246 | "format": "JSON|XML|URLENC",
247 | "url": "URI OF EXTERNAL ENDPOINT",
248 | "headers": JSON,
249 | "data": JSON,
250 | "href": "http://localhost:5050/api/invoke/{id}/request",
251 | "links": {
252 | "response": {
253 | "href": "http://localhost:5050/api/invoke/{id}/response"
254 | }
255 | }
256 | }
257 | }
258 |
259 | ### /api/invoke/{id}/response
260 |
261 | Get response from external API invocation given by {id}.
262 | If *mode*=SYNC, response with external API output data is returned inside */api/call* response.
263 | If *mode*=ASYNC, response could be acknowledgment message (when async call has not been finished) or
264 | response from external API call (if async processing has finished).
265 |
266 | **Method:** GET
267 | **Accept:** application/json
268 | **HTTP return codes:**
269 |
270 | * 200 - OK
271 |
272 | #### Output message format
273 |
274 | Response when only acknowlegment is available.
275 |
276 | {
277 | "response": {
278 | "success": "true|false",
279 | "errorCode": "0 if no error, else otherwise",
280 | "errorDescr": "Error description",
281 | "statusCode": "HTTP status code from external API invoke",
282 | "id": "Universal Unique Identifier (UUID)",
283 | "href": "http://localhost:5050/api/invoke/{id}/response",
284 | "links": {
285 | "request": {
286 | "href": "http://localhost:5050/api/invoke/{id}/request"
287 | }
288 | }
289 | }
290 | }
291 |
292 | Response when external API has finished and output is available.
293 | It contains *data* attribute with JSON representation of external API output.
294 |
295 | {
296 | "response": {
297 | "success": "true|false",
298 | "errorCode": "0 if no error, else otherwise",
299 | "errorDescr": "Error description",
300 | "data": JSON WITH EXTERNAL API OUTPUT,
301 | "statusCode": "HTTP status code from external API invoke",
302 | "id": "Universal Unique Identifier (UUID)",
303 | "href": "http://localhost:5050/api/invoke/{id}/response",
304 | "links": {
305 | "request": {
306 | "href": "http://localhost:5050/api/invoke/{id}/request"
307 | }
308 | }
309 | }
310 | }
311 |
312 | ### api/health-checks
313 |
314 | Run all health checks and return their values.
315 |
316 | **Method:** GET
317 |
318 | ### api/health-check/:name
319 |
320 | Run health check defined by the given :name.
321 |
322 | **Method:** GET
323 |
324 | Defined health checks:
325 |
326 | * *apigateway*
327 |
328 | # Run Tests
329 |
330 | ## Prerequisites
331 |
332 | ### Mountebank - for stubbing and mocking
333 |
334 | Unit tests use stubs provided by [mountebank](http://www.mbtest.org) - really nice and practical framework.
335 |
336 | In order to make them working install node.js together with npm package manager. I recommend to use [Node version manager](https://github.com/creationix/nvm).
337 |
338 | $ curl https://raw.githubusercontent.com/creationix/nvm/v0.17.0/install.sh | bash
339 | $ nvm install v0.11.8
340 | $ curl https://www.npmjs.org/install.sh | sh
341 |
342 | Install mountebank globally:
343 |
344 | $ npm install -g mountebank --production
345 |
346 | After that mountebank server should be available with command:
347 |
348 | $ mb
349 |
350 | #### Required FIXes
351 |
352 | 1. [node.js](http://nodejs.org) version greater than v.0.11.8
353 |
354 | v0.11.8 has a bug: request.method is null for HTTP DELETE calls [issue](https://github.com/joyent/node/issues/6461).
355 |
356 | 2. Fix Internal Server Error (500) if predicate attributes have null value.
357 |
358 | [Pull request #53](https://github.com/bbyars/mountebank/pull/53) that solves this issue.
359 |
360 | 3. For load testing fix logging mechanism
361 |
362 | [Required commit](https://github.com/bbyars/mountebank/commit/2f1915702ab9674d08ec8d46a7e6886c8c8b426f) or latest (non production version).
363 |
364 |
365 | ### Redis - for requests persistance and statistics
366 |
367 | API Gateway uses [Redis](http://redis.io) key/value store for persisting requests and their responses and collecting statistics.
368 | There are tests that require Redis to be installed or accessible.
369 |
370 | Install redis in your system:
371 |
372 | $ curl -O http://download.redis.io/releases/redis-2.8.17.tar.gz
373 | $ tar -xvzf redis-2.8.17.tar.gz
374 | $ cd redis-2.8.17.tar.gz
375 | $ make
376 | $ make install
377 |
378 | Then in your system the following commands should be visible: redis-server (start redis server), redis-cli (start redis command line shell).
379 |
380 | ### Running dependencies
381 |
382 | When **./gradlew test** starts it automatically sets up dependencies: *mountebank* and *redis*. There are two gradle tasks:
383 |
384 | * **runEnv** - starts servers required for testing:
385 | * **MounteBank** - external API stubs
386 | * **Redis** - key/value store
387 | * **cleanEnv** - stops servers used for testing
388 |
389 | The assumption is that these servers are available on specific ports. If you change them please look at **stopEnv** task
390 | in *build.gradle* file. There is table of ports in there.
391 |
392 | ## Tests
393 |
394 | Please look at each groovy class from test folder.
395 | Some of them, especially for functional testing with some real services, are annotated with @Ignore annotation (feature of [Spock](https://code.google.com/p/spock/) BDD testing framework).
396 | Remove or comment it out in order to run them while testing.
397 |
398 | Run tests with command:
399 |
400 | $ ./gradlew test
401 |
402 | Above command automatically sets up dependencies: *MounteBank* and *Redis* for testing.
403 | Inside *build.gradle* file there are two helpfull gradle tasks:
404 |
405 | * **runEnv** - starts servers required for testing:
406 | * **MounteBank** - external API stubs
407 | * **Redis** - key/value store
408 | * **cleanEnv** - stops servers used for testing
409 |
410 | The assumption is that these servers are available on specific ports.
411 | If you change them please look at **stopEnv** task definition. There is a table of ports in there.
412 |
413 | task stopEnv(type: FreePorts) {
414 | // port: 2525 - mb (MounteBank),
415 | // port: 6379 - redis-server (Redis data store)
416 | ports = [2525, 6379]
417 | }
418 |
419 | The following tasks from *build.gradle* do the job:
420 |
421 | startMounteBank - start mountebank server with *mb* shell command
422 | initMounteBank - initialize stubs configuration with *./src/test/resources/imposter.json* file.
423 | testFinished - kill spwaned processes attached to mountebank ports
424 |
425 | # Key/value data storage
426 |
427 | API Gateway keeps highly dynamic data in key/value store - [Redis](http://redis.io/).
428 | It is used for:
429 |
430 | * statistics
431 | * requests and their corresponding responses
432 | * if mode=ASYNC, response is stored for future retrival
433 | * logging of top requests and responses
434 |
435 | ## Statistics
436 |
437 | ### Usage
438 |
439 | Collecting number of requests:
440 |
441 | * usage/year:{yyyy}
442 | * usage/year:{yyyy}/month:{mm}
443 | * usage/year:{yyyy}/month:{mm}/day:{dd}
444 |
445 | To get statistic value:
446 |
447 | $ redis-cli> get usage/year:2014
448 |
449 | ### Requests store
450 |
451 | Every request is stored as Redis hash and has structure:
452 |
453 | * key: **request:UUID** - where UUID is unique ID of request
454 | * field: **request**, value: **request serialized to JSON**
455 | * field: **response**, value: **response serialized to JSON**
456 | * field: **aresponse**, value: **async response serialized to JSON**
457 |
458 | If mode=ASYNC, **response** field stores first answer that is result of request registration and request sending.
459 | Then **aresponse** field stores final response from service call.
460 |
461 | If mode=SYNC, **response** field stores final response from service call.
462 |
463 | To get request and response for particular UUID
464 |
465 | $ redis-cli> hget request:UUID request
466 | $ redis-cli> hget request:UUID response
467 | $ redis-cli> hget request:UUID aresponse
468 |
469 | ### Requests log
470 |
471 | Every request's id is stored in sorted set (by timestamp).
472 |
473 | * key: **request-log**
474 | * score: **timestamp** - datetime converted to timestamp
475 | * member: **request:UUID** - where UUID is unique ID of request
476 |
477 | To get log of last 20 requests:
478 |
479 | $ redis-cli> zrange request-log 0 20
480 |
481 | # Example API calls
482 |
483 | ## Get API endpoints
484 |
485 | $ curl -X GET -H "Accept: application/vnd.api+json" http://localhost:5050/api
486 | or
487 | $ curl -X GET -H "Accept: application/json" http://localhost:5050/api
488 |
489 | ## Example: [Salesforce.com Account API](docs/Salesforce_API.md)
490 |
491 |
492 |
493 | ## Example: HipChat - get history of chats
494 |
495 | Example HipChat API call:
496 |
497 | $ curl -X POST -H "Content-Type: application/json" -d '{"method": "GET", "mode": "SYNC", "format": "JSON", "url": "https://api.hipchat.com/v2/room/online4m.com/history/latest?auth_token=YOUR_TOKEN", "data": {"max-results": {"l": 10}}}' -i http://localhost:5050/api/call
498 |
499 | ## Example: Twitter query with OAUTH authorization
500 |
501 | Before any API call you have to register your application in twitter. By doing this you get unique client id and client secret.
502 | These attributes are needed to ask for access token. Access token is used in all subsequent api calls.
503 |
504 | Point your browser to [apps.twitter.com](https://apps.twitter.com), click the button *Create New App* and register your application.
505 |
506 | Next, request for a access token.
507 |
508 | $ curl -X POST -H "Content-Type: application/json" -d '{"method": "POST", "mode": "SYNC", "format": "URLENC", "url": "https://api.twitter.com/oauth2/token", "data": {"grant_type": "client_credentials", "client_id", "YOUR_APP_ID", "client_secret", "YOUR_APP_SECRET"}}' -i http://localhost:5050/api/call
509 |
510 | As result you should get:
511 |
512 | {
513 | "errorCode":"0",
514 | "data": {
515 | "access_token":"ACCESS_TOKEN_URLENCODED",
516 | "token_type":"bearer"
517 | },
518 | "success":true
519 | }
520 |
521 | Now you are ready to call, for example, twitter's search API. But now to request add headers map:
522 |
523 | "headers": {
524 | "Authorization": "Bearer ACCESS_TOKEN_URLENCODED"
525 | }
526 |
527 | and invocation:
528 |
529 | $ curl -X POST -H "Content-Type: application/json" -d '{"method": "GET", "mode": "SYNC", "format": "JSON", "url": "https://api.twitter.com/1.1/search/tweets.json", "headers": {"Authorization": " Bearer ACCESS_TOKEN_URLENCODED"}, "data": {"q": "ratpackweb"}' -i http://localhost:5050/api/call
530 |
531 | # Commands to be used while developing
532 |
533 | Test synchronous external service invocation:
534 |
535 | curl -X POST -H "Content-Type: application/json" -d@./src/test/resources/testdata.json http://localhost:5050/api/call
536 |
537 | Test asynchronous external service invocation
538 |
539 | curl -X POST -H "Content-Type: application/json" -d@./src/test/resources/testdataasync.json http://localhost:5050/api/call
540 |
541 | Load test. Change **-c** from 1 to more clients. Change **-r** from 1 to more repetition.
542 |
543 | siege -c 1 -r 1 -H 'Content-Type: application/json' 'http://localhost:5050/api/call POST < ./src/test/resources/testdata.json'
544 |
545 | # TODO:
546 |
547 | * Add ASYNC calls with response callbacks and storing responses in local storage
548 | * Add EVENT async calls without waiting for response
549 |
550 | # Project structure
551 |
552 | In this project you get:
553 |
554 | * A Gradle build file with pre-built Gradle wrapper
555 | * A tiny home page at src/ratpack/templates/index.html (it's a template)
556 | * A routing file at src/ratpack/ratpack.groovy
557 | * Reloading enabled in build.gradle
558 | * A standard project structure:
559 |
560 |