├── .gitignore ├── README.md ├── assets ├── favicon.ico └── sample.png ├── bin ├── Configuration.js ├── ProjectUtilities.js ├── QuickLogger.js ├── RateMeter.js ├── UrlFlexParser.js ├── proxy.js └── test.js ├── conf ├── config.json ├── config.xml ├── en.json └── es.json ├── notes.md └── package.json /.gitignore: -------------------------------------------------------------------------------- 1 | # Logs 2 | logs 3 | *.log 4 | npm-debug.log* 5 | 6 | # Runtime data 7 | pids 8 | *.pid 9 | *.seed 10 | 11 | # Directory for instrumented libs generated by jscoverage/JSCover 12 | lib-cov 13 | 14 | # Coverage directory used by tools like istanbul 15 | coverage 16 | 17 | # nyc test coverage 18 | .nyc_output 19 | 20 | # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) 21 | .grunt 22 | 23 | # node-waf configuration 24 | .lock-wscript 25 | 26 | # Compiled binary addons (http://nodejs.org/api/addons.html) 27 | build/Release 28 | 29 | # Dependency directories 30 | node_modules 31 | jspm_packages 32 | 33 | # Optional npm cache directory 34 | .npm 35 | 36 | # Optional REPL history 37 | .node_repl_history 38 | 39 | *~ 40 | .DS_Store 41 | ._* 42 | .idea/ 43 | /out/ 44 | .idea_modules/ 45 | 46 | node_modules/ 47 | *.sqlite 48 | 49 | # Test configurations 50 | config-test.xml 51 | config-test.json 52 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # node.js ArcGIS Proxy Server 2 | 3 | This is an implementation of an ArcGIS proxy server using node.js. While this server can proxy most http requests, it is specifically designed to act on behalf of ArcGIS type services following the [ArcGIS Resource Proxy](https://github.com/Esri/resource-proxy/) specification. The proxy handles support for: 4 | 5 | * Accessing cross domain resources. 6 | * URL transformations. 7 | * Requests that exceed 2048 characters. 8 | * Accessing resources secured with ArcGIS token based authentication. 9 | * Accessing resources secured with [OAuth 2.0 app login](https://developers.arcgis.com/en/authentication). 10 | * Transaction logging. 11 | * Resource based rate limiting to help manage credit consumption. 12 | 13 | ## Instructions 14 | 15 | * Install or update [node.js](https://nodejs.org/en/), version 6 LTS is recommended. 16 | * Download and unzip the .zip file or clone this repository. You can download [a released version](https://github.com/Esri/resource-proxy/releases) (recommended) or the [most recent daily build](https://github.com/Esri/resource-proxy/archive/master.zip). 17 | * install the node.js dependencies: 18 | 19 | ``` 20 | npm install 21 | ``` 22 | 23 | * Edit the proxy configuration file: 24 | * Choose either one of (conf/config.json or conf/config.xml, but not both) configuration format. 25 | * Use a text editor to set up your proxy configuration settings. 26 | * Decide which port your server should listen on (default is 3692). 27 | * Determine which URLs you are going to proxy by editing the `serverUrls` section. 28 | * Set your `allowedReferrers`. While "*" may be ok for testing we highly recommend whitelisting referrers in production. 29 | * Review the full documentation for [proxy configuration settings](https://github.com/Esri/resource-proxy/#proxy-configuration-settings). 30 | * Start the node server from a command line: 31 | 32 | ``` 33 | npm start 34 | ``` 35 | 36 | * Test that the proxy is installed and available by running a browser on the local machine then navigate to the port and url: 37 | 38 | ``` 39 | http://localhost:{port}/ping 40 | ``` 41 | 42 | * Test that the proxy is able to forward requests directly in the browser using one of your `serverUrls` definitions, such as: 43 | 44 | ``` 45 | http://localhost:{port}/proxy/http/services.arcgisonline.com/ArcGIS/rest/services/?f=pjson 46 | ``` 47 | 48 | * Check the current status of your proxy server: 49 | 50 | ``` 51 | http://localhost:{port}/status 52 | ``` 53 | 54 | Once you deploy to an infrastructure on the public internet replace `localhost` with the host name you install the proxy server on. 55 | 56 | ## Requirements 57 | 58 | * [node.js](https://nodejs.org/en/) version 6.0 or higher (recommended.) 59 | * sudo access rights so you can install files, open a TCP/IP port. 60 | * File access read/write access for the log file and the sqlite database. 61 | * Server administration and networking background to securely run your server. 62 | 63 | ## Folders and Files 64 | 65 | The proxy consists of the following files: 66 | * `package.json`: the node configuration. 67 | * `conf/config.json`: This file contains the [configuration settings for the proxy](https://github.com/Esri/resource-proxy/#proxy-configuration-settings). This is where you define all the resources that are allowed to use the proxy. 68 | * `conf/config.xml`: This file contains the [configuration settings for the proxy](https://github.com/Esri/resource-proxy/#proxy-configuration-settings). This is where you define all the resources that are allowed to use the proxy. 69 | * `node_modules/`: after you run `npm install` this folder holds all the node dependencies. 70 | * `bin/`: folder containing the proxy runtime scripts. 71 | 72 | ## Running your proxy server 73 | 74 | Follow these instructions to setup and operate your proxy server. 75 | 76 | ### Logging 77 | 78 | The proxy server can log to the console, a log file, or both. There are 4 levels of logging: 79 | 80 | 1. Errors: only log error conditions. Errors typically indicate normal functionality that has failed and should be addressed. 81 | 2. Warnings: show warnings and error conditions in the log. Warnings are soft errors that may require intervention. 82 | 3. Information: show informational messages, warnings, and errors. Information messages are useful for debugging your server setup. 83 | 4. None: show no log messages. 84 | 85 | You can configure the log file name and path to where it should go on your server. 86 | 87 | ### Example Configurations 88 | 89 | The node proxy supports JSON and XML configuration. Sample configurations are located in the `/conf` folder. 90 | 91 | If you change the configuration file you must restart the server. 92 | 93 | ### Requests 94 | 95 | To properly use this proxy server in a secure manner you must white-list all requests. See the documentation regarding `allowedReferrers`. The 96 | samples use `*` because we don't know where you are calling the proxy server from, but this is very dangerous. Do not deploy to production without 97 | white-listing allowed referrers or you could end up proxying nefarious requests. 98 | 99 | ### Rate Limiting 100 | 101 | You can set up rate limits on your proxied resources. This is the rate a resource may be accessed within the given time period for all referrers (requests). 102 | For example, a `rateLimit` of 120 requests within a `rateLimitPeriod` of 60 minutes specifies no more than 120 requests over the course of 1 hour, or 2 requests per minute. 103 | This is a sliding time window from the first request. Once the limit is reached access will not be granted until the next time interval. 104 | 105 | ## Issues 106 | 107 | Found a bug or want to request a new feature? Check out previously logged [Issues](https://github.com/Esri/resource-proxy/issues) and/or our [FAQ](https://github.com/Esri/resource-proxy/blob/master/FAQ.md). If you don't see what you're looking for, feel free to submit a [new issue](https://github.com/Esri/resource-proxy/issues/new). 108 | 109 | ## Contributing 110 | 111 | Esri welcomes contributions from anyone and everyone. Please see our [guidelines for contributing](https://github.com/esri/contributing). 112 | 113 | ## License 114 | 115 | Copyright 2017 Esri 116 | 117 | Licensed under the Apache License, Version 2.0 (the "License"); 118 | You may not use this file except in compliance with the License. 119 | You may obtain a copy of the License at 120 | http://www.apache.org/licenses/LICENSE-2.0 121 | 122 | Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for specific language governing permissions and limitations under the license. 123 | -------------------------------------------------------------------------------- /assets/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jf990/resource-proxy-node/b72e87baa46b70d95c54610f8294ba51a64690b5/assets/favicon.ico -------------------------------------------------------------------------------- /assets/sample.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jf990/resource-proxy-node/b72e87baa46b70d95c54610f8294ba51a64690b5/assets/sample.png -------------------------------------------------------------------------------- /bin/Configuration.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Configuration file parser, validator, and accessor. Calling loadConfigurationFile returns a promise that will 3 | * resolve once the config is loaded, parsed, and validated. 4 | * 5 | * See README for the configuration file format. 6 | */ 7 | 8 | const fs = require('fs'); 9 | const joinPath = require('path.join'); 10 | const path = require('path'); 11 | const loadJsonFile = require('load-json-file'); 12 | const ProjectUtilities = require('./ProjectUtilities'); 13 | const QuickLogger = require('./QuickLogger'); 14 | const UrlFlexParser = require('./UrlFlexParser'); 15 | const xml2js = require('xml2js'); 16 | 17 | const defaultRequireRootPath = '../'; // when run from npm command line 18 | const defaultConfigurationRootPath = '../'; // when run from npm command line 19 | const defaultConfigurationFilePath = 'conf'; 20 | const defaultConfigurationFileName = 'config'; 21 | const defaultConfigurationTestFileName = 'config-test'; 22 | const defaultConfigurationFileType = 'xml'; 23 | const defaultOAuthEndpoint = 'https://www.arcgis.com/sharing/oauth2/'; 24 | 25 | var configuration = { 26 | testMode: false, 27 | language: 'en', 28 | mustMatch: true, 29 | logLevel: QuickLogger.LOGLEVEL.ERROR.value, 30 | logConsole: true, 31 | logFunction: null, 32 | localPingURL: '/ping', 33 | localEchoURL: '/echo', 34 | localStatusURL: '/status', 35 | staticFilePath: null, 36 | port: 3333, // 80 37 | useHTTPS: false, 38 | httpsKeyFile: null, 39 | httpsCertificateFile: null, 40 | httpsPfxFile: null, 41 | listenURI: null, 42 | allowedReferrers: ['*'], 43 | allowAnyReferrer: false, 44 | serverURLs: [], 45 | stringTable: null 46 | }; 47 | var configurationComplete = false; 48 | 49 | /** 50 | * Look at the command line for any configuration overrides. 51 | */ 52 | function parseCommandLineOptions() { 53 | process.argv.forEach(function(value, index) { 54 | console.log('argv[' + index + ']: ' + value); 55 | if (value == 'test') { 56 | configuration.testMode = true; 57 | console.log('Setting TEST mode'); 58 | } 59 | }); 60 | } 61 | 62 | /** 63 | * Return true if the server URL definition for this resource is to support user login (user name+password). We use this to 64 | * get the secure token. 65 | * @param serverURLInfo {object} the server URL definition to check. 66 | * @returns {boolean} 67 | */ 68 | function isUserLogin (serverURLInfo) { 69 | if (serverURLInfo != null) { 70 | return serverURLInfo.username !== undefined && serverURLInfo.username.trim().length > 0 && serverURLInfo.password !== undefined && serverURLInfo.password.trim().length > 0; 71 | } else { 72 | return false; 73 | } 74 | } 75 | 76 | /** 77 | * Return true if the server URL definition for this resource is to support app login (clientId). We use this to 78 | * get the secure token with OAuth. 79 | * @param serverURLInfo {object} the server URL definition to check. 80 | * @returns {boolean} 81 | */ 82 | function isAppLogin (serverURLInfo) { 83 | if (serverURLInfo != null) { 84 | return serverURLInfo.clientid !== undefined && serverURLInfo.clientid.trim().length > 0 && serverURLInfo.clientsecret !== undefined && serverURLInfo.clientsecret.trim().length > 0; 85 | } else { 86 | return false; 87 | } 88 | } 89 | 90 | /** 91 | * Allow read-only access to testMode setting. 92 | * @returns {boolean} 93 | */ 94 | function isTestMode() { 95 | return configuration.testMode; 96 | } 97 | 98 | /** 99 | * Return an entry in the language translation strings table, and replace any tokens in that string from a key/value object. 100 | * Each target key in the string is identified with {key} and matched to an entry in the tokens object then replaced with its value. 101 | * 102 | * @param key {string} The key to look up in the string table. 103 | * @param tokens {object} key/value for token replacement in the string. 104 | * @returns {string} If the key is not found in the string table then the key is returned. 105 | */ 106 | function getStringTableEntry(key, tokens) { 107 | if (configuration.stringTable == null) { 108 | return key; 109 | } else if (configuration.stringTable[key] === undefined) { 110 | return key; 111 | } else if (tokens === undefined || tokens == null) { 112 | return configuration.stringTable[key]; 113 | } else { 114 | return ProjectUtilities.tokenReplace(configuration.stringTable[key], tokens); 115 | } 116 | } 117 | 118 | /** 119 | * Determine if the configuration is valid enough to start the server. If it is not valid any reasons are 120 | * written to the log file and the server is not started. 121 | * @returns {boolean} true if valid enough. 122 | */ 123 | function isConfigurationValid () { 124 | var isValid, 125 | serverUrl, 126 | i; 127 | 128 | // allowedReferrers != empty 129 | // port >= 80 <= 65535 130 | // either httpsKeyFile && httpsCertificateFile or httpsPfxFile 131 | // at least one serverUrls 132 | isValid = QuickLogger.setConfiguration(configuration); 133 | if (configuration.listenURI == null) { 134 | QuickLogger.logErrorEvent(getStringTableEntry('No URI to listen for', null)); 135 | isValid = false; 136 | } else if (configuration.listenURI.length == 0) { 137 | QuickLogger.logErrorEvent(getStringTableEntry('No URI to listen for', null)); 138 | isValid = false; 139 | } 140 | if (configuration.serverUrls == null) { 141 | QuickLogger.logErrorEvent(getStringTableEntry('You must configure serverUrls', null)); 142 | isValid = false; 143 | } else if (configuration.serverUrls.length == 0) { 144 | QuickLogger.logErrorEvent(getStringTableEntry('You must configure one serverUrl', null)); 145 | isValid = false; 146 | } else { 147 | for (i = 0; i < configuration.serverUrls.length; i ++) { 148 | serverUrl = configuration.serverUrls[i]; 149 | if (serverUrl.errorMessage != '') { 150 | isValid = false; 151 | QuickLogger.logErrorEvent(getStringTableEntry('Error in server URL definition', {url: serverUrl.url, error: serverUrl.errorMessage})); 152 | } 153 | } 154 | } 155 | // TODO: We do not validate the individual server URLs but maybe we should? 156 | if (configuration.allowedReferrers == null) { 157 | configuration.allowedReferrers = ['*']; 158 | QuickLogger.logWarnEvent(getStringTableEntry('You should configure at least one referrer', null)); 159 | } else if (configuration.allowedReferrers.length == 0) { 160 | configuration.allowedReferrers = ['*']; 161 | QuickLogger.logWarnEvent(getStringTableEntry('You should configure at least one referrer', null)); 162 | } 163 | return isValid; 164 | } 165 | 166 | /** 167 | * After we load and parse the configuration file we go through every attribute and attempt to 168 | * validate the data, normalize the data, and pre-cache certain values to reduce stress at runtime. 169 | * This function does not return anything, it updates the configuration data structure in-place. 170 | * Use isConfigurationValid() after this function to validate the configuration is good enough to start with. 171 | * @param json {object} - the object we are parsing and validating. 172 | * @param schema {string} - indicates which configuration schema we loaded, either 'json' or 'xml' 173 | */ 174 | function postParseConfigurationFile(json, schema) { 175 | var proxyConfigSection, 176 | serverUrlsSection, 177 | allowedReferrersSection, 178 | allowedReferrers, 179 | referrerToCheckParts, 180 | referrerValidated, 181 | serverUrls, 182 | serverUrl, 183 | urlParts, 184 | logLevel, 185 | i, 186 | languageFile, 187 | invalidSetting; 188 | 189 | if (json !== null) { 190 | if (schema == 'json') { 191 | proxyConfigSection = json.ProxyConfig; 192 | if (proxyConfigSection === undefined) { 193 | proxyConfigSection = json.proxyConfig; 194 | } else { 195 | proxyConfigSection = null; 196 | } 197 | } else if (schema === 'xml') { 198 | proxyConfigSection = json.ProxyConfig['$']; 199 | } else { 200 | proxyConfigSection = null; 201 | } 202 | if (proxyConfigSection !== undefined && proxyConfigSection !== undefined) { 203 | if (proxyConfigSection.language !== undefined && proxyConfigSection.language != 'en') { 204 | languageFile = defaultRequireRootPath + defaultConfigurationFilePath + '/' + proxyConfigSection.language + '.json'; 205 | if (fs.existsSync(languageFile)) { 206 | configuration.language = proxyConfigSection.language; 207 | configuration.stringTable = require(languageFile); 208 | } 209 | } 210 | if (proxyConfigSection.useHTTPS !== undefined) { 211 | if (typeof proxyConfigSection.useHTTPS === 'string') { 212 | configuration.useHTTPS = proxyConfigSection.useHTTPS.toLocaleLowerCase().trim() === 'true' || proxyConfigSection.useHTTPS === '1'; 213 | } else { 214 | configuration.useHTTPS = proxyConfigSection.useHTTPS; 215 | } 216 | } 217 | if (proxyConfigSection.port !== undefined) { 218 | if (typeof proxyConfigSection.port === 'string') { 219 | configuration.port = parseInt(proxyConfigSection.port, 10); 220 | } else { 221 | configuration.port = proxyConfigSection.port; 222 | } 223 | } 224 | if (proxyConfigSection.mustMatch !== undefined) { 225 | if (typeof proxyConfigSection.mustMatch === 'string') { 226 | configuration.mustMatch = proxyConfigSection.mustMatch.toLocaleLowerCase().trim() === 'true' || proxyConfigSection.mustMatch === '1'; 227 | } else { 228 | configuration.mustMatch = proxyConfigSection.mustMatch; 229 | } 230 | } else { 231 | configuration.mustMatch = true; 232 | } 233 | if (proxyConfigSection.matchAllReferrer !== undefined) { 234 | if (typeof proxyConfigSection.matchAllReferrer === 'string') { 235 | configuration.matchAllReferrer = proxyConfigSection.matchAllReferrer.toLocaleLowerCase().trim() === 'true' || proxyConfigSection.matchAllReferrer === '1'; 236 | } else { 237 | configuration.matchAllReferrer = proxyConfigSection.matchAllReferrer; 238 | } 239 | } else { 240 | configuration.matchAllReferrer = true; 241 | } 242 | if (proxyConfigSection.logToConsole !== undefined) { 243 | if (typeof proxyConfigSection.logToConsole === 'string') { 244 | configuration.logToConsole = proxyConfigSection.logToConsole.toLocaleLowerCase().trim() === 'true' || proxyConfigSection.logToConsole === '1'; 245 | } else { 246 | configuration.logToConsole = proxyConfigSection.logToConsole == true; 247 | } 248 | } else { 249 | configuration.logToConsole = false; 250 | } 251 | if (proxyConfigSection.logFile !== undefined) { 252 | configuration.logFileName = proxyConfigSection.logFile; 253 | } else if (proxyConfigSection.logFileName !== undefined) { 254 | configuration.logFileName = proxyConfigSection.logFileName; 255 | } 256 | if (proxyConfigSection.logFilePath !== undefined) { 257 | configuration.logFilePath = proxyConfigSection.logFilePath; 258 | } 259 | if (proxyConfigSection.logLevel !== undefined) { 260 | invalidSetting = true; 261 | for (logLevel in QuickLogger.LOGLEVEL) { 262 | if (QuickLogger.LOGLEVEL.hasOwnProperty(logLevel)) { 263 | if (QuickLogger.LOGLEVEL[logLevel].label == proxyConfigSection.logLevel.toUpperCase()) { 264 | configuration.logLevel = QuickLogger.LOGLEVEL[logLevel].value; 265 | invalidSetting = false; 266 | break; 267 | } 268 | } 269 | } 270 | if (invalidSetting) { 271 | console.log(getStringTableEntry('Undefined logging level', {level: proxyConfigSection.logLevel})); 272 | } 273 | } else { 274 | console.log(getStringTableEntry('No logging level requested', null)); 275 | } 276 | // allowedReferrers can be a single string, items separated with comma, or an array of strings. 277 | // Make sure we end up with an array of strings. 278 | if (proxyConfigSection.allowedReferers !== undefined) { 279 | allowedReferrersSection = proxyConfigSection.allowedReferers; 280 | } else if (proxyConfigSection.allowedReferrers !== undefined) { 281 | allowedReferrersSection = proxyConfigSection.allowedReferrers; 282 | } else { 283 | allowedReferrersSection = null; 284 | } 285 | if (allowedReferrersSection !== null) { 286 | if (Array.isArray(allowedReferrersSection)) { 287 | // create a new array from the existing array 288 | allowedReferrers = allowedReferrersSection.slice(); 289 | } else if (allowedReferrersSection.indexOf(',') >= 0) { 290 | // create an array of the comma separated referrer list 291 | allowedReferrers = allowedReferrersSection.split(','); 292 | } else { 293 | // create a new array from a single string 294 | allowedReferrers = [allowedReferrersSection]; 295 | } 296 | // make a cache of the allowed referrers so checking at runtime is easier and avoids parsing the referrer on each lookup 297 | configuration.allowedReferrers = []; 298 | for (i = 0; i < allowedReferrers.length; i ++) { 299 | referrerValidated = { 300 | protocol: '*', 301 | hostname: '*', 302 | path: '*', 303 | referrer: '*' 304 | }; 305 | if (allowedReferrers[i] == "*") { 306 | // TODO: this may not be necessary because when we match a * we don't check the individual parts 307 | configuration.allowAnyReferrer = true; 308 | configuration.allowedReferrers.push(referrerValidated); 309 | } else { 310 | referrerToCheckParts = UrlFlexParser.parseAndFixURLParts(allowedReferrers[i].toLowerCase().trim()); 311 | if (referrerToCheckParts.protocol != undefined) { 312 | referrerValidated.protocol = referrerToCheckParts.protocol; 313 | } 314 | if (referrerToCheckParts.hostname != undefined) { 315 | referrerValidated.hostname = referrerToCheckParts.hostname; 316 | referrerValidated.path = referrerToCheckParts.path; 317 | } else { 318 | referrerValidated.hostname = referrerToCheckParts.path; 319 | } 320 | referrerValidated.referrer = UrlFlexParser.fullReferrerURLFromParts(referrerValidated); // used for the database key for this referrer match 321 | configuration.allowedReferrers.push(referrerValidated); 322 | } 323 | } 324 | } 325 | if (configuration.useHTTPS) { 326 | if (proxyConfigSection.httpsKeyFile !== undefined) { 327 | configuration.httpsKeyFile = proxyConfigSection.httpsKeyFile; 328 | } 329 | if (proxyConfigSection.httpsCertificateFile !== undefined) { 330 | configuration.httpsCertificateFile = proxyConfigSection.httpsCertificateFile; 331 | } 332 | if (proxyConfigSection.httpsPfxFile !== undefined) { 333 | configuration.httpsPfxFile = proxyConfigSection.httpsPfxFile; 334 | } 335 | } 336 | // listenURI can be a single string or an array of strings 337 | if (proxyConfigSection.listenURI !== undefined) { 338 | if (Array.isArray(proxyConfigSection.listenURI)) { 339 | configuration.listenURI = proxyConfigSection.listenURI.slice(); 340 | } else { 341 | configuration.listenURI = [proxyConfigSection.listenURI]; 342 | } 343 | } 344 | if (proxyConfigSection.pingPath !== undefined) { 345 | configuration.localPingURL = proxyConfigSection.pingPath; 346 | } 347 | if (proxyConfigSection.echoPath !== undefined) { 348 | configuration.localEchoURL = proxyConfigSection.echoPath; 349 | } 350 | if (proxyConfigSection.statusPath !== undefined) { 351 | configuration.localStatusURL = proxyConfigSection.statusPath; 352 | } 353 | if (proxyConfigSection.staticFilePath !== undefined) { 354 | configuration.staticFilePath = defaultConfigurationRootPath + proxyConfigSection.staticFilePath; 355 | if ( ! fs.existsSync(configuration.staticFilePath)) { 356 | configuration.staticFilePath = null; 357 | console.log(getStringTableEntry('Invalid static file path', {path: proxyConfigSection.staticFilePath})); 358 | } 359 | } 360 | } 361 | 362 | // serverURLs is an array of objects 363 | if (schema == 'json') { 364 | serverUrlsSection = json.ServerUrls; 365 | if (serverUrlsSection === undefined) { 366 | serverUrlsSection = json.serverUrls; 367 | } else { 368 | serverUrlsSection = null; 369 | } 370 | } else if (schema === 'xml') { 371 | serverUrlsSection = json.ProxyConfig.ServerUrls; 372 | if (serverUrlsSection === undefined) { 373 | serverUrlsSection = json.ProxyConfig.serverUrls; 374 | } 375 | if (serverUrlsSection !== undefined && Array.isArray(serverUrlsSection) && serverUrlsSection.length == 1) { 376 | serverUrlsSection = serverUrlsSection[0]; 377 | if (serverUrlsSection.serverUrl !== undefined) { 378 | serverUrlsSection = serverUrlsSection.serverUrl; 379 | } else if (serverUrlsSection.ServerUrl !== undefined) { 380 | serverUrlsSection = serverUrlsSection.ServerUrl; 381 | } 382 | } 383 | } else { 384 | serverUrlsSection = null; 385 | } 386 | configuration.serverUrls = []; 387 | if (serverUrlsSection !== undefined && serverUrlsSection !== null) { 388 | if (Array.isArray(serverUrlsSection)) { 389 | serverUrls = serverUrlsSection.slice(); // if array copy the array 390 | } else { 391 | serverUrls = [serverUrlsSection]; // if single object make it an array of 1 392 | } 393 | // iterate the array of services and validate individual settings 394 | for (i = 0; i < serverUrls.length; i ++) { 395 | serverUrl = serverUrls[i]; 396 | if (schema == 'xml' && serverUrl['$'] !== undefined) { 397 | // the xml parser put attributes in a dummy object "$" 398 | serverUrl = serverUrl['$']; 399 | } else if (schema == 'json' && serverUrl.serverUrl !== undefined) { 400 | // if the config file uses the old format {serverUrls: { serverUrl: { ... }} then convert it to the newer format. 401 | serverUrl = serverUrl.serverUrl; 402 | } 403 | serverUrl.errorMessage = ''; 404 | urlParts = UrlFlexParser.parseAndFixURLParts(serverUrl.url); 405 | if (urlParts != null) { 406 | serverUrl.protocol = urlParts.protocol; 407 | serverUrl.hostname = urlParts.hostname; 408 | serverUrl.path = urlParts.path; 409 | serverUrl.port = urlParts.port; 410 | serverUrl.query = urlParts.query; 411 | if (serverUrl.protocol == null || serverUrl.protocol == '') { 412 | serverUrl.protocol = '*'; 413 | } 414 | if (serverUrl.protocol.charAt(serverUrl.protocol.length - 1) == ':') { 415 | serverUrl.protocol = serverUrl.protocol.substr(0, serverUrl.protocol.length - 1); 416 | } 417 | if (serverUrl.hostname == null || serverUrl.hostname == '') { 418 | serverUrl.hostname = serverUrl.path; 419 | serverUrl.path = '*'; 420 | } 421 | if (serverUrl.port == null || serverUrl.port == '') { 422 | serverUrl.port = '*'; 423 | } 424 | } 425 | if (serverUrl.matchAll !== undefined) { 426 | if (typeof serverUrl.matchAll === 'string') { 427 | serverUrl.matchAll = serverUrl.matchAll.toLocaleLowerCase().trim() === 'true' || serverUrl.matchAll == '1'; 428 | } 429 | } else { 430 | serverUrl.matchAll = true; 431 | } 432 | if (serverUrl.rateLimit !== undefined) { 433 | serverUrl.rateLimit = parseInt(serverUrl.rateLimit); 434 | if (serverUrl.rateLimit < 0) { 435 | serverUrl.rateLimit = 0; 436 | } 437 | } else { 438 | serverUrl.rateLimit = 0; 439 | } 440 | if (serverUrl.rateLimitPeriod !== undefined) { 441 | serverUrl.rateLimitPeriod = parseInt(serverUrl.rateLimitPeriod); 442 | if (serverUrl.rateLimitPeriod < 0) { 443 | serverUrl.rateLimitPeriod = 0; 444 | } 445 | } else { 446 | serverUrl.rateLimitPeriod = 0; 447 | } 448 | if (serverUrl.rateLimit > 0 && serverUrl.rateLimitPeriod > 0) { 449 | serverUrl.useRateMeter = true; 450 | serverUrl.rate = serverUrl.rateLimit / serverUrl.rateLimitPeriod / 60; // how many we give out per second 451 | serverUrl.ratePeriodSeconds = 1 / serverUrl.rate; // how many seconds in 1 rate period 452 | } else { 453 | serverUrl.useRateMeter = false; 454 | serverUrl.rate = 0; 455 | serverUrl.ratePeriodSeconds = 0; 456 | } 457 | if (serverUrl.hostRedirect !== undefined && serverUrl.hostRedirect.trim().length > 0) { 458 | // If this entry specifies a host redirect then we will set everything up now instead of reparsing on every request 459 | serverUrl.parsedHostRedirect = UrlFlexParser.parseAndFixURLParts(serverUrl.hostRedirect.trim()); 460 | serverUrl.isHostRedirect = true; 461 | if (serverUrl.parsedHostRedirect.path == '' || serverUrl.parsedHostRedirect.path == '*') { 462 | // if the redirect does not specify a path, use the path from the request. Otherwise we override the request path with the redirect path. 463 | serverUrl.parsedHostRedirect.path = serverUrl.path; 464 | serverUrl.parsedHostRedirect.pathname = serverUrl.path; 465 | } 466 | } else { 467 | serverUrl.isHostRedirect = false; 468 | } 469 | if (serverUrl.parameterOverride !== undefined && serverUrl.parameterOverride.toString().trim().length > 0) { 470 | serverUrl.parameterOverride = serverUrl.parameterOverride.toLowerCase(); 471 | if (serverUrl.parameterOverride == 'referrer' || serverUrl.parameterOverride == 'true' || serverUrl.parameterOverride == '1') { 472 | serverUrl.parameterOverride = true; 473 | } else if (serverUrl.parameterOverride == 'config' || serverUrl.parameterOverride == 'configuration' || serverUrl.parameterOverride == 'false' || serverUrl.parameterOverride == '0') { 474 | serverUrl.parameterOverride = false; 475 | } else { 476 | serverUrl.errorMessage = getStringTableEntry('unexpected value for parameterOverride', {value: serverUrl.parameterOverride}); 477 | serverUrl.parameterOverride = false; 478 | } 479 | } else { 480 | serverUrl.parameterOverride = false; 481 | } 482 | serverUrl.mayRequireToken = false; 483 | if (ProjectUtilities.isPropertySet(serverUrl, 'clientId') || ProjectUtilities.isPropertySet(serverUrl, 'clientSecret') || ProjectUtilities.isPropertySet(serverUrl, 'oauth2Endpoint')) { 484 | serverUrl.clientId = ProjectUtilities.getIfPropertySet(serverUrl, 'clientId', ''); 485 | serverUrl.clientSecret = ProjectUtilities.getIfPropertySet(serverUrl, 'clientSecret', ''); 486 | serverUrl.oauth2Endpoint = ProjectUtilities.getIfPropertySet(serverUrl, 'oauth2Endpoint', defaultOAuthEndpoint); 487 | if (serverUrl.clientId.length < 1 || serverUrl.clientSecret.length < 1 || serverUrl.oauth2Endpoint < 1) { 488 | serverUrl.errorMessage = getStringTableEntry('OAuth requires clientId, clientSecret, oauth2Endpoint', null); 489 | } 490 | if (serverUrl.oauth2Endpoint.charAt(serverUrl.oauth2Endpoint.length - 1) != '/') { 491 | serverUrl.oauth2Endpoint += '/'; 492 | } 493 | } 494 | if (ProjectUtilities.isPropertySet(serverUrl, 'username') || ProjectUtilities.isPropertySet(serverUrl, 'password')) { 495 | serverUrl.username = ProjectUtilities.getIfPropertySet(serverUrl, 'username', ''); 496 | serverUrl.password = ProjectUtilities.getIfPropertySet(serverUrl, 'password', ''); 497 | if (serverUrl.username.length < 1 || serverUrl.password.length < 1) { 498 | serverUrl.errorMessage = getStringTableEntry('Must provide username/password', null); 499 | } 500 | } 501 | if (ProjectUtilities.isPropertySet(serverUrl, 'accessToken')) { 502 | // todo: should we attempt to validate the token? 503 | serverUrl.mayRequireToken = true; 504 | } 505 | serverUrl.isUserLogin = isUserLogin(serverUrl); 506 | serverUrl.isAppLogin = isAppLogin(serverUrl); 507 | serverUrl.mayRequireToken = serverUrl.mayRequireToken || serverUrl.isUserLogin || serverUrl.isAppLogin; 508 | serverUrl.totalRequests = 0; 509 | serverUrl.firstRequest = 0; 510 | serverUrl.lastRequest = 0; 511 | 512 | // TODO: Should we attempt to validate any of the following parameters? 513 | // domain; 514 | // tokenParamName; 515 | 516 | configuration.serverUrls.push(serverUrl); 517 | } 518 | } 519 | } 520 | } 521 | 522 | /** 523 | * Load the configuration file and process it by copying anything that looks valid into our 524 | * internal configuration object. This function loads asynchronously so it returns before the 525 | * file is loaded or processed. 526 | * @param configFile {string} path to the configuration file. 527 | */ 528 | function loadConfigurationFile (configFile) { 529 | var promise, 530 | stringTablePath; 531 | 532 | try { 533 | stringTablePath = defaultRequireRootPath + defaultConfigurationFilePath + '/en.json'; 534 | configuration.stringTable = require(stringTablePath); 535 | } catch (exception) { 536 | var currentPath = path.dirname(fs.realpathSync(__filename)); 537 | QuickLogger.logErrorEvent('Cannot load strings table from ' + stringTablePath + ' from ' + currentPath); 538 | } 539 | promise = new Promise(function(resolvePromise, rejectPromise) { 540 | if (configFile == undefined || configFile == null || configFile.length == 0) { 541 | if (configuration.testMode) { 542 | configFile = joinPath(defaultConfigurationRootPath + defaultConfigurationFilePath, defaultConfigurationTestFileName); 543 | } else { 544 | configFile = joinPath(defaultConfigurationRootPath + defaultConfigurationFilePath, defaultConfigurationFileName); 545 | } 546 | if (defaultConfigurationFileType != null && defaultConfigurationFilePath.length > 0) { 547 | configFile += '.' + defaultConfigurationFileType; 548 | } 549 | } 550 | QuickLogger.logInfoEvent(getStringTableEntry('Loading configuration from', {file: configFile})); 551 | if (ProjectUtilities.isFileTypeJson(configFile)) { 552 | loadJsonFile(configFile).then(function (jsonObject) { 553 | postParseConfigurationFile(jsonObject, 'json'); 554 | configurationComplete = true; 555 | if (isConfigurationValid()) { 556 | resolvePromise(); 557 | } else { 558 | rejectPromise(new Error(getStringTableEntry('Configuration file not valid', null))); 559 | } 560 | }, function (error) { 561 | QuickLogger.logErrorEvent(getStringTableEntry('Invalid configuration file format', {error: error.toString()})); 562 | }); 563 | } else { 564 | var xmlParser = new xml2js.Parser(); 565 | fs.readFile(configFile, function(fileError, xmlData) { 566 | if (fileError == null) { 567 | xmlParser.parseString(xmlData, function (xmlError, xmlObject) { 568 | if (xmlError == null) { 569 | postParseConfigurationFile(xmlObject, 'xml'); 570 | configurationComplete = true; 571 | if (isConfigurationValid()) { 572 | resolvePromise(); 573 | } else { 574 | rejectPromise(new Error(getStringTableEntry('Configuration file not valid', null))); 575 | } 576 | } else { 577 | rejectPromise(xmlError); 578 | } 579 | }); 580 | } else { 581 | console.log("File error on " + configFile + " at path " + path.dirname(fs.realpathSync(__filename))); 582 | rejectPromise(fileError); 583 | } 584 | }); 585 | } 586 | }); 587 | return promise; 588 | } 589 | 590 | parseCommandLineOptions(); 591 | module.exports.configuration = configuration; 592 | module.exports.isTestMode = isTestMode; 593 | module.exports.isConfigurationValid = isConfigurationValid; 594 | module.exports.loadConfigurationFile = loadConfigurationFile; 595 | module.exports.getStringTableEntry = getStringTableEntry; 596 | 597 | if (configuration.testMode) { 598 | var test = require('./test.js'); 599 | } -------------------------------------------------------------------------------- /bin/ProjectUtilities.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Project utility functions: a compendium of miscellaneous JavaScript helper functions that we tend to reuse on 3 | * lots of projects. 4 | */ 5 | 6 | /** 7 | * Convert time in milliseconds into a printable hh:mm:ss string. Hours is not constrained. 8 | * @param timeInMilliseconds 9 | * @returns {string} 10 | */ 11 | module.exports.formatMillisecondsToHHMMSS = function(timeInMilliseconds) { 12 | var hours, 13 | minutes, 14 | seconds = timeInMilliseconds / 1000; 15 | hours = Math.floor(seconds / 3600); 16 | minutes = Math.floor(seconds / 60) % 60; 17 | seconds = Math.floor(seconds) % 60; 18 | return (hours < 10 ? '0' : '') + hours + ':' + ((minutes < 10 ? '0' : '') + minutes) + ':' + (seconds < 10 ? '0' : '') + seconds; 19 | }; 20 | 21 | /** 22 | * Determine if the subject string starts with the needle string. Performs a case insensitive comparison. 23 | * @param subject 24 | * @param needle 25 | * @returns {boolean} 26 | */ 27 | module.exports.startsWith = function(subject, needle) { 28 | var subjectLowerCase = subject.toLowerCase(), 29 | needleLowerCase = needle.toLowerCase(); 30 | return subjectLowerCase.indexOf(needleLowerCase) == 0; 31 | }; 32 | 33 | /** 34 | * Determine if the subject string ends with the needle string. Performs a case insensitive comparison. 35 | * @param subject 36 | * @param needle 37 | * @returns {boolean} 38 | */ 39 | module.exports.endsWith = function(subject, needle) { 40 | var subjectLowerCase = subject.toLowerCase(), 41 | needleLowerCase = needle.toLowerCase(), 42 | startIndex = subjectLowerCase.length - needleLowerCase.length; 43 | return subjectLowerCase.indexOf(needleLowerCase, startIndex) == startIndex; 44 | }; 45 | 46 | /** 47 | * Determine if a given configuration variable is set. Set would mean it is a property on the object and it is not empty. 48 | * @param object {object} subject to search. 49 | * @param key {string} key to look up in object. 50 | * @returns {boolean} true if key is a property of object. 51 | */ 52 | module.exports.isPropertySet = function(object, key) { 53 | var isSet = false; 54 | if (object[key] !== undefined) { 55 | isSet = object[key].toString().trim().length > 0; 56 | } 57 | return isSet; 58 | }; 59 | 60 | /** 61 | * Determine if a given configuration variable is set. Set would mean it is a property on the object and it is not empty. 62 | * @param object {object} subject to search. 63 | * @param key {string} key to look up in object. 64 | * @param defaultValue {*} value to return if key is not found in object, or if the key in object has an empty value. 65 | * @returns {*} either the value of key in object, or the default value. 66 | */ 67 | module.exports.getIfPropertySet = function(object, key, defaultValue) { 68 | if (object[key] !== undefined && object[key].toString().trim().length > 0) { 69 | return object[key]; 70 | } else { 71 | return defaultValue; 72 | } 73 | }; 74 | 75 | /** 76 | * Add a key/value pair to an existing object only if the key does not already exist or if the key exists but 77 | * it is empty. If the key exists with a non-empty value then its value is not changed. 78 | * @param object {object} to check and possibly alter. 79 | * @param key {string} key to look up in object. 80 | * @param value {*} value to set to key if key does not exist in object. 81 | * @returns {object} return the object 82 | */ 83 | module.exports.addIfPropertyNotSet = function(object, key, value) { 84 | if (object[key] === undefined || object[key] == null || object[key].toString().trim().length == 0) { 85 | object[key] = value; 86 | } 87 | return object; 88 | }; 89 | 90 | /** 91 | * Return the current document query string as an object with 92 | * key/value pairs converted to properties. 93 | * 94 | * @method queryStringToObject 95 | * @param urlParameterString {string} A query string to parse as the key value pairs (key=value&key=value) string. 96 | * @return {object} result The query string converted to an object of key/value pairs. 97 | */ 98 | module.exports.queryStringToObject = function(urlParameterString) { 99 | var match, 100 | search = /([^&=]+)=?([^&]*)/g, 101 | decode = function (s) { 102 | return decodeURIComponent(s.replace(/\+/g, ' ')); 103 | }, 104 | result = {}; 105 | if (urlParameterString.charAt(0) == '?') { 106 | urlParameterString = urlParameterString.substr(1); 107 | } 108 | while (match = search.exec(urlParameterString)) { 109 | result[decode(match[1])] = decode(match[2]); 110 | } 111 | return result; 112 | }; 113 | 114 | /** 115 | * Return the query string representation of an object with key/value pairs converted to string. 116 | * Does not handle recursion, but will flatten an array. Values are url encoded. ? is not added to the result. 117 | * 118 | * @method objectToQueryString 119 | * @param {object} object The object of key/value pairs. 120 | * @return {string} urlParamterString A query string (key=value&key=value). 121 | */ 122 | module.exports.objectToQueryString = function(object) { 123 | var urlParameterString = '', 124 | key, 125 | value; 126 | 127 | if (object !== undefined && object != null) { 128 | for (key in object) { 129 | if (object.hasOwnProperty(key)) { 130 | value = object[key]; 131 | if (value === undefined || value === null) { 132 | continue; 133 | } else if (Array.isArray(value)) { 134 | value = value.join(','); 135 | } else { 136 | value = value.toString(); 137 | } 138 | value = encodeURIComponent(value); 139 | key = encodeURIComponent(key); 140 | urlParameterString += (urlParameterString.length == 0 ? '' : '&') + key + '=' + value; 141 | } 142 | } 143 | } 144 | return urlParameterString; 145 | }; 146 | 147 | /** 148 | * Look for the token in a string that is assumed to be either a URL query string or a JSON string. If the 149 | * token is found the value is returned. This is useful when you need to pull out one value from a large string 150 | * and you don't want to convert that large string into yet another memory-hogging object/array data structure and 151 | * then traverse the structure to try to identify one value. 152 | * @param source {string} string to search and extract 153 | * @param token {string} the token we are looking for in source. 154 | * @return {string} the value of the token, '' if not found. 155 | */ 156 | module.exports.findTokenInString = function(source, token) { 157 | var found, 158 | searchToken, 159 | value = ''; 160 | 161 | if (source !== undefined && token !== undefined && source.trim().length > 0 && token.trim().length > 0) { 162 | searchToken = '(\\?|&|\\/|)' + token + '='; 163 | found = source.search(searchToken); 164 | if (found >= 0) { 165 | // found query string style &token=value, cut from = to next & or EOS 166 | value = source.substr(found); 167 | found = value.indexOf('='); 168 | if (found >= 0) { 169 | value = value.substr(found + 1); 170 | found = value.indexOf('&'); 171 | if (found > 0) { 172 | value = value.substr(0, found); 173 | } 174 | } 175 | } else { 176 | // found json style "token": "value", get quoted value 177 | searchToken = '"' + token +'":'; 178 | found = source.search(searchToken); 179 | if (found >= 0) { 180 | value = source.substr(found + searchToken.length); 181 | searchToken = '"([^"]*)"'; // find next quoted string 182 | value = value.match(searchToken); 183 | if (value != null) { 184 | value = value[1]; 185 | } else { 186 | value = ''; 187 | } 188 | } 189 | } 190 | } 191 | return value; 192 | }; 193 | 194 | /** 195 | * Look for the token in a string that is assumed to be either a URL query string or a JSON string. If the 196 | * token is found the value is returned. This is useful when you need to pull out one value from a large string 197 | * and you don't want to convert that large string into yet another memory-hogging object/array data structure and 198 | * then traverse the structure to try to identify one value. 199 | * @param source {string} string to search and extract 200 | * @param token {string} the token we are looking for in source. 201 | * @return {number} the value of the token, 0 if not found. 202 | */ 203 | module.exports.findNumberAfterTokenInString = function(source, token) { 204 | var found, 205 | searchToken, 206 | value = 0; 207 | 208 | if (source !== undefined && token !== undefined && source.trim().length > 0 && token.trim().length > 0) { 209 | searchToken = '(\\?|&|\\/|)' + token + '='; 210 | found = source.search(searchToken); 211 | if (found >= 0) { 212 | // found query string style &token=value, cut from = to next & or EOS 213 | value = source.substr(found); 214 | found = value.indexOf('='); 215 | if (found >= 0) { 216 | value = value.substr(found + 1); 217 | found = value.indexOf('&'); 218 | if (found > 0) { 219 | value = value.substr(0, found); 220 | } 221 | } 222 | } else { 223 | // found json style "token": value, get next number value 224 | searchToken = '"' + token +'":'; 225 | found = source.search(searchToken); 226 | if (found >= 0) { 227 | value = source.substr(found + searchToken.length); 228 | value = parseInt(value); 229 | } 230 | } 231 | } 232 | return value; 233 | }; 234 | 235 | /** 236 | * Replace occurrences of {token} with matching keyed values from parameters array. 237 | * 238 | * @param {string} text text containing tokens to be replaced. Tokens are surrounded with {}. 239 | * @param {Array} parameters array/object of key/value pairs to match keys as tokens in text and replace with value. 240 | * @return {string} text replaced string. 241 | */ 242 | module.exports.tokenReplace = function (text, parameters) { 243 | var token, 244 | regexMatch; 245 | 246 | if (Array.isArray(parameters) || (parameters !== null && typeof parameters === 'object')) { 247 | for (token in parameters) { 248 | if (parameters.hasOwnProperty(token)) { 249 | regexMatch = new RegExp("\{" + token + "\}", 'g'); 250 | text = text.replace(regexMatch, parameters[token]); 251 | } 252 | } 253 | } 254 | return text; 255 | }; 256 | 257 | /** 258 | * Return true if the file name appears to be a json file type (because it ends with .json). 259 | * @param fileName 260 | * @returns {boolean} 261 | */ 262 | module.exports.isFileTypeJson = function (fileName) { 263 | var regex = /\.json$/i; 264 | return regex.test(fileName, 'i'); 265 | }; 266 | 267 | /** 268 | * Determine if object has no added properties. 269 | * @param object 270 | * @returns {boolean} 271 | */ 272 | module.exports.isEmptyObject = function (object) { 273 | for (var property in object) { 274 | if (object.hasOwnProperty(property)) { 275 | return false; 276 | } 277 | } 278 | return true; 279 | }; 280 | -------------------------------------------------------------------------------- /bin/QuickLogger.js: -------------------------------------------------------------------------------- 1 | /** 2 | * "Quick" and simple logging function. Logs messages to a log file. 3 | * Created on 8/24/16. 4 | */ 5 | 6 | const fs = require('fs'); 7 | 8 | var defaultLogFileName = 'arcgis-proxy.txt', 9 | logFileName = 'arcgis-proxy-node.log', 10 | logToConsole = true, 11 | logLevelValue = 9; 12 | 13 | 14 | // LOGLEVELs control what type of logging will appear in the log file and on the console. 15 | module.exports.LOGLEVEL = { 16 | ALL: {label: "ALL", value: 9, key: "A"}, 17 | INFO: {label: "INFO", value: 5, key: "I"}, 18 | WARN: {label: "WARN", value: 4, key: "W"}, 19 | ERROR: {label: "ERROR", value: 3, key: "E"}, 20 | NONE: {label: "NONE", value: 0, key: "X"} 21 | }; 22 | 23 | /** 24 | * Determine if a requested log level is greater or equal to the current log level in effect. This is helpful if 25 | * you want to do something based on a particular logging level or higher. 26 | * @param logLevelLabel 27 | * @returns {boolean} 28 | */ 29 | module.exports.ifLogLevelGreaterOrEqual = function(logLevelLabel) { 30 | var logInfo = this.getLogLevelInfoFromLabel(logLevelLabel); 31 | if (logInfo != null) { 32 | return logInfo.value >= logLevelValue; 33 | } else { 34 | return false; 35 | } 36 | }; 37 | 38 | /** 39 | * Check the configuration and verify access to the log file. The configuration object should have the following 40 | * attributes, any of which are optional and when not found a suitable default is used: 41 | * { 42 | * logLevel: "ALL", 43 | * logToConsole: true, 44 | * logFilePath: "./", 45 | * logFileName: "file-name.txt", 46 | * } 47 | * @param configuration {object} see above. 48 | * @returns {boolean} true if a valid configuration is consumed, false if something is invalid and we cannot function. 49 | */ 50 | module.exports.setConfiguration = function(configuration) { 51 | var logFilepath, 52 | logLevelInfo, 53 | isValid = false; 54 | 55 | logToConsole = configuration.logConsole !== undefined ? configuration.logConsole == true : false; 56 | logLevelValue = configuration.logLevel !== undefined ? configuration.logLevel : this.LOGLEVEL.NONE.value; 57 | if (configuration.logFilePath != null || configuration.logFileName != null) { 58 | if (configuration.logFilePath == null) { 59 | logFilePath = './'; 60 | } else if (configuration.logFilePath.charAt(configuration.logFilePath.length - 1) != '/') { 61 | logFilePath = configuration.logFilePath + '/'; 62 | } else { 63 | logFilePath = configuration.logFilePath; 64 | } 65 | if (configuration.logFileName != null) { 66 | if (configuration.logFileName.charAt(0) == '/') { 67 | logFileName = logFilePath + configuration.logFileName.substr(1); 68 | } else { 69 | logFileName = logFilePath + configuration.logFileName; 70 | } 71 | } else { 72 | logFileName = logFilePath + defaultLogFileName; 73 | } 74 | } else { 75 | logFileName = './' + defaultLogFileName; 76 | } 77 | if (logFileName != null) { 78 | try { 79 | fs.accessSync(logFilePath, fs.constants.R_OK | fs.constants.W_OK); 80 | isValid = true; 81 | } catch (error) { 82 | this.logEventImmediately(this.LOGLEVEL.ERROR.value, 'No write access to log file ' + logFilePath + ": " + error.toString()); 83 | logFileName = null; 84 | isValid = false; 85 | } 86 | } 87 | return isValid; 88 | }; 89 | 90 | /** 91 | * Helper function to log an INFO level event. 92 | * @param message 93 | */ 94 | module.exports.logInfoEvent = function(message) { 95 | this.logEvent(this.LOGLEVEL.INFO.value, message); 96 | }; 97 | 98 | /** 99 | * Helper function to log an WARN level event. 100 | * @param message 101 | */ 102 | module.exports.logWarnEvent = function(message) { 103 | this.logEvent(this.LOGLEVEL.WARN.value, message); 104 | }; 105 | 106 | /** 107 | * Helper function to log an ERROR level event. 108 | * @param message 109 | */ 110 | module.exports.logErrorEvent = function(message) { 111 | this.logEvent(this.LOGLEVEL.ERROR.value, message); 112 | }; 113 | 114 | /** 115 | * Log a message to a log file only if a log file was defined and we have write access to it. This 116 | * function appends a new line on the end of each call. 117 | * 118 | * @param logLevelForMessage {int} the log level value used to declare the level of logging this event represents. If this value 119 | * is less than the configuration log level then this event is not logged. 120 | * @param message {string} the message to write to the log file. 121 | */ 122 | module.exports.logEvent = function(logLevelForMessage, message) { 123 | if (logLevelForMessage <= logLevelValue) { 124 | if (logFileName != null) { 125 | fs.appendFile(logFileName, this.formatLogMessage(this.formatLogLevelKey(logLevelForMessage) + message), {flag: 'a'}, function (error) { 126 | if (error != null) { 127 | console.log('*** Error writing to log file ' + logFileName + ": " + error.toString()); 128 | throw error; 129 | } 130 | }); 131 | } 132 | if (logToConsole) { 133 | console.log(message); 134 | } 135 | } 136 | }; 137 | 138 | /** 139 | * Adds current date and CRLF to a log message. 140 | * @param message 141 | * @returns {string} 142 | */ 143 | module.exports.formatLogMessage = function(message) { 144 | var today = new Date(); 145 | return today.toISOString() + ": " + message.toString() + '\n'; 146 | }; 147 | 148 | /** 149 | * Return a formatted key representing the log level that was used to log the event. This way a log processor can 150 | * see the level that matched the log event. 151 | * @param logLevel 152 | * @returns {String} Log level identifier key with formatting. 153 | */ 154 | module.exports.formatLogLevelKey = function(logLevel) { 155 | var logInfo = this.getLogLevelInfoFromValue(logLevel); 156 | if (logInfo != null) { 157 | return '[' + logInfo.key + '] '; 158 | } else { 159 | return ''; 160 | } 161 | }; 162 | 163 | /** 164 | * Given a log level value return the related log level info. 165 | * @param logLevelValue the integer value of the log level we are interested in. 166 | * @returns {*} Object if match, null if undefined log level value. 167 | */ 168 | module.exports.getLogLevelInfoFromValue = function(logLevelValue) { 169 | var logInfoKey, 170 | logInfo; 171 | 172 | for (logInfoKey in this.LOGLEVEL) { 173 | if (this.LOGLEVEL.hasOwnProperty(logInfoKey)) { 174 | logInfo = this.LOGLEVEL[logInfoKey]; 175 | if (logInfo.value == logLevelValue) { 176 | return logInfo; 177 | } 178 | } 179 | } 180 | return null; 181 | }; 182 | 183 | /** 184 | * Given a log level label return the related log level info. 185 | * @param logLevelLabel the string label of the log level we are interested in. 186 | * @returns {*} Object if match, null if undefined log level value. 187 | */ 188 | module.exports.getLogLevelInfoFromLabel = function(logLevelLabel) { 189 | var logInfoKey, 190 | logInfo; 191 | 192 | for (logInfoKey in this.LOGLEVEL) { 193 | if (this.LOGLEVEL.hasOwnProperty(logInfoKey)) { 194 | logInfo = this.LOGLEVEL[logInfoKey]; 195 | if (logInfo.label == logLevelLabel) { 196 | return logInfo; 197 | } 198 | } 199 | } 200 | return null; 201 | }; 202 | 203 | /** 204 | * Synchronous file write for logging when we are in a critical situation, like shut down. 205 | * @param logLevelForMessage {int} logging level for this message. 206 | * @param message {string} a message to show in the log. 207 | */ 208 | module.exports.logEventImmediately = function(logLevelForMessage, message) { 209 | if (logLevelForMessage <= logLevelValue) { 210 | if (logFileName != null) { 211 | fs.appendFileSync(logFileName, this.formatLogMessage(message)); 212 | } 213 | if (logToConsole) { 214 | console.log(message); 215 | } 216 | } 217 | }; 218 | 219 | /** 220 | * Return size of the log file. 221 | */ 222 | module.exports.getLogFileSize = function() { 223 | var fstatus, 224 | result; 225 | 226 | try { 227 | if (logFileName != null) { 228 | fstatus = fs.statSync(logFileName); 229 | if (fstatus != null) { 230 | result = Math.round(fstatus.size / 1000) + 'K'; 231 | } else { 232 | result = 'Log file error.'; 233 | } 234 | } else { 235 | result = 'No log file.'; 236 | } 237 | } catch (exception) { 238 | result = 'Log file error ' + exception.toLocaleString(); 239 | } 240 | return result; 241 | }; 242 | -------------------------------------------------------------------------------- /bin/RateMeter.js: -------------------------------------------------------------------------------- 1 | /** 2 | * RateMeter class. Interface with persistent thread-safe storage engine to coordinate 3 | * the rate monitoring data and tracking. We are using sqlite3 with JavaScript to provide 4 | * a persistent storage engine that will work when multiple Node processes are running 5 | * and contending for access to the data store. 6 | * 7 | * Each row in the database table tracks the resource utilization of each entry in the serverURLs table. 8 | * 9 | * Since accessing the database requires asynchronous functions, most of the query functions 10 | * return a promise that will later resolve with the result. 11 | * 12 | */ 13 | 14 | const sqlite3 = require('sqlite3'); 15 | const fs = require('fs'); 16 | 17 | 18 | module.exports = function (serverURLs, allowedReferrers, logFunction) { 19 | var dbName = "proxy.sqlite"; 20 | var dbFileAccessMode = fs.constants.R_OK | fs.constants.W_OK; 21 | var dbConnection = null; 22 | var isNewDatabase = false; 23 | var serverURLConfig = serverURLs; 24 | var serverAllowedReferrers = allowedReferrers; 25 | var errorLoggingFunction = logFunction; 26 | 27 | /** 28 | * Internal database error logger that formats a nice error message and then calls the provided 29 | * error logging function to actually log it. This way RateMeter doesn't have to know anything 30 | * about how the app wants to handle logging. 31 | * @param fromWhere {string} an indication where in this module the error occurred. 32 | * @param sql {string|null} the sql query if you want to show it in the log 33 | * @param params {Array|string|null} the parameters provided to the sql query 34 | * @param databaseError {Exception|null} the error object, if not null then must support toString() 35 | */ 36 | function logDatabaseError(fromWhere, sql, params, databaseError) { 37 | var message; 38 | 39 | if (errorLoggingFunction != null) { 40 | message = 'Internal database error'; 41 | if (databaseError != null) { 42 | message += ': ' + databaseError.toString(); 43 | } 44 | if (fromWhere != null && fromWhere.length > 0) { 45 | message += ' in ' + fromWhere; 46 | } 47 | if (sql != null && sql.length > 0) { 48 | message += ' ' + sql; 49 | } 50 | if (params != null) { 51 | if (typeof params == Array && params.length > 0) { 52 | message += '(' + params.join + ')'; 53 | } else { 54 | message += '(' + params.toString() + ')'; 55 | } 56 | } 57 | errorLoggingFunction(message); 58 | } 59 | } 60 | 61 | /** 62 | * A time function to return fractions of a second. 63 | * @returns {number} 64 | */ 65 | function getMicroTime() { 66 | return new Date().getTime() / 1000; 67 | } 68 | 69 | /** 70 | * Create the entire database and all records we are going to rate monitor. We create all records in advance because 71 | * we know them now and they won't change and we will save runtime overhead by not creating new rows and indexing. 72 | */ 73 | function createDatabaseIfNotExists() { 74 | var sql, 75 | params, 76 | serverURL, 77 | rate, 78 | timeOfAccess = getMicroTime(), 79 | serverIndex, 80 | referrerIndex; 81 | 82 | if (dbConnection != null) { 83 | dbConnection.serialize(function() { 84 | dbConnection.run('CREATE TABLE IF NOT EXISTS ips (id INTEGER PRIMARY KEY, url VARCHAR(255) not null, referrer VARCHAR(255) not null, count INTEGER not null default(0), rate INTEGER not null default(0), time INTEGER not null default(0), total INTEGER not null default(0), rejected INTEGER not null default(0))'); 85 | dbConnection.run('CREATE UNIQUE INDEX IF NOT EXISTS url_referrer ON ips (url, referrer)'); 86 | dbConnection.run('DELETE from ips'); 87 | sql = 'INSERT OR IGNORE INTO ips (url, referrer, count, rate, time, total, rejected) VALUES (?, ?, ?, ?, ?, ?, ?)'; 88 | for (serverIndex = 0; serverIndex < serverURLConfig.length; serverIndex ++) { 89 | serverURL = serverURLConfig[serverIndex]; 90 | if (serverURL.useRateMeter) { 91 | for (referrerIndex = 0; referrerIndex < serverAllowedReferrers.length; referrerIndex ++) { 92 | params = [serverURL.url, serverAllowedReferrers[referrerIndex].referrer, 0, serverURL.rate, timeOfAccess, 0, 0]; 93 | dbConnection.run(sql, params); 94 | } 95 | } 96 | } 97 | }); 98 | } 99 | } 100 | 101 | /** 102 | * Refresh the entire table. This function will rebuild the entire table. When you call this function all current 103 | * counters and rate meters are removed. 104 | * @param newServerUrlTable 105 | * @param newReferrers 106 | */ 107 | function refreshServerUrls(newServerUrlTable, newReferrers) { 108 | var serverIndex, 109 | referrerIndex, 110 | serverURL, 111 | timeOfAccess = getMicroTime(), 112 | sql, 113 | params; 114 | 115 | serverURLConfig = newServerUrlTable; 116 | serverAllowedReferrers = newReferrers; 117 | if (dbConnection != null) { 118 | dbConnection.serialize(function() { 119 | dbConnection.run('TRUNCATE TABLE ips'); 120 | sql = 'INSERT OR IGNORE INTO ips (url, referrer, count, rate, time, total, rejected) VALUES (?, ?, ?, ?, ?, ?, ?)'; 121 | for (serverIndex = 0; serverIndex < newServerUrlTable.length; serverIndex ++) { 122 | serverURL = newServerUrlTable[serverIndex]; 123 | if (serverURL.useRateMeter) { 124 | for (referrerIndex = 0; referrerIndex < newReferrers.length; referrerIndex ++) { 125 | params = [serverURL.url, newReferrers[referrerIndex], 0, serverURL.rate, timeOfAccess, 0, 0]; 126 | dbConnection.run(sql, params); 127 | } 128 | } 129 | } 130 | }); 131 | } 132 | } 133 | 134 | /** 135 | * Return the database connection. A new connection is created if one did not already exist. 136 | * @returns {*} 137 | */ 138 | function openDatabase() { 139 | if (dbConnection == null) { 140 | isNewDatabase = false; // ! fs.accessSync(dbName, dbFileAccessMode); // TODO: why does this fail? 141 | dbConnection = new sqlite3.Database(dbName, sqlite3.OPEN_READWRITE | sqlite3.OPEN_CREATE, createDatabaseIfNotExists); 142 | if (dbConnection != null) { 143 | if (isNewDatabase) { 144 | fs.chmodSync(dbName, '770'); 145 | } 146 | } 147 | } 148 | return dbConnection; 149 | } 150 | 151 | /** 152 | * Close the database freeing any connections and resources consumed. 153 | */ 154 | function closeDatabase() { 155 | if (dbConnection != null) { 156 | dbConnection.close(); 157 | dbConnection = null; 158 | } 159 | } 160 | 161 | /** 162 | * Query the database for an aggregate count of all monitored connections for a given URL we are monitoring. 163 | * @param url {string} If null or '' then return a sum of all totals, otherwise use this as a key to look up the total for. 164 | * @returns {Promise} The resolve function is passed the total as the first parameter. 165 | */ 166 | function getTotalCount(url) { 167 | var sql, 168 | params, 169 | promise; 170 | 171 | promise = new Promise(function(resolvePromise, rejectPromise) { 172 | if (dbConnection == null) { 173 | openDatabase(); 174 | } 175 | if (dbConnection != null) { 176 | if (url != null && url.length > 0) { 177 | sql = "SELECT sum(total) as total FROM ips where url=?"; 178 | params = [url]; 179 | } else { 180 | sql = "SELECT sum(total) as total FROM ips"; 181 | params = []; 182 | } 183 | dbConnection.get(sql, params, function (error, queryResult) { 184 | if (error != null) { 185 | logDatabaseError('getTotalCount', sql, params, error); 186 | rejectPromise(error); 187 | } else { 188 | resolvePromise(queryResult.total); 189 | } 190 | }); 191 | } 192 | }); 193 | return promise; 194 | } 195 | 196 | /** 197 | * Produce a dump of all rows in the table. Returns a Promise that will pass an array of table rows as 198 | * its first parameter to the resolve function. 199 | * @returns {Promise} We promise to eventually return an array of all rows in the table. 200 | */ 201 | function allRowsAsArray() { 202 | var promise, 203 | sql, 204 | params; 205 | 206 | promise = new Promise(function(resolvePromise, rejectPromise) { 207 | if (dbConnection == null) { 208 | openDatabase(); 209 | } 210 | if (dbConnection != null) { 211 | sql = "SELECT id, url, referrer, total, count, rejected, rate, time FROM ips"; 212 | params = []; 213 | dbConnection.all(sql, params, function (error, queryResult) { 214 | if (error != null) { 215 | logDatabaseError('allRowsAsArray', sql, params, error); 216 | rejectPromise(error); 217 | } else { 218 | resolvePromise(queryResult); 219 | } 220 | }); 221 | } else { 222 | rejectPromise(Error('Not able to open or create the database.')); 223 | } 224 | }); 225 | return promise; 226 | } 227 | 228 | /** 229 | * Determine if the monitored resource (by its id) is under it's allotted rate monitor cap. When returning true 230 | * this function also updates the monitored rate. 231 | * @param referrer {string} the referrer to track. 232 | * @param serverURL {object} the URL info we are tracking that matches this request. 233 | * @returns {Promise} returns a Promise where the resolve function is passed a boolean that is false if this resource exceeded its rate. 234 | */ 235 | function isUnderMeterCap(referrer, serverURL) { 236 | var timeOfRequest = getMicroTime(), 237 | newCount, 238 | refreshTime, 239 | sql, 240 | params, 241 | promise, 242 | isOK = false; 243 | 244 | promise = new Promise(function(resolvePromise, rejectPromise) { 245 | if (dbConnection != null) { 246 | // read db by url to get current data (since other threads may also be updating it.) 247 | // check if count exceeded 248 | // if not, update record with new count and timestamp. 249 | 250 | dbConnection.serialize(function () { 251 | sql = "SELECT id, url, referrer, total, count, rate, time FROM ips WHERE referrer=? and url=?"; 252 | params = [referrer, serverURL.url]; 253 | 254 | dbConnection.get(sql, params, function (error, queryResult) { 255 | if (error != null) { 256 | logDatabaseError('selectLastRequest', sql, params, error); 257 | rejectPromise(error); 258 | } else { 259 | if (queryResult != null) { 260 | if (queryResult.count == 0 || (queryResult.time + serverURL.ratePeriodSeconds <= timeOfRequest)) { 261 | // either the first time in, or the prior time window has expired 262 | newCount = 1; 263 | refreshTime = timeOfRequest; 264 | isOK = true; 265 | } else if (queryResult.count < serverURL.rate) { 266 | // in the current time window we have not yet given out the maximum number of hits 267 | newCount = queryResult.count + 1; 268 | refreshTime = queryResult.time; 269 | isOK = true; 270 | // } else { 271 | // already gave out the limit for the current time window 272 | // isOK = false; 273 | } 274 | if (isOK) { 275 | sql = "UPDATE ips SET total=total+1, count=?, time=? WHERE id=?"; 276 | params = [newCount, refreshTime, queryResult.id]; 277 | dbConnection.run(sql, params, function (error) { 278 | if (error != null) { 279 | logDatabaseError('updateRequest', sql, params, error); 280 | } 281 | }); 282 | } else { 283 | sql = "UPDATE ips SET rejected=rejected+1 WHERE id=?"; 284 | params = [queryResult.id]; 285 | dbConnection.run(sql, params, function (error) { 286 | if (error != null) { 287 | logDatabaseError('updateRequest', sql, params, error); 288 | } 289 | }); 290 | } 291 | resolvePromise(isOK); 292 | } else { 293 | error = new Error('no record exists for ' + referrer + ', ' + url); 294 | logDatabaseError('selectLastRequest', sql, params, error); 295 | resolvePromise(error); 296 | } 297 | } 298 | }); 299 | }); 300 | } else { 301 | rejectPromise(new Error('Database connection was not open. Call start() first.')); 302 | } 303 | }); 304 | return promise; 305 | } 306 | 307 | /** 308 | * This is the public API: 309 | */ 310 | return { 311 | /** 312 | * Start should be called before monitoring begins. This opens the database connection and manages 313 | * on connection resource for the node thread we are running on. If start() is not called then each 314 | * call to isExceeded will open and close its own database connection. 315 | */ 316 | start: function() { 317 | openDatabase(); 318 | }, 319 | 320 | /** 321 | * Call stop when shutting down or monitoring is no longer needed. This closes the database connection 322 | * and frees any resources consumed by this object. 323 | */ 324 | stop: function() { 325 | closeDatabase(); 326 | }, 327 | 328 | /** 329 | * Determine if the resource (given its URL and the referrer who is accessing it) has exceeded its rate 330 | * monitoring cap. Returns a Promise that will resolve when the query completes. The Promise is passed 331 | * either true when the current rate is less than the required maximum rate or false after it has been 332 | * exceeded. start() must be called before this function or it will fail. 333 | * @param referrer {string} referrer we are monitoring. 334 | * @param url {string} url of the resource we are monitoring requested by referrer. 335 | * @returns {Promise} A single boolean value is passed to the resolve function that will be true while 336 | * under the rate cap, and false when exceeding the rate cap. 337 | */ 338 | isUnderRate: function (referrer, url) { 339 | return isUnderMeterCap(referrer, url); 340 | }, 341 | 342 | /** 343 | * If the serverURLs table changes after the constructor was called you can repopulate it 344 | * by calling this method with the new table. This will drop all active rate counters and 345 | * start them again. 346 | * @param serverUrls 347 | * @returns {*} 348 | */ 349 | refreshUrlTable: function(serverUrls) { 350 | return refreshServerUrls(serverUrls); 351 | }, 352 | 353 | /** 354 | * Produces an array of objects of all rows in the table. This function returns a Promise that will 355 | * resolve with the array of database rows, each row is an object. 356 | * @returns {Promise} 357 | */ 358 | databaseDump: function() { 359 | return allRowsAsArray(); 360 | } 361 | } 362 | }; 363 | -------------------------------------------------------------------------------- /bin/UrlFlexParser.js: -------------------------------------------------------------------------------- 1 | /** 2 | * A URL parser that also handles the weird rules we employ in our proxy service. 3 | * Accepted URL formats: 4 | * http://service.host.tld/proxy?http://services.arcgisonline.com/ArcGIS/rest/services/?f=pjson 5 | * http://service.host.tld/sproxy?http://services.arcgisonline.com/ArcGIS/rest/services/?f=pjson 6 | * http://service.host.tld/proxy/http/services.arcgisonline.com/ArcGIS/rest/services/?f=pjson 7 | * http://service.host.tld/proxy/http://services.arcgisonline.com/ArcGIS/rest/services/?f=pjson 8 | * http://service.host.tld/path/path?query=string&key=value 9 | * https://service.host.tld/path/path?query=string&key=value 10 | * *://service.host.tld/path/path?query=string&key=value 11 | * //service.host.tld/path/path?query=string&key=value 12 | * //*.host.tld/path/path?query=string&key=value 13 | * *.host.tld/path/path?query=string&key=value 14 | * *.host.tld/*?query=string&key=value 15 | * 16 | * Any piece is optional and defaults are used if a part is not identified. 17 | * 18 | * The part after the proxy path is taken as the service to proxy to. It is looked up in the serviceURLs table 19 | * and if matched the service information of that entry is used to make the request with the service. What the 20 | * service responds with is then passed back to the caller. 21 | * 22 | * TODO: This would be better if it were an object/class definition with properties and methods. The current 23 | * implementation is a result of code refactoring that later grew beyond its original design. 24 | */ 25 | 26 | const urlParser = require('url'); 27 | const ProjectUtilities = require('./ProjectUtilities'); 28 | const QuickLogger = require('./QuickLogger'); 29 | 30 | var allowAnyReferrer; 31 | var matchAllReferrer; 32 | var useHTTPS = true; 33 | 34 | 35 | /** 36 | * Internal helper function to log a message at the assigned log level. 37 | * @param message {string} 38 | */ 39 | function logMessage(message) { 40 | if (message != null && message.length > 0) { 41 | QuickLogger.logInfoEvent(message); 42 | } 43 | } 44 | 45 | /** 46 | * Given a configuration object takes the values we can use and copy them to local variables. 47 | * @param configuration {object} 48 | */ 49 | module.exports.setConfiguration = function(configuration) { 50 | if (configuration.logLevel !== undefined) { 51 | logLevel = configuration.logLevel; 52 | } else { 53 | logLevel = 0; 54 | } 55 | if (configuration.logFunction !== undefined) { 56 | logFunction = configuration.logFunction; 57 | } else { 58 | logFunction = null; 59 | } 60 | if (configuration.useHTTPS !== undefined) { 61 | useHTTPS = configuration.useHTTPS; 62 | } else { 63 | useHTTPS = false; 64 | } 65 | if (configuration.matchAllReferrer !== undefined) { 66 | matchAllReferrer = configuration.matchAllReferrer; 67 | } else { 68 | matchAllReferrer = true; 69 | } 70 | if (configuration.allowAnyReferrer !== undefined) { 71 | allowAnyReferrer = configuration.allowAnyReferrer; 72 | } else { 73 | allowAnyReferrer = false; 74 | } 75 | }; 76 | 77 | /** 78 | * Parse the URL and produce an object with all the parts of the URL. Returns an object in the form 79 | * { 80 | * url: = the original URL 81 | * protocol: = protocol extracted from URL or * 82 | * hostname: = host extracted from URL or * 83 | * port: = port number extracted from URL or * 84 | * pathname: = path extracted from URL, starting with / and default will be / 85 | * query: = query string extracted from URL without the ?, or "" 86 | * } 87 | * @param url {string} 88 | * @returns {object} 89 | */ 90 | module.exports.parseAndFixURLParts = function(url) { 91 | var urlParts = urlParser.parse(url), 92 | delimiter; 93 | 94 | if (urlParts != null) { 95 | if (urlParts.protocol == null || urlParts.protocol == '') { 96 | urlParts.protocol = '*'; 97 | } else { 98 | delimiter = urlParts.protocol.indexOf(':'); 99 | if (delimiter > 0) { 100 | urlParts.protocol = urlParts.protocol.substr(0, delimiter); 101 | } 102 | } 103 | if (urlParts.hostname == null || urlParts.hostname == '') { 104 | if (urlParts.pathname == null || urlParts.pathname == '') { 105 | urlParts.pathname = '*'; 106 | } 107 | urlParts.hostname = urlParts.pathname; 108 | urlParts.pathname = '*'; 109 | delimiter = urlParts.hostname.indexOf('//'); 110 | if (delimiter >= 0) { 111 | urlParts.hostname = urlParts.hostname.substr(delimiter + 2); 112 | } 113 | while (urlParts.hostname.charAt(0) == '/') { 114 | urlParts.hostname = urlParts.hostname.substr(1); 115 | } 116 | delimiter = urlParts.hostname.indexOf('/'); 117 | if (delimiter > 0) { 118 | urlParts.pathname = urlParts.hostname.substr(delimiter); 119 | urlParts.hostname = urlParts.hostname.substr(0, delimiter); 120 | } 121 | } 122 | if (urlParts.port == null || urlParts.port == '') { 123 | urlParts.port = '*'; 124 | } 125 | if (urlParts.pathname == null || urlParts.pathname == '' || urlParts.pathname == '/*' || urlParts.pathname == '/') { 126 | urlParts.pathname = '*'; 127 | } 128 | urlParts.path = urlParts.pathname; 129 | urlParts.host = urlParts.hostname; 130 | } 131 | return urlParts; 132 | }; 133 | 134 | /** 135 | * Return a copied object 136 | * @param urlParts 137 | * @returns {*} 138 | */ 139 | module.exports.copyURLParts = function(urlParts) { 140 | if (urlParts != null) { 141 | var copiedUrlParts = urlParser.parse(urlParts.href); 142 | copiedUrlParts.protocol = urlParts.protocol; 143 | copiedUrlParts.host = urlParts.host; 144 | copiedUrlParts.hostname = urlParts.hostname; 145 | copiedUrlParts.pathname = urlParts.pathname; 146 | copiedUrlParts.path = urlParts.path; 147 | copiedUrlParts.port = urlParts.port; 148 | copiedUrlParts.query = urlParts.query; 149 | return copiedUrlParts; 150 | } else { 151 | return null; 152 | } 153 | }; 154 | 155 | /** 156 | * Break apart the full URL request and determine its constituent parts. This is a bit non-standard due 157 | * to the special case handling of ? and &. Examples: 158 | * /proxy/http/host.domain.tld/path/path?q=1&t=2 159 | * /proxy?http://host.domain.tld/path/path?q=1&t=2 160 | * /proxy&http://host.domain.tld/path/path?q=1&t=2 161 | * Returns: object: 162 | * listenPath: the base URL pattern we are to be listening for 163 | * proxyPath: the URI/URL pattern to proxy 164 | * protocol: if part of the URI pattern we extract it 165 | * query: part after a ? in the URL in case we need to pass that along 166 | * @param url {string} url we want to parse 167 | * @param listenUriList {Array} list of Uri's we are listening for. e.g. '/proxy'. 168 | * @returns {{listenPath: string, proxyPath: string, query: string, protocol: string}} 169 | */ 170 | module.exports.parseURLRequest = function(url, listenUriList) { 171 | var result = { 172 | listenPath: '', 173 | proxyPath: '', 174 | query: '', 175 | protocol: '*' 176 | }, 177 | charDelimiter, 178 | lookFor, 179 | i, 180 | isMatch = false; 181 | 182 | url = decodeURI(url); 183 | if (url != null && url.length > 0) { 184 | // brute force take anything after http or https 185 | // TODO: regex pattern is '[\/|\?|&]http[s]?[:]?\/' we should consider that vs. the brute force method here. 186 | lookFor = '/https/'; 187 | charDelimiter = url.indexOf(lookFor); 188 | if (charDelimiter >= 0) { 189 | isMatch = true; 190 | result.protocol = 'https'; 191 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 192 | url = url.substr(0, charDelimiter); 193 | } else { 194 | lookFor = '?https://'; 195 | charDelimiter = url.indexOf(lookFor); 196 | if (charDelimiter >= 0) { 197 | isMatch = true; 198 | result.protocol = 'https'; 199 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 200 | url = url.substr(0, charDelimiter); 201 | } else { 202 | lookFor = '&https://'; 203 | charDelimiter = url.indexOf(lookFor); 204 | if (charDelimiter >= 0) { 205 | isMatch = true; 206 | result.protocol = 'https'; 207 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 208 | url = url.substr(0, charDelimiter); 209 | } 210 | } 211 | } 212 | if (! isMatch) { 213 | lookFor = '/http/'; 214 | charDelimiter = url.indexOf(lookFor); 215 | if (charDelimiter >= 0) { 216 | isMatch = true; 217 | result.protocol = 'http'; 218 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 219 | url = url.substr(0, charDelimiter); 220 | } else { 221 | lookFor = '?http://'; 222 | charDelimiter = url.indexOf(lookFor); 223 | if (charDelimiter >= 0) { 224 | isMatch = true; 225 | result.protocol = 'http'; 226 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 227 | url = url.substr(0, charDelimiter); 228 | } else { 229 | lookFor = '&http://'; 230 | charDelimiter = url.indexOf(lookFor); 231 | if (charDelimiter >= 0) { 232 | isMatch = true; 233 | result.protocol = 'http'; 234 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 235 | url = url.substr(0, charDelimiter); 236 | } 237 | } 238 | } 239 | } 240 | if (! isMatch) { 241 | // possible there was a wildcard protocol, now how do we figure that out? 242 | lookFor = '/*/'; 243 | charDelimiter = url.indexOf(lookFor); 244 | if (charDelimiter >= 0) { 245 | result.protocol = '*'; 246 | result.proxyPath = url.substr(charDelimiter + lookFor.length - 1); 247 | url = url.substr(0, charDelimiter); 248 | } else { 249 | // TODO: if just ? or & how do we know if a path or a query string? 250 | for (i = 0; i < listenUriList.length; i ++) { 251 | lookFor = listenUriList[i]; 252 | if (lookFor.charAt(lookFor.length) != '/') { 253 | lookFor += '/'; 254 | } 255 | charDelimiter = url.indexOf(lookFor); 256 | if (charDelimiter == 0) { 257 | isMatch = true; 258 | result.protocol = '*'; // TODO: can protocol be something other than http[?]://? 259 | result.proxyPath = url.substr(charDelimiter + lookFor.length); 260 | url = listenUriList[i]; 261 | break; 262 | } 263 | } 264 | } 265 | } 266 | result.listenPath = url; 267 | lookFor = '?'; // take anything after a ? as the query string 268 | if (result.proxyPath == '') { 269 | result.proxyPath = url; 270 | } 271 | charDelimiter = result.proxyPath.indexOf(lookFor); 272 | if (charDelimiter >= 0) { 273 | result.query = result.proxyPath.substr(charDelimiter + 1); 274 | result.proxyPath = result.proxyPath.substr(0, charDelimiter); 275 | } 276 | } 277 | return result; 278 | }; 279 | 280 | /** 281 | * Combine two components of a URL or file path to make sure they are separated by one and only one /. 282 | * @param firstPart 283 | * @param secondPart 284 | * @returns {string} firstPart + '/' + secondPart. 285 | */ 286 | module.exports.combinePath = function(firstPart, secondPart) { 287 | if (firstPart != null && firstPart.length > 0) { 288 | while (firstPart.charAt(firstPart.length - 1) == '/') { 289 | firstPart = firstPart.substr(0, firstPart.length - 1); 290 | } 291 | } else { 292 | firstPart = ''; 293 | } 294 | if (secondPart != null && secondPart.length > 0) { 295 | while (secondPart.charAt(0) == '/') { 296 | secondPart = secondPart.substr(1); 297 | } 298 | } else { 299 | secondPart = ''; 300 | } 301 | return firstPart + '/' + secondPart; 302 | }; 303 | 304 | /** 305 | * Look at two domains and see if they match by taking into account any * wildcards. 306 | * @param wildCardDomain 307 | * @param referrer {string} 308 | * @returns {boolean} true if domains match 309 | */ 310 | module.exports.testDomainsMatch = function(wildCardDomain, referrer) { 311 | var isMatch = true, 312 | i, 313 | domainParts, 314 | referrerParts; 315 | 316 | domainParts = wildCardDomain.split('.'); 317 | referrerParts = referrer.split('.'); 318 | if (domainParts.length == referrerParts.length) { 319 | for (i = 0; i < domainParts.length; i ++) { 320 | if (domainParts[i] != '*' && domainParts[i] != referrerParts[i]) { 321 | isMatch = false; 322 | break; 323 | } 324 | } 325 | } else { 326 | isMatch = false; 327 | } 328 | return isMatch; 329 | }; 330 | 331 | /** 332 | * Determine if two protocols match, accounting for wildcard in the first but not in the second. 333 | * Should also ignore :// (TODO?) 334 | * @param sourceProtocol 335 | * @param targetProtocol 336 | * @returns {boolean} 337 | */ 338 | module.exports.testProtocolsMatch = function(sourceProtocol, targetProtocol) { 339 | return sourceProtocol == '*' || sourceProtocol == targetProtocol; 340 | }; 341 | 342 | /** 343 | * Compare two URL parts objects to determine if they match. Matching takes into account partial paths and 344 | * wildcards. 345 | * @param urlPartsSource 346 | * @param urlPartsTarget 347 | * @returns {boolean} returns true if the two objects are considered a match. 348 | */ 349 | module.exports.parsedUrlPartsMatch = function(urlPartsSource, urlPartsTarget) { 350 | var isMatch = false, 351 | errorMessage = ''; 352 | 353 | if (this.testDomainsMatch(urlPartsSource.hostname, urlPartsTarget.hostname)) { 354 | if (urlPartsSource.protocol == "*" || urlPartsTarget.protocol == "*" || urlPartsSource.protocol == urlPartsTarget.protocol) { 355 | if (urlPartsSource.matchAll) { 356 | isMatch = urlPartsTarget.path == '*' || urlPartsTarget.path == urlPartsSource.path; 357 | if (isMatch) { 358 | errorMessage = "parsedUrlPartsMatch path " + urlPartsSource.path + " " + urlPartsTarget.path + " match."; 359 | } else { 360 | errorMessage = "parsedUrlPartsMatch path " + urlPartsSource.path + " " + urlPartsTarget.path + " don't match."; 361 | } 362 | } else { 363 | isMatch = urlPartsTarget.path == '*' || ProjectUtilities.startsWith(urlPartsTarget.path, urlPartsSource.path); 364 | if (isMatch) { 365 | errorMessage = "parsedUrlPartsMatch path " + urlPartsSource.path + " " + urlPartsTarget.path + " match."; 366 | } else { 367 | errorMessage = "parsedUrlPartsMatch path " + urlPartsSource.path + " " + urlPartsTarget.path + " don't match."; 368 | } 369 | } 370 | } else { 371 | errorMessage = "parsedUrlPartsMatch protocol " + urlPartsSource.protocol + " " + urlPartsTarget.protocol + " don't match."; 372 | } 373 | } else { 374 | errorMessage = "parsedUrlPartsMatch domains " + urlPartsSource.hostname + " " + urlPartsTarget.hostname + " don't match."; 375 | } 376 | if (errorMessage != '') { 377 | logMessage(errorMessage); 378 | } 379 | return isMatch; 380 | }; 381 | 382 | 383 | /** 384 | * Determine if the referrer matches one of the configured allowed referrers. If it does, return the string 385 | * we store in our table as the look-up key for this referrer. If no match, return null. 386 | * @param referrer {string} referer (sic) received from http request 387 | * @param allowedReferrers {Array} array of parsed referrer URL objects to match referrer against. 388 | * @returns {string} the referrer we want to use when referring to this referrer. 389 | */ 390 | module.exports.validatedReferrerFromReferrer = function(referrer, allowedReferrers) { 391 | var validReferrer = null, 392 | i, 393 | noMatchReason = '', 394 | referrerToCheckParts, 395 | referrerParts; 396 | 397 | if (allowAnyReferrer) { 398 | validReferrer = '*'; 399 | } else if (referrer != undefined && referrer != null && referrer.length > 0) { 400 | referrerParts = this.parseAndFixURLParts(referrer.toLowerCase().trim()); 401 | if (referrerParts.hostname == null) { 402 | referrerParts.hostname = '*'; 403 | } 404 | for (i = 0; i < allowedReferrers.length; i ++) { 405 | referrerToCheckParts = allowedReferrers[i]; 406 | if (this.testProtocolsMatch(referrerToCheckParts.protocol, referrerParts.protocol)) { 407 | if (referrerToCheckParts.hostname == '*' || this.testDomainsMatch(referrerToCheckParts.hostname, referrerParts.hostname)) { 408 | if (referrerToCheckParts.path == '*' || referrerToCheckParts.path == referrerParts.path) { 409 | validReferrer = referrerToCheckParts.referrer; 410 | break; 411 | } else if (! matchAllReferrer && ProjectUtilities.startsWith(referrerParts.path, referrerToCheckParts.path)) { 412 | validReferrer = referrerToCheckParts.referrer; 413 | break; 414 | } else { 415 | noMatchReason = 'referrer path ' + referrerParts.path + ' does not match ' + referrerToCheckParts.path; 416 | } 417 | } else { 418 | noMatchReason = 'referrer hostname ' + referrerParts.hostname + ' does not match ' + referrerToCheckParts.hostname; 419 | } 420 | } else { 421 | noMatchReason = 'referrer protocol ' + referrerParts.protocol + ' does not match ' + referrerToCheckParts.protocol; 422 | } 423 | } 424 | } else { 425 | noMatchReason = 'referrer could not be determined and referrer match is required.'; 426 | } 427 | if (noMatchReason != '') { 428 | logMessage('validatedReferrerFromReferrer no match because ' + noMatchReason); 429 | } 430 | return validReferrer; 431 | }; 432 | 433 | /** 434 | * Try to determine the protocol to use given the parameters. This does a best-guess by prioritizing the 435 | * serverURLInfo definition, then the request that came in, and then use what the referrer came in with. 436 | * Finally if none of that produce a usable protocol we use the configuration default setting. 437 | * @param referrer {string} the url of the referrer 438 | * @param urlRequestedParts 439 | * @param serverURLInfo 440 | * @returns {string} the protocol we should use for this request. 441 | */ 442 | module.exports.getBestMatchProtocol = function(referrer, urlRequestedParts, serverURLInfo) { 443 | var protocol = null, 444 | referrerParts = this.parseAndFixURLParts(referrer); 445 | 446 | if (serverURLInfo.protocol == '*') { 447 | if (urlRequestedParts.protocol == '*') { 448 | if (referrerParts.protocol !== undefined && referrerParts.protocol != '*') { 449 | protocol = referrerParts.protocol; 450 | } 451 | } else { 452 | protocol = urlRequestedParts.protocol; 453 | } 454 | } else { 455 | protocol = serverURLInfo.protocol; 456 | } 457 | if (protocol === undefined || protocol == null || protocol == '*') { 458 | protocol = useHTTPS ? 'https' : 'http'; 459 | } 460 | return protocol; 461 | }; 462 | 463 | /** 464 | * Try to determine the port to use given the parameters. This does a best-guess by prioritizing the 465 | * serverURLInfo definition, then the request that came in, and then use what the referrer came in with. 466 | * Finally if none of that produce a usable port we use the configuration default setting. 467 | * @param referrer {string} the url of the referrer 468 | * @param urlRequestedParts 469 | * @param serverURLInfo 470 | * @returns {number} the port we should use for this request. 471 | */ 472 | module.exports.getBestMatchPort = function(referrer, urlRequestedParts, serverURLInfo) { 473 | var port = 80, 474 | referrerParts = this.parseAndFixURLParts(referrer); 475 | 476 | if (serverURLInfo.port == '*') { 477 | if (urlRequestedParts.port == '*') { 478 | if (referrerParts.port !== undefined && referrerParts.port != '*') { 479 | port = referrerParts.port; 480 | } 481 | } else { 482 | port = urlRequestedParts.port; 483 | } 484 | } else { 485 | port = serverURLInfo.port; 486 | } 487 | if (port === undefined || port == null || port == '*') { 488 | port = 80; 489 | } 490 | return port; 491 | }; 492 | 493 | /** 494 | * When we break apart full URLs acting as referrers into their constituent parts (e.g. using url.parse()) this function 495 | * will take that url object and return a single string representing the original referrer. 496 | * @param urlParts 497 | * @returns {*} 498 | */ 499 | module.exports.fullReferrerURLFromParts = function(urlParts) { 500 | if (urlParts != null) { 501 | if (urlParts.protocol == '*' && urlParts.hostname == '*' && urlParts.path == '*') { 502 | return '*'; 503 | } else { 504 | return urlParts.protocol + '://' + urlParts.hostname + (urlParts.path.charAt(0) == '/' ? urlParts.path : '/' + urlParts.path); 505 | } 506 | } else { 507 | return '*'; 508 | } 509 | }; 510 | 511 | /** 512 | * Given an object representing our URL parts structure this function will return a URL string 513 | * combining the constituent parts. This function will make some assumptions based on the data: 514 | * - if protocol is * it will use https based on the global useHTTPS configuration setting. 515 | * - if port is not null, *, or 80 it will add port: to the url, otherwise it ignores port. 516 | * - if path is * it uses / instead, however if path ends with * it will remain. 517 | * - if there is a query string it will be appended to the end of the path. 518 | * @param urlParts {object} the url parts object we recombine into a full URL. 519 | * @param overRideParameters {string} optional parameter. If provided, this overrides any parameters specified in the urlParts. 520 | * @returns {string} 521 | */ 522 | module.exports.buildFullURLFromParts = function(urlParts, overRideParameters) { 523 | var url; 524 | url = urlParts.protocol == '*' ? (useHTTPS ? 'https' : 'http') : urlParts.protocol; 525 | url += '://'; 526 | url += urlParts.hostname; 527 | if (urlParts.port != '*' && urlParts.port != 80) { 528 | url += ':' + urlParts.port; 529 | } 530 | if (urlParts.pathname === undefined || urlParts.pathname == null || urlParts.pathname == '*' || urlParts.pathname.trim().length == 0) { 531 | url += '/'; 532 | } else { 533 | url += urlParts.pathname; 534 | } 535 | if (overRideParameters !== undefined && overRideParameters != null && overRideParameters != '') { 536 | if (typeof overRideParameters === "string") { 537 | if (overRideParameters.charAt(0) == '?') { 538 | url += overRideParameters; 539 | } else { 540 | url += '?' + overRideParameters; 541 | } 542 | } 543 | } else if (urlParts.query !== undefined && urlParts.query != null && urlParts.query.length > 0) { 544 | if (urlParts.query.charAt(0) == '?') { 545 | url += urlParts.query; 546 | } else { 547 | url += '?' + urlParts.query; 548 | } 549 | } 550 | return url; 551 | }; 552 | 553 | /** 554 | * Determine if the URL parts structure is valid enough to use as a URL. 555 | * @param urlParts 556 | * @returns {boolean} 557 | */ 558 | module.exports.isValidURL = function(urlParts) { 559 | return urlParts.protocol !== undefined && urlParts.protocol != null && urlParts.protocol.trim().length > 0 560 | && urlParts.hostname !== undefined && urlParts.hostname != null && urlParts.hostname.trim().length > 0 561 | && urlParts.pathname !== undefined && urlParts.pathname != null; 562 | }; 563 | 564 | /** 565 | * Our parsing technique breaks resource requests into their individual pieces to make matching easier at 566 | * runtime (we don't have to parse everything on every request) but that requires us to reassemble a valid 567 | * resource request from those pieces plus any additional information that came in with the request. 568 | * @param referrer {string} the validated referrer we are tracking (can be "*"). 569 | * @param urlRequestedParts {object} request parsed from parseURLRequest() 570 | * @param serverURLInfo {object} the serverUrl config that matches this request 571 | * @returns {string} a full URL to use to complete the request. 572 | */ 573 | module.exports.buildURLFromReferrerRequestAndInfo = function(referrer, urlRequestedParts, serverURLInfo) { 574 | var proxyRequest = serverURLInfo.url, 575 | delimiter; 576 | 577 | // make sure the url has a protocol. We allow url definitions to use no protocol or '*' or '*://' to mean 578 | // any protocol. 579 | delimiter = proxyRequest.indexOf('://'); 580 | if (delimiter < 0) { 581 | // no :// 582 | proxyRequest = this.getBestMatchProtocol(referrer, urlRequestedParts, serverURLInfo) + '://' + proxyRequest; 583 | } else if (delimiter == 0) { 584 | // has just :// 585 | proxyRequest = this.getBestMatchProtocol(referrer, urlRequestedParts, serverURLInfo) + proxyRequest; 586 | } else if (delimiter == 1) { 587 | // has just ?://, check if * 588 | if (proxyRequest.charAt(0) == '*') { 589 | proxyRequest = this.getBestMatchProtocol(referrer, urlRequestedParts, serverURLInfo) + proxyRequest.substr(1); 590 | } 591 | //} else { 592 | // already has some protocol so just leave it alone 593 | } 594 | // add the query string to the end of the url 595 | if (serverURLInfo.query != null && serverURLInfo.query != '') { 596 | proxyRequest += '?' + serverURLInfo.query; 597 | } else if (urlRequestedParts.query != null && urlRequestedParts.query != '') { 598 | proxyRequest += '?' + urlRequestedParts.query; 599 | } 600 | return proxyRequest; 601 | }; 602 | 603 | /** 604 | * Combine the parameters from the request and the server url configuration where the parameters 605 | * specified in the request will override any defined in the configuration, otherwise any parameters 606 | * specified in either are combined. 607 | * @param request - the node http/http request. It may also include parameters for the request. 608 | * @param urlParts - the parsed url parts of the request 609 | * @param serverURLInfo - the server url configuration matching the request. It may have its own parameters. 610 | * @param requestOverridesConfig - true to prioritize request, false to prioritize config. 611 | * @returns {object} - recombined parameters. 612 | */ 613 | module.exports.combineParameters = function(request, urlParts, serverURLInfo, requestOverridesConfig) { 614 | var configuredParameters = {}, 615 | requestParameters = null, 616 | key; 617 | 618 | if (typeof requestOverridesConfig === 'undefined') { 619 | requestOverridesConfig = true; 620 | } 621 | if (requestOverridesConfig) { 622 | if (serverURLInfo.query !== undefined && serverURLInfo.query != null && serverURLInfo.query.length > 0) { 623 | configuredParameters = ProjectUtilities.queryStringToObject(serverURLInfo.query); 624 | } 625 | } 626 | if (request.method == 'GET') { 627 | if (urlParts.query !== undefined && urlParts.query != null && urlParts.query.length > 0) { 628 | requestParameters = urlParts.query; 629 | } 630 | } else if (request.method == 'POST') { 631 | // TODO: If POST then where are the post params? 632 | requestParameters = request.query; 633 | } 634 | if (requestParameters != null) { 635 | requestParameters = ProjectUtilities.queryStringToObject(requestParameters); 636 | for (key in requestParameters) { 637 | if (requestParameters.hasOwnProperty(key)) { 638 | configuredParameters[key] = requestParameters[key]; 639 | } 640 | } 641 | } 642 | if ( ! requestOverridesConfig && serverURLInfo.query != null) { 643 | requestParameters = ProjectUtilities.queryStringToObject(serverURLInfo.query); 644 | for (key in requestParameters) { 645 | if (requestParameters.hasOwnProperty(key)) { 646 | configuredParameters[key] = requestParameters[key]; 647 | } 648 | } 649 | } 650 | return configuredParameters; 651 | }; 652 | -------------------------------------------------------------------------------- /bin/proxy.js: -------------------------------------------------------------------------------- 1 | /** 2 | * A proxy server built with node.js and tailored to the ArcGIS platform. See README for description 3 | * of functionality and configuration. 4 | * 5 | * John's to-do list: 6 | * * Set configuration.testMode when there is a node command line parameter "test" 7 | * * test hostRedirect test with http://local.arcgis.com:3333/proxy/geo.arcgis.com/ArcGIS/rest/info/ 8 | * - redirect host name but no path, uses path of request 9 | * - redirect host name with path, uses path of serverUrl and ignores request 10 | * * Resolving query parameters, combining query parameters from serverURL and request. Always params of 11 | * serverUrl override anything provided by request. 12 | * * http://route.arcgis.com/arcgis/rest/services/World/ClosestFacility/NAServer/ClosestFacility_World/solveClosestFacility => http://local.arcgis.com:3333/proxy/http/route.arcgis.com/arcgis/rest/services/World/ClosestFacility/NAServer/ClosestFacility_World/solveClosestFacility?f=json 13 | * 14 | * * transform application/vnd.ogc.wms_xml to text/xml 15 | * * adding token to request without a token 16 | * * replace token to a request that has a token but we dont want to use it 17 | * * If proxied request fails due to 499/498, catching that and retry with credentials or refresh token 18 | * * username/password 19 | * * tokenServiceUri 20 | * * oauth, clientId, clientSecret, oauthEndpoint, accessToken 21 | * * POST 22 | * * FILES 23 | * * Clean config files of test data or make a separate version for testing 24 | */ 25 | 26 | const proxyVersion = "0.1.5"; 27 | const http = require('http'); 28 | const https = require('https'); 29 | const httpProxy = require('http-proxy'); 30 | const connector = require('connect'); 31 | const bodyParser = require('body-parser'); 32 | const fs = require('fs'); 33 | const path = require('path'); 34 | const urlParser = require('url'); 35 | const BufferHelper = require('bufferhelper'); 36 | const OS = require('os'); 37 | const zlib = require('zlib'); 38 | const nodeStatic = require('node-static'); 39 | const RateMeter = require('./RateMeter'); 40 | const ProjectUtilities = require('./ProjectUtilities'); 41 | const QuickLogger = require('./QuickLogger'); 42 | const UrlFlexParser = require('./UrlFlexParser'); 43 | const Configuration = require('./Configuration'); 44 | 45 | const defaultOAuthServiceEndPoint = 'https://www.arcgis.com/sharing/oauth2'; 46 | const defaultTokenEndPoint = '/sharing/generateToken/'; 47 | const defaultAGOLRestPath = '/rest/'; 48 | const defaultAGOLRestPathStart = '/rest/info'; 49 | const defaultAGOLSharePath = '/sharing/'; 50 | const defaultAGOLSharePathStart = '/sharing/rest/info'; 51 | const defaultPortalServicePath = '/arcgis/rest/info'; 52 | 53 | var configuration = Configuration.configuration; 54 | var httpServer; 55 | var proxyServer; 56 | var staticFileServer = null; 57 | var rateMeter = null; 58 | var serverStartTime = null; 59 | var attemptedRequests = 0; 60 | var validProcessedRequests = 0; 61 | var errorProcessedRequests = 0; 62 | var configurationComplete = false; 63 | var waitingToRunIntegrationTests = false; 64 | 65 | 66 | /** 67 | * Look up the urlRequested in the serverUrls configuration and return the matching object. 68 | * @param urlRequestedParts the object returns from parseURLRequest() 69 | * @returns {object} null if no match, otherwise the parsed and corrected URL scheme to proxy to. 70 | */ 71 | function getServerUrlInfo (urlRequestedParts) { 72 | var i, 73 | urlParts, 74 | serverUrls, 75 | serverUrl, 76 | serverUrlMatched = null; 77 | 78 | if (urlRequestedParts.proxyPath == null || urlRequestedParts.proxyPath == '') { 79 | return serverUrlMatched; 80 | } 81 | // clean and normalize the path we receive so it looks like a standard URL pattern. This usually means 82 | // translating /host.domain.tld/path/path into something else. 83 | urlParts = UrlFlexParser.parseAndFixURLParts(urlRequestedParts.proxyPath); 84 | serverUrls = configuration.serverUrls; 85 | urlParts.protocol = urlRequestedParts.protocol; 86 | if (urlParts.protocol.charAt(urlParts.protocol.length - 1) == ':') { 87 | urlParts.protocol = urlParts.protocol.substr(0, urlParts.protocol.length - 1); 88 | } 89 | if (urlParts.path == null || urlParts.path == '') { 90 | urlParts.path = urlRequestedParts.proxyPath; 91 | } 92 | // if we don't parse a host name then we are going to assume the host name is encoded in the path, 93 | // then take that piece out of the path 94 | if (urlParts.hostname == null || urlParts.hostname == '') { 95 | urlParts.hostname = urlParts.path; 96 | while (urlParts.hostname.length > 1 && urlParts.hostname.charAt(0) == '/') { 97 | urlParts.hostname = urlParts.hostname.substr(1); 98 | } 99 | i = urlParts.hostname.indexOf('/'); 100 | if (i >= 0) { 101 | urlParts.path = urlParts.hostname.substr(i); 102 | urlParts.hostname = urlParts.hostname.substr(0, i); 103 | } 104 | urlParts.path = urlParts.path.replace(urlParts.hostname, ''); 105 | } 106 | if (urlParts.port == null || urlParts.port == '') { 107 | urlParts.port = '*'; 108 | } 109 | if (urlParts.query == null) { 110 | urlParts.query = urlRequestedParts.query; 111 | } 112 | for (i = 0; i < serverUrls.length; i ++) { 113 | serverUrl = serverUrls[i]; 114 | if (UrlFlexParser.parsedUrlPartsMatch(urlParts, serverUrl)) { // (matchAll && urlRequested == serverUrl.url) || ( ! matchAll && startsWith(serverUrl.url, urlRequested))) { 115 | QuickLogger.logInfoEvent('getServerUrlInfo ' + urlRequestedParts.proxyPath + ' matching ' + serverUrl.url); 116 | serverUrlMatched = serverUrl; 117 | break; 118 | } else { 119 | QuickLogger.logInfoEvent('getServerUrlInfo ' + urlRequestedParts.proxyPath + ' no match ' + serverUrl.url); 120 | } 121 | } 122 | return serverUrlMatched; 123 | } 124 | 125 | /** 126 | * Determine if the URI requested is one of the URIs we are supposed to be listening for in listenURI[]. 127 | * On the node.js server we can listen on any URI request we must specify the path we will accept. 128 | * If mustMatch is false then we will listen for anything! (Not sure if this is really useful.) 129 | * @param uri the uri that is being requested. Look this up in the serviceURLs table to make sure it is 130 | * something we are supposed to service. Matching is not case sensitive. 131 | * @returns {boolean} true if valid request. 132 | */ 133 | function isValidURLRequest (uri) { 134 | var isMatch = false, 135 | uriCheckFor = uri.toLowerCase(), 136 | i; 137 | 138 | if (configuration.mustMatch) { 139 | for (i = 0; i < configuration.listenURI.length; i ++) { 140 | if (uriCheckFor == configuration.listenURI[i].toLowerCase()) { 141 | isMatch = true; 142 | break; 143 | } 144 | } 145 | } 146 | return isMatch; 147 | } 148 | 149 | /** 150 | * Given an ArcGIS Online URL scheme, convert it to a path that would enable us to retrieve a valid 151 | * token endpoint URL, allowing us to ask for a token. This function is Promise based, it will return 152 | * a promise that will either resolve with the URL (since it must do a network query to get it) or 153 | * an error if we could not figure it out. 154 | * 155 | * NOTE: I took this logic from PHP resource proxy. 156 | * 157 | * @param url {string} a URL to transform. 158 | * @returns {Promise} The promise that will resolve with the new URL or an error. 159 | */ 160 | function getTokenEndpointFromURL(url) { 161 | var searchFor, 162 | index, 163 | tokenUrl, 164 | tokenUrlParts, 165 | method, 166 | parameters, 167 | tokenServiceUri = null; 168 | 169 | return new Promise(function(resolvePromise, rejectPromise) { 170 | // Convert request URL into a token endpoint URL. Look for '/rest/' in the requested URL (could be 'rest/services', 'rest/community'...) 171 | searchFor = defaultAGOLRestPath; 172 | index = url.indexOf(searchFor); 173 | if (index >= 0) { 174 | tokenUrl = url.substr(0, index) + defaultAGOLRestPathStart; 175 | } else { 176 | searchFor = defaultAGOLSharePath; 177 | index = url.indexOf(searchFor); 178 | if (index >= 0) { 179 | tokenUrl = url.substr(0, index) + defaultAGOLSharePathStart; 180 | } else { 181 | tokenUrl = url + defaultPortalServicePath; 182 | } 183 | } 184 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Transform url to token endpoint', {url: url, tokenEndpoint: tokenUrl})); 185 | parameters = { 186 | f: 'json' 187 | }; 188 | method = 'GET'; 189 | tokenUrlParts = UrlFlexParser.parseAndFixURLParts(tokenUrl); 190 | if (tokenUrlParts != null && tokenUrlParts.hostname != null && tokenUrlParts.pathname != null && tokenUrlParts.protocol != null) { 191 | httpRequestPromiseResponse(tokenUrlParts.hostname, tokenUrlParts.pathname, method, tokenUrlParts.protocol == 'https', parameters).then( 192 | function (serverResponse) { 193 | var authInfo = JSON.parse(serverResponse); 194 | if (authInfo != null && authInfo.authInfo !== undefined) { 195 | tokenServiceUri = authInfo.authInfo.tokenServicesUrl; 196 | } 197 | if (tokenServiceUri == null) { 198 | // If no tokenServicesUrl, try to find owningSystemUrl as token endpoint 199 | if (authInfo.owningSystemUrl !== undefined) { 200 | tokenServiceUri = authInfo.owningSystemUrl + defaultTokenEndPoint; 201 | } 202 | } 203 | if (tokenServiceUri != null) { 204 | resolvePromise(tokenServiceUri); 205 | } else { 206 | rejectPromise(new Error(Configuration.getStringTableEntry('Unable to transform to token endpoint', {url: url, tokenUrl: tokenUrl}))); 207 | } 208 | }, 209 | function (serverError) { 210 | rejectPromise(serverError); 211 | } 212 | ); 213 | } else { 214 | rejectPromise(new Error(Configuration.getStringTableEntry('Unable to transform to usable URL', {url: url, tokenUrl: tokenUrl}))); 215 | } 216 | }); 217 | } 218 | 219 | /** 220 | * If the server URL configuration is such that a username/password is used to get a token then 221 | * this function will attempt to contact the service with the user credentials and get a valid token 222 | * on behalf of that user. This function is very asynchronous it may make several network requests 223 | * before it gets the token. 224 | * @param referrer {string} who we want the service to think is making the request. 225 | * @param serverUrlInfo {object} our configuration object for this service. 226 | * @returns {Promise} A promise to resolve with the new token or reject with an error. 227 | */ 228 | function getNewTokenFromUserNamePasswordLogin(referrer, serverUrlInfo) { 229 | var parameters, 230 | method = 'POST', 231 | tokenServiceUriParts, 232 | token; 233 | 234 | return new Promise(function(resolvePromise, rejectPromise) { 235 | if (ProjectUtilities.isPropertySet(serverUrlInfo, 'username') && ProjectUtilities.isPropertySet(serverUrlInfo, 'password')) { 236 | parameters = { 237 | request: 'getToken', 238 | f: 'json', 239 | referer: referrer, 240 | expiration: 60, 241 | username: serverUrlInfo.username, 242 | password: serverUrlInfo.password 243 | }; 244 | getTokenEndpointFromURL(serverUrlInfo.url).then( 245 | function (tokenServiceUri) { 246 | tokenServiceUriParts = UrlFlexParser.parseAndFixURLParts(tokenServiceUri); 247 | httpRequestPromiseResponse(tokenServiceUriParts.host, tokenServiceUriParts.path, method, tokenServiceUriParts.protocol == 'https', parameters).then( 248 | function (responseBody) { 249 | token = ProjectUtilities.findTokenInString(responseBody, 'token'); 250 | resolvePromise(token); 251 | }, 252 | function (error) { 253 | rejectPromise(error); 254 | } 255 | ); 256 | }, 257 | function (error) { 258 | rejectPromise(error); 259 | } 260 | ); 261 | } else { 262 | rejectPromise(new Error(Configuration.getStringTableEntry('Username and password must be set', null))); 263 | } 264 | }); 265 | } 266 | 267 | /** 268 | * OAuth 2.0 mode authentication "App Login" - authenticating using oauth2Endpoint, clientId, and clientSecret specified 269 | * in configuration. Because this is an http request (or several) it is promise based. The token is passed to the 270 | * promise resolve function or an error is passed to the promise reject function. 271 | * @param serverURLInfo 272 | * @param requestUrl 273 | * @return {Promise} 274 | */ 275 | function performAppLogin(serverURLInfo) { 276 | if (serverURLInfo.oauth2Endpoint === undefined || serverURLInfo.oauth2Endpoint == null) { 277 | serverURLInfo.oauth2Endpoint = defaultOAuthServiceEndPoint; 278 | } 279 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Service is secured by', {oauth2Endpoint: serverURLInfo.oauth2Endpoint})); 280 | var tokenRequestPromise = new Promise(function(resolvePromise, rejectPromise) { 281 | var oauth2Endpoint = serverURLInfo.oauth2Endpoint + 'token', 282 | parameters = { 283 | client_id: serverURLInfo.clientId, 284 | client_secret: serverURLInfo.clientSecret, 285 | grant_type: 'client_credentials', 286 | f: 'json' 287 | }, 288 | oauthUrlParts = UrlFlexParser.parseAndFixURLParts(oauth2Endpoint), 289 | tokenResponse; 290 | 291 | httpRequestPromiseResponse(oauthUrlParts.hostname, oauthUrlParts.pathname, 'POST', oauthUrlParts.protocol == 'https', parameters).then( 292 | function(serverResponse) { 293 | tokenResponse = ProjectUtilities.findTokenInString(serverResponse, 'token'); 294 | if (tokenResponse.length > 0) { 295 | exchangePortalTokenForServerToken(tokenResponse, serverURLInfo).then(resolvePromise, rejectPromise); 296 | } else { 297 | rejectPromise(new Error(Configuration.getStringTableEntry('App login could not get a token', {response: serverResponse}))); 298 | } 299 | }, 300 | function(error) { 301 | rejectPromise(error); 302 | } 303 | ); 304 | }); 305 | return tokenRequestPromise; 306 | } 307 | 308 | /** 309 | * Decide which method to login the user. 310 | * @param serverURLInfo 311 | * @param requestUrl 312 | * @returns {Promise} Returns the JSON reply from the server which contains the token when it succeeds, or returns an error when it fails. 313 | */ 314 | function performUserLogin(serverURLInfo, requestUrl) { 315 | // standalone ArcGIS Server/ArcGIS Online token-based authentication 316 | var requestUrlParts = UrlFlexParser.parseAndFixURLParts(requestUrl), 317 | tokenResponse, 318 | parameters; 319 | 320 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Service requires user login', null)); 321 | var tokenRequestPromise = new Promise(function(resolvePromise, rejectPromise) { 322 | // if a request is already being made to generate a token, just let it go. 323 | if (requestUrlParts.pathname.toLowerCase().indexOf('/generatetoken') >= 0) { 324 | parameters = { 325 | request: 'getToken', 326 | f: 'json', 327 | referer: referrer, 328 | expiration: 60, 329 | username: serverURLInfo.username, 330 | password: serverURLInfo.password 331 | }; 332 | httpRequestPromiseResponse(requestUrlParts.hostname, requestUrlParts.pathname, 'POST', requestUrlParts.protocol == 'https', parameters).then( 333 | function(serverResponse) { 334 | tokenResponse = ProjectUtilities.findTokenInString(serverResponse, 'token'); 335 | if (tokenResponse.length > 0) { 336 | resolvePromise(tokenResponse); 337 | } else { 338 | rejectPromise(new Error(Configuration.getStringTableEntry('User login could not get a token', {response: serverResponse}))); 339 | } 340 | }, 341 | function(error) { 342 | rejectPromise(error); 343 | } 344 | ); 345 | } else { 346 | getNewTokenFromUserNamePasswordLogin(referrer, serverURLInfo).then(resolvePromise, rejectPromise); 347 | } 348 | }); 349 | return tokenRequestPromise; 350 | } 351 | 352 | /** 353 | * If the serverURLInfo specifies credentials to allow login, then attempt to login and authenticate with the service. 354 | * @param serverURLInfo {object} the server URL we are conversing with. 355 | * @param requestUrl {string} URL to the login service. 356 | * @returns {Promise} Resolves with the new token, or rejects with an error. 357 | */ 358 | function getNewTokenIfCredentialsAreSpecified(serverURLInfo, requestUrl) { 359 | return new Promise(function(resolvePromise, rejectPromise) { 360 | if (serverURLInfo.isAppLogin) { 361 | performAppLogin(serverURLInfo).then(resolvePromise, rejectPromise); 362 | } else if (serverURLInfo.isUserLogin) { 363 | performUserLogin(serverURLInfo, requestUrl).then(resolvePromise, rejectPromise); 364 | } else { 365 | rejectPromise(new Error(Configuration.getStringTableEntry('No method configured to authenticate', {url: serverURLInfo.url}))); 366 | } 367 | }); 368 | } 369 | 370 | /** 371 | * Use the token we have and exchange it for a long-lived server token. This is an AGOL specific workflow because of the path transformation. 372 | * @param portalToken {string} user's short-lived token. 373 | * @param serverURLInfo {object} the server URL we are conversing with. 374 | * @returns {Promise} The promise to return the token from the server, once it arrives. 375 | */ 376 | function exchangePortalTokenForServerToken(portalToken, serverURLInfo) { 377 | var responsePromise = new Promise(function(resolvePromise, rejectPromise) { 378 | var parameters = { 379 | token: portalToken, 380 | serverURL: serverURLInfo.url, 381 | f: 'json' 382 | }, 383 | uri = serverURLInfo.oauth2Endpoint.replace('/oauth2', '/generateToken'), 384 | oauthUrlParts = UrlFlexParser.parseAndFixURLParts(uri), 385 | host = oauthUrlParts.hostname, 386 | path = oauthUrlParts.path, 387 | tokenResponse; 388 | 389 | httpRequestPromiseResponse(host, path, 'POST', UrlFlexParser.getBestMatchProtocol('*', oauthUrlParts, serverURLInfo) == 'https', parameters).then( 390 | function(serverResponse) { 391 | tokenResponse = ProjectUtilities.findTokenInString(serverResponse, 'token'); 392 | if (tokenResponse.length > 0) { 393 | resolvePromise(tokenResponse); 394 | } else { 395 | rejectPromise(new Error(Configuration.getStringTableEntry('Could not get a token from server response', {response: serverResponse}))); 396 | } 397 | }, 398 | function(error) { 399 | rejectPromise(error); 400 | } 401 | ); 402 | }); 403 | return responsePromise; 404 | } 405 | 406 | /** 407 | * Issue an HTTP request and wait for a response from the server. An http request is an asynchronous request 408 | * using Node's http client. This is promised based, so the function returns a promise that will resolve with 409 | * the server response or fail with an error. 410 | * @param host {string} host server to contact www.sever.tld 411 | * @param path {string} path at server to request. Should begin with /. 412 | * @param method {string} GET|POST 413 | * @param useHttps {boolean} false uses http (80), true uses https (443) 414 | * @param parameters {object} request parameters object of key/values. Gets converted into a query string or post body depending on method. 415 | * @return {Promise} You get a promise that will resolve with the server response or fail with an error. 416 | */ 417 | function httpRequestPromiseResponse(host, path, method, useHttps, parameters) { 418 | var responsePromise = new Promise(function(resolvePromise, rejectPromise) { 419 | var httpRequestOptions = { 420 | hostname: host, 421 | path: path, 422 | method: method 423 | }, 424 | requestBody = ProjectUtilities.objectToQueryString(parameters), 425 | requestHeaders = {}, 426 | responseStatus = 0, 427 | responseBody = '', 428 | request; 429 | 430 | var handleServerResponse = function(response) { 431 | responseStatus = response.statusCode; 432 | if (responseStatus > 399) { 433 | rejectPromise(new Error('Error ' + responseStatus + ' on ' + host + path)); 434 | } else { 435 | response.on('data', function (chunk) { 436 | responseBody += chunk; 437 | }); 438 | response.on('end', function () { 439 | // if response looks like "{"error":{"code":498,"message":"Invalid token.","details":[]}}" then ERROR 440 | if (ProjectUtilities.startsWith(responseBody, '{"error":')) { 441 | responseStatus = ProjectUtilities.findNumberAfterTokenInString(responseBody, 'code'); 442 | rejectPromise(new Error('Error ' + responseStatus + ' on ' + host + path)); 443 | } else { 444 | resolvePromise(responseBody); 445 | } 446 | }); 447 | } 448 | }; 449 | 450 | if (method == 'POST') { 451 | requestHeaders['Content-Type'] = 'application/x-www-form-urlencoded'; 452 | requestHeaders['Content-Length'] = Buffer.byteLength(requestBody); 453 | } else if (method == 'GET' && requestBody.length > 0) { 454 | httpRequestOptions.path += '?' + requestBody; 455 | requestBody = ''; 456 | } 457 | httpRequestOptions.headers = requestHeaders; 458 | if (useHttps) { 459 | httpRequestOptions.protocol = 'https:'; 460 | request = https.request(httpRequestOptions, handleServerResponse); 461 | } else { 462 | httpRequestOptions.protocol = 'http:'; 463 | request = http.request(httpRequestOptions, handleServerResponse); 464 | } 465 | request.on('error', function(error) { 466 | rejectPromise(error); 467 | }); 468 | request.end(requestBody); 469 | }); 470 | return responsePromise; 471 | } 472 | 473 | /** 474 | * Calling this function means the request has passed all tests and we are going to contact the proxied service 475 | * and try to reply back to the caller with what it responds with. We will do any token refresh here if necessary. 476 | * @param urlRequestedParts - our object of the request components. 477 | * @param serverURLInfo - the matching server url configuration for this request. 478 | * @param referrer {string} the validated referrer we are tracking (can be "*"). 479 | * @param request - the http server request object. 480 | * @param response - the http server response object. 481 | * @return {boolean} true if the request was processed, false if we got an error. 482 | */ 483 | function processValidatedRequest (urlRequestedParts, serverURLInfo, referrer, request, response) { 484 | var statusCode = 200, 485 | statusMessage, 486 | proxyRequest, 487 | parsedHostRedirect, 488 | hostname, 489 | contentType, 490 | parametersCombined = '', 491 | parameters; 492 | 493 | if (serverURLInfo != null) { 494 | // TODO: GET - combine params from query with url request 495 | // TODO: POST, PUT - combine params from query or form with url request 496 | // TODO: test FILES 497 | // TODO: Handle Auth, oauth 498 | 499 | if (proxyServer != null) { 500 | serverURLInfo.lastRequest = new Date(); 501 | if (serverURLInfo.firstRequest == 0) { 502 | serverURLInfo.firstRequest = serverURLInfo.lastRequest; 503 | } 504 | serverURLInfo.totalRequests ++; 505 | 506 | // Combine query parameters of the current request with the configuration. 507 | parameters = UrlFlexParser.combineParameters(request, urlRequestedParts, serverURLInfo, serverURLInfo.parameterOverride); 508 | 509 | // if no token was provided in the request but one is in the configuration then use the configured token. 510 | if (ProjectUtilities.isPropertySet(serverURLInfo, 'accessToken')) { 511 | ProjectUtilities.addIfPropertyNotSet(parameters, 'token', serverURLInfo.accessToken); 512 | } else if (ProjectUtilities.isPropertySet(serverURLInfo, 'token')) { 513 | ProjectUtilities.addIfPropertyNotSet(parameters, 'token', serverURLInfo.token); 514 | } 515 | 516 | if ( ! ProjectUtilities.isEmptyObject(parameters)) { 517 | parametersCombined = ProjectUtilities.objectToQueryString(parameters); 518 | } 519 | if (serverURLInfo.isHostRedirect) { 520 | // Host Redirect means either replace the host and use path from the request when parsedHostRedirect has no path, 521 | // or redirect to host and path from parsedHostRedirect when there is a path, 522 | // then replace everything else received in the request (query, auth). 523 | parsedHostRedirect = serverURLInfo.parsedHostRedirect; 524 | hostname = parsedHostRedirect.hostname; 525 | proxyRequest = UrlFlexParser.buildFullURLFromParts(parsedHostRedirect, parametersCombined); 526 | } else { 527 | hostname = serverURLInfo.hostname; 528 | proxyRequest = UrlFlexParser.buildURLFromReferrerRequestAndInfo(referrer, urlRequestedParts, serverURLInfo); 529 | } 530 | 531 | // TODO: Combine parameters of the two requests, current request parameters override configured parameters 532 | // TODO: !!!! Fuck! can't do this here, as we do not have the body yet. 533 | if (request.method == 'POST' || request.method == 'PUT') { 534 | contentType = request.headers['Content-Type']; 535 | if (contentType.indexOf('x-www-form-urlencoded') >= 0) { 536 | // Its a post we have to read the entire body and extract the token form parameter if it's there. 537 | 538 | } else if (contentType.indexOf('multipart') >= 0) { 539 | // this sucks. A post with files means we need to parse the whole thing, find the form, hold the file(s) 540 | // in memory, find the form parameters, see if there is a token, then resend it all to the server. 541 | 542 | } 543 | } 544 | // Fix the request to transform it from our proxy server into a spoof of the matching request against the 545 | // proxied service 546 | request.url = proxyRequest; 547 | request.headers.host = hostname; 548 | 549 | 550 | 551 | // TODO: if a token based request we should check if the token we have is any good and if not generate a new token 552 | 553 | 554 | // TODO: Not really sure this worked if the proxy generates an error as we are not catching any error from the proxied service 555 | validProcessedRequests ++; 556 | QuickLogger.logInfoEvent("==> Issuing proxy request [" + request.method + "]" + request.url + " for " + proxyRequest); 557 | proxyServer.web(request, response, { 558 | target: proxyRequest, 559 | ignorePath: true 560 | }, proxyResponseError); 561 | } else { 562 | statusCode = 500; 563 | statusMessage = Configuration.getStringTableEntry('Internal error', null); 564 | sendErrorResponse(urlRequestedParts.proxyPath, response, statusCode, statusMessage); 565 | } 566 | } else { 567 | statusCode = 403; 568 | statusMessage = Configuration.getStringTableEntry('Proxy has not been set up for', {referrer: referrer, path: urlRequestedParts.listenPath}); 569 | if (QuickLogger.ifLogLevelGreaterOrEqual('INFO')) { 570 | statusMessage += Configuration.getStringTableEntry('Proxy has not been set up for extra', {path: urlRequestedParts.listenPath}); 571 | } 572 | sendErrorResponse(urlRequestedParts.proxyPath, response, statusCode, statusMessage); 573 | } 574 | return statusCode != 200; 575 | } 576 | 577 | /** 578 | * Respond to a ping request. A ping tells a client we are alive and gives out some status response. 579 | * @param referrer {string} - who asked for it. 580 | * @param response {object} - http response object. 581 | */ 582 | function sendPingResponse (referrer, response) { 583 | var statusCode = 200, 584 | responseBody = { 585 | "Proxy Version": proxyVersion, 586 | "Configuration File": "OK", 587 | "Log File": "OK", 588 | "referrer": referrer 589 | }; 590 | sendJSONResponse(response, statusCode, responseBody); 591 | validProcessedRequests ++; 592 | QuickLogger.logInfoEvent("Ping request from " + referrer); 593 | } 594 | 595 | /** 596 | * Respond to an echo request. Echo back exactly what the client sent us. 597 | * @param request {object} - http request object. 598 | * @param response {object} - http response object. 599 | */ 600 | function sendEchoResponse (referrer, request, response) { 601 | QuickLogger.logInfoEvent("Echo request from " + referrer); 602 | request.pipe(response); 603 | } 604 | 605 | /** 606 | * Respond to a server status request. 607 | * @param referrer - who asked for it. 608 | * @param response - http response object. 609 | */ 610 | function sendStatusResponse (referrer, response) { 611 | try { 612 | var timeNow = new Date(), 613 | i, 614 | serverUrl, 615 | serverUrls = configuration.serverUrls, 616 | responseObject = { 617 | "Proxy Version": proxyVersion, 618 | "Configuration File": "OK", 619 | "Log File": QuickLogger.getLogFileSize(), 620 | "Up-time": ProjectUtilities.formatMillisecondsToHHMMSS(timeNow - serverStartTime), 621 | "Requests": attemptedRequests, 622 | "Requests processed": validProcessedRequests + 1, // because this is a valid request that hasn't been counted yet 623 | "Requests rejected": errorProcessedRequests, 624 | "Referrers Allowed": configuration.allowedReferrers.map(function (allowedReferrer) { 625 | return allowedReferrer.referrer; 626 | }).join(', '), 627 | "Referrer": referrer, 628 | "URL Stats": [], 629 | "Rate Meter": [] 630 | }; 631 | for (i = 0; i < serverUrls.length; i ++) { 632 | serverUrl = serverUrls[i]; 633 | if ( ! serverUrl.useRateMeter) { 634 | responseObject['URL Stats'].push({ 635 | 'url': serverUrl.url.substring(0, 100) + (serverUrl.url.length > 100 ? '...' : ''), 636 | 'total': serverUrl.totalRequests, 637 | 'firstRequest': serverUrl.firstRequest == 0 ? '-' : serverUrl.firstRequest.toLocaleString(), 638 | 'lastRequest': serverUrl.lastRequest == 0 ? '-' : serverUrl.lastRequest.toLocaleString() 639 | }); 640 | } 641 | } 642 | if (rateMeter != null) { 643 | rateMeter.databaseDump().then(function (responseIsArrayOfTableRows) { 644 | responseObject['Rate Meter'] = responseIsArrayOfTableRows; 645 | reportHTMLStatusResponse(responseObject, response); 646 | }, function (databaseError) { 647 | responseObject.error = databaseError.toLocaleString(); 648 | reportHTMLStatusResponse(responseObject, response); 649 | }); 650 | } 651 | } catch (exception) { 652 | sendErrorResponse('status', response, 500, Configuration.getStringTableEntry('System error processing request', {message: exception.toLocaleString()})); 653 | } 654 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Status request from', {referrer: referrer})); 655 | } 656 | 657 | /** 658 | * Create an HTML dump of some valuable information regarding the current status of this proxy server. 659 | * @param responseObject {Object} we iterate this object as the information to report. 660 | * @param response {Object} the http response object to write to. 661 | */ 662 | function reportHTMLStatusResponse (responseObject, response) { 663 | var responseBody, 664 | key, 665 | value, 666 | row, 667 | rowKey, 668 | rowValue, 669 | tableRow, 670 | i, 671 | statusCode = 200; 672 | 673 | // TODO: extract HTML template into separate loadable files or string table 674 | responseBody = '\n\n\n\n' + Configuration.getStringTableEntry('Resource Proxy Status title', null) + '\n\n\n\n

' + Configuration.getStringTableEntry('Resource Proxy Status title', null) + '

'; 675 | for (key in responseObject) { 676 | if (responseObject.hasOwnProperty(key)) { 677 | value = responseObject[key]; 678 | if (value instanceof Array) { // Arrays get displayed as tables 679 | responseBody += '

' + key + ':

'; 680 | for (i = 0; i < value.length; i ++) { 681 | tableRow = ''; 682 | row = value[i]; 683 | for (rowKey in row) { 684 | if (row.hasOwnProperty(rowKey)) { 685 | if (i == 0) { 686 | responseBody += ''; 687 | } 688 | rowValue = row[rowKey]; 689 | tableRow += ''; 690 | } 691 | } 692 | responseBody += '' + tableRow; 693 | } 694 | if (value.length == 0) { 695 | responseBody += ''; 696 | } 697 | responseBody += '
' + rowKey + '' + rowValue + '
' + Configuration.getStringTableEntry('Empty row', null) + '
' 698 | } else { 699 | responseBody += '

' + key + ': ' + value + '

\n'; 700 | } 701 | } 702 | } 703 | responseBody += '\n\n'; 704 | response.writeHead(statusCode, { 705 | 'Content-Length': Buffer.byteLength(responseBody), 706 | 'Content-Type': 'text/html' 707 | }); 708 | response.write(responseBody); 709 | response.end(); 710 | } 711 | 712 | /** 713 | * Perform necessary node http-server functions to send reply in JSON format. 714 | * @param response - node http-server response object. 715 | * @param statusCode - a valid http status code (e.g. 200, 404, etc) 716 | * @param responseObject - a javascript object that is converted to JSON and sent back as the body. 717 | */ 718 | function sendJSONResponse (response, statusCode, responseObject) { 719 | var responseBody = JSON.stringify(responseObject); 720 | response.writeHead(statusCode, { 721 | 'Content-Length': Buffer.byteLength(responseBody), 722 | 'Content-Type': 'application/json' 723 | }); 724 | response.write(responseBody); 725 | response.end(); 726 | } 727 | 728 | /** 729 | * Reply with an error JSON object describing what may have gone wrong. This is used if there is 730 | * an error calling this proxy service, not for errors with the proxied service. 731 | * @param urlRequested the path that was requested. 732 | * @param response the response object so we can complete the response. 733 | * @param errorCode the error code we want to report to the caller. 734 | * @param errorMessage the error message we want to report to the caller. 735 | */ 736 | function sendErrorResponse (urlRequested, response, errorCode, errorMessage) { 737 | var responseBody = { 738 | error: { 739 | code: errorCode, 740 | details: errorMessage, 741 | message: errorMessage 742 | }, 743 | request: urlRequested 744 | }; 745 | sendJSONResponse(response, errorCode, responseBody); 746 | errorProcessedRequests ++; 747 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Request error with info', {error: errorMessage, code: errorCode, url: urlRequested})); 748 | } 749 | 750 | /** 751 | * Determine if this request is within the rate meter threshold. If it is we continue to processValidatedRequest(). 752 | * If it is not we generate the client reply here. Because the rate meter check is asynchronous and this function will 753 | * return before the check is complete it was just easier to deal with all subsequent processing here instead of 754 | * turning this into a promise. Something to reconsider for the next update. 755 | * @param referrer {string} the validated referrer we are tracking (can be "*"). 756 | * @param requestParts - the parsed URL that is being requested 757 | * @param serverURLInfo - the serverUrls object matching this request 758 | * @param request - the http request object, needed to pass on to processValidatedRequest or error response 759 | * @param response - the http response object, needed to pass on to processValidatedRequest or error response 760 | * @return {int} status code, but since this function is asynchronous the code is mostly meaningless 761 | * TODO: This function should return a promise that if resolved then calls processValidatedRequest 762 | */ 763 | function checkRateMeterThenProcessValidatedRequest(referrer, requestParts, serverURLInfo, request, response) { 764 | var statusCode = 200; 765 | if (rateMeter != null) { 766 | rateMeter.isUnderRate(referrer, serverURLInfo).then(function (isUnderCap) { 767 | if (isUnderCap) { 768 | processValidatedRequest(requestParts, serverURLInfo, referrer, request, response); 769 | } else { 770 | statusCode = 429; // TODO: or is it 402? or 420? 771 | QuickLogger.logWarnEvent(Configuration.getStringTableEntry('RateMeter blocking access to', {url: serverURLInfo.url, referrer: referrer})); 772 | sendErrorResponse(request.url, response, statusCode, Configuration.getStringTableEntry('Metered requests exceeded', null)); 773 | } 774 | }, function (error) { 775 | statusCode = 420; 776 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('RateMeter failed on', {url: serverURLInfo.url, referrer: referrer, error: error.toString()})); 777 | sendErrorResponse(request.url, response, statusCode, Configuration.getStringTableEntry('Metered resource status failed', null)); 778 | }); 779 | } else { 780 | statusCode = 500; 781 | } 782 | return statusCode; 783 | } 784 | 785 | /** 786 | * When the server receives a request we come here with the node http/https request object and 787 | * we fill in the response object. 788 | * @param request 789 | * @param response 790 | */ 791 | function processRequest(request, response) { 792 | var requestParts = UrlFlexParser.parseURLRequest(request.url, configuration.listenURI), 793 | serverURLInfo, 794 | referrer; 795 | 796 | attemptedRequests ++; 797 | if (requestParts != null) { 798 | referrer = request.headers['referer']; 799 | if (referrer == null || referrer.length < 1) { 800 | referrer = '*'; 801 | } else { 802 | referrer = referrer.toLowerCase().trim(); 803 | } 804 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('New request from', {referrer: referrer, path: requestParts.proxyPath})); 805 | referrer = UrlFlexParser.validatedReferrerFromReferrer(referrer, configuration.allowedReferrers); 806 | if (referrer != null) { 807 | if (requestParts.listenPath == configuration.localPingURL) { 808 | sendPingResponse(referrer, response); 809 | } else if (requestParts.proxyPath == configuration.localEchoURL) { 810 | sendEchoResponse(referrer, request, response); 811 | } else if (requestParts.listenPath == configuration.localStatusURL) { 812 | sendStatusResponse(referrer, response); 813 | } else { 814 | if (isValidURLRequest(requestParts.listenPath)) { 815 | serverURLInfo = getServerUrlInfo(requestParts); 816 | if (serverURLInfo != null) { 817 | request.serverUrlInfo = serverURLInfo; 818 | if (serverURLInfo.useRateMeter) { 819 | checkRateMeterThenProcessValidatedRequest(referrer, requestParts, serverURLInfo, request, response); 820 | } else { 821 | processValidatedRequest(requestParts, serverURLInfo, referrer, request, response); 822 | } 823 | } else if (! configuration.mustMatch) { 824 | // TODO: I think we should remove this feature 825 | // when mustMatch is false we accept absolutely anything (why, again, are we doing this?) so blindly forward the request on and cross your fingers someone actually thinks this is a good idea. 826 | serverURLInfo = UrlFlexParser.parseAndFixURLParts(requestParts.listenPath); 827 | serverURLInfo = { 828 | url: serverURLInfo.hostname + serverURLInfo.path, 829 | protocol: requestParts.protocol, 830 | hostname: serverURLInfo.hostname, 831 | path: serverURLInfo.path, 832 | port: serverURLInfo.port, 833 | rate: 0, 834 | rateLimitPeriod: 0 835 | }; 836 | processValidatedRequest(requestParts, serverURLInfo, referrer, request, response); 837 | } else { 838 | sendErrorResponse(request.url, response, 404, Configuration.getStringTableEntry('Resource not found', {url: request.url})); 839 | } 840 | } else { 841 | // try to serve a static resource. proxyServeFile will always send its own response, including 404 if resource not found. 842 | proxyServeFile(request, response); 843 | } 844 | } 845 | } else { 846 | sendErrorResponse(request.url, response, 403, Configuration.getStringTableEntry('Referrer not allowed', {referrer: referrer})); 847 | } 848 | } else { 849 | sendErrorResponse(request.url, response, 403, Configuration.getStringTableEntry('Invalid request 403', null)); 850 | } 851 | } 852 | 853 | /** 854 | * If the proxy target responds with an error we catch it here. 855 | * @param error 856 | */ 857 | function proxyResponseError(error, proxyRequest, proxyResponse, proxyTarget) { 858 | if (proxyResponse.status === undefined) { 859 | proxyResponse.status = 502; 860 | } 861 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('proxyResponseError caught error', {code: error.code, description: error.description, target: proxyTarget, status: proxyResponse.status})); 862 | sendErrorResponse(proxyRequest.url, proxyResponse, proxyResponse.status, Configuration.getStringTableEntry('Proxy request error', {code: error.code, description: error.description})); 863 | } 864 | 865 | /** 866 | * If the proxy target responds with an error we catch it here. I believe this is only for socket errors 867 | * as I have yet to catch any errors here. 868 | * @param proxyError 869 | */ 870 | function proxyErrorHandler(proxyError, proxyRequest, proxyResponse) { 871 | sendErrorResponse(proxyRequest.url, proxyResponse, 500, Configuration.getStringTableEntry('Proxy error 500', {error: proxyError.toString()})); 872 | } 873 | 874 | /** 875 | * The proxy service gives us a chance to alter the request before forwarding it to the proxied server. This is a place 876 | * where we could rewrite any inbound parameters and check any tokens, or add tokens to the proxied request. 877 | * @param proxyReq {ClientRequest} 878 | * @param proxyRequest {IncomingMessage} 879 | * @param proxyResponse {ServerResponse} 880 | * @param options {object} 881 | */ 882 | function proxyRequestRewrite(proxyReq, proxyRequest, proxyResponse, options) { 883 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('proxyRequestRewrite alter request before service', null)); 884 | } 885 | 886 | /** 887 | * The proxy service gives us a chance to alter the response before sending it back to the client. We are using 888 | * this to check for failed authentication replies. If the service is using tokens we can attempt to resolve 889 | * the expired or missing token only if our service definition has the required attributes. 890 | * @param serviceResponse - response from the service 891 | * @param proxyRequest - original request object 892 | * @param proxyResponse - response object from the proxy 893 | * @param options 894 | */ 895 | function proxyResponseRewrite(serviceResponse, proxyRequest, proxyResponse) { 896 | QuickLogger.logInfoEvent("proxyResponseRewrite opportunity to alter response before writing it."); 897 | var serverUrlInfo = proxyRequest.serverUrlInfo || {mayRequireToken: false}; 898 | if (serviceResponse.headers['content-type'] !== undefined) { 899 | var lookFor = 'application/vnd.ogc.wms_xml'; 900 | var replaceWith = 'text/xml'; 901 | serviceResponse.headers['content-type'] = serviceResponse.headers['content-type'].replace(lookFor, replaceWith); 902 | } 903 | if (serverUrlInfo.mayRequireToken) { 904 | // TODO: See if we got error 498/499. if so we need to generate a token. To do this we need to review the server reply and see if it failed because of a bad/missing token 905 | checkServerResponseForMissingToken(proxyResponse, serviceResponse.headers['content-encoding'], function (body) { 906 | var errorCode, 907 | newTokenIsRequired = false; 908 | 909 | if (body) { 910 | var errorCode = body.error.code; 911 | if (errorCode == 403 || errorCode == 498 || errorCode == 499) { 912 | newTokenIsRequired = true; 913 | } 914 | } 915 | return newTokenIsRequired; 916 | }); 917 | } 918 | } 919 | 920 | /** 921 | * Event we receive once the serviced request has completed. 922 | * @param proxyRequest 923 | * @param proxyResponse 924 | * @param serviceResponse 925 | */ 926 | function proxyResponseComplete(proxyRequest, proxyResponse, serviceResponse) { 927 | if (proxyResponse) { 928 | var buffer = serviceResponse.body; 929 | if (buffer != null && buffer.length > 0) { 930 | QuickLogger.logInfoEvent("proxyResponseComplete got something as reply from service."); 931 | } 932 | } 933 | } 934 | 935 | /** 936 | * Handle a request for specific files we can serve from the proxy server. Files are served from the assets folder. 937 | * Note this function always sends a response back to the requesting client. 938 | * @param request {object} the request being made. 939 | * @param response {object} the proxy server's response object. 940 | */ 941 | function proxyServeFile(request, response) { 942 | var responseStatus = 200, 943 | responseMessage; 944 | 945 | if (request.method == 'GET') { 946 | var requestedUrl = urlParser.parse(request.url, true), 947 | action = requestedUrl.pathname; 948 | 949 | if (action == '/') { 950 | responseStatus = 404; 951 | responseMessage = Configuration.getStringTableEntry('Resource not found', {url: action}); 952 | } else if (configuration.staticFilePath != null) { 953 | // serve static assets requests from the local folder. 954 | if (staticFileServer == null) { 955 | staticFileServer = new nodeStatic.Server(configuration.staticFilePath); 956 | } 957 | if (staticFileServer != null) { 958 | staticFileServer.serve(request, response); 959 | } 960 | } 961 | } else { 962 | responseStatus = 405; 963 | responseMessage = 'Method not supported.'; 964 | } 965 | if (responseStatus != 200) { 966 | sendErrorResponse(request.url, response, responseStatus, responseMessage); 967 | } 968 | } 969 | 970 | /** 971 | * Helper to easily determine the content type is JSON. Unfortunately, AGOL sends back text/plain when it means application/json. 972 | * @param contentType 973 | * @returns {boolean} 974 | */ 975 | function isContentTypeJSON(contentType) { 976 | return ['application/json', 'text/plain'].indexOf(contentType.toLowerCase()) >= 0; 977 | } 978 | 979 | /** 980 | * For the serverURLs that we manage credentials for, monitor the server responses to see if we can tell if 981 | * the server has sent us a refreshed token or the server decided to deny us access because of failed 982 | * token. In those cases we can correct the situation by getting a new token and trying again. 983 | * @param proxyResponse - monitor the response from the proxied server. 984 | * @param contentType - we need to know the content type of the response so we know how to look at it. 985 | * @param checkForMissingToken - a function we can call to find the token in the response body. 986 | */ 987 | function checkServerResponseForMissingToken(proxyResponse, contentType, checkForMissingToken) { 988 | var buffer = new BufferHelper(), 989 | tokenIsMissing = false, 990 | responseWrite = proxyResponse.write, 991 | responseEnd = proxyResponse.end, 992 | encoding; 993 | 994 | // TODO: Content type can be deflate and gzip we need to handle both of those. 995 | 996 | // Rewrite response method and get the content body. 997 | proxyResponse.write = function (data) { 998 | buffer.concat(data); 999 | // TODO: Make sure buffer does not grow to large. We should have a threshold. 1000 | }; 1001 | 1002 | proxyResponse.end = function () { 1003 | // TODO: I really don't like this. We are going to parse the entire response to see if we receive the specific error we 1004 | // are looking for. if it is an error this is pretty small, but if its not an error we could be parsing a rather monstrous amount of json! only to convert it back to string! 1005 | // Maybe better to regex match '{"error":{"code":500' => '\"error\":[\s]*{[\s]*\"code\":[\s]*[\d]*' 1006 | // check content-type make sure it is text or json 1007 | // check content size make sure it is reasonable 1008 | var body = '', 1009 | decodedBody; 1010 | try { 1011 | encoding = (proxyResponse._headers['content-encoding'] || 'utf8').toLowerCase(); 1012 | if (buffer.length > 0) { 1013 | body = buffer.toBuffer().toString(); 1014 | if (encoding == 'deflate') { 1015 | decodedBody = zlib.deflateSync(buffer); 1016 | } else if (encoding == 'gzip') { 1017 | decodedBody = zlib.gunzipSync(buffer); 1018 | } else { 1019 | decodedBody = body; 1020 | } 1021 | tokenIsMissing = checkForMissingToken(decodedBody); 1022 | } 1023 | } catch (e) { 1024 | console.log('JSON.parse error:', e.message); 1025 | console.log('JSON.parse error from: ' + decodedBody || body); 1026 | } 1027 | if ( ! tokenIsMissing) { 1028 | // Call the response method 1029 | responseWrite.call(proxyResponse, body); 1030 | responseEnd.call(proxyResponse); 1031 | } else { 1032 | // TODO: discard this response. get a new token from the token generator. retry the request with the new token. Send back the new response instead. 1033 | responseWrite.call(proxyResponse, Configuration.getStringTableEntry('Could not generate a new token', null)); 1034 | responseEnd.call(proxyResponse); 1035 | } 1036 | }; 1037 | } 1038 | 1039 | /** 1040 | * Run the server. This function never returns. You have to kill the process, such as ^C or kill. 1041 | * All connection requests are forwarded to processRequest(q, r). 1042 | */ 1043 | function startServer () { 1044 | var httpsOptions, 1045 | hostName, 1046 | proxyServerOptions = {}; 1047 | 1048 | try { 1049 | UrlFlexParser.setConfiguration(configuration); 1050 | serverStartTime = new Date(); 1051 | hostName = OS.hostname() + ' (' + OS.type() + ', ' + OS.release() + ')'; 1052 | QuickLogger.logInfoEvent("Starting proxy version " + proxyVersion + " running on " + hostName + " via " + (configuration.useHTTPS ? 'HTTPS' : 'HTTP') + " server on port " + configuration.port + " -- " + serverStartTime.toLocaleString()); 1053 | 1054 | // The RateMeter depends on the configuration.serverUrls being valid. 1055 | rateMeter = RateMeter(configuration.serverUrls, configuration.allowedReferrers, QuickLogger.logErrorEvent.bind(QuickLogger)); 1056 | rateMeter.start(); 1057 | 1058 | // If we are to run an https server we need to load the certificate and the key 1059 | if (configuration.useHTTPS) { 1060 | if (configuration.httpsPfxFile !== undefined) { 1061 | httpsOptions = { 1062 | pfx: fs.readFileSync(configuration.httpsPfxFile) 1063 | }; 1064 | } else if (configuration.httpsKeyFile !== undefined && configuration.httpsCertificateFile !== undefined) { 1065 | httpsOptions = { 1066 | key: fs.readFileSync(configuration.httpsKeyFile), 1067 | cert: fs.readFileSync(configuration.httpsCertificateFile) 1068 | }; 1069 | } else { 1070 | httpsOptions = {}; 1071 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Missing HTTPS proxy configuration', null)); 1072 | } 1073 | httpServer = https.createServer(httpsOptions, processRequest); 1074 | } else { 1075 | httpServer = http.createServer(processRequest); 1076 | } 1077 | if (httpServer != null) { 1078 | httpServer.on('clientError', function (error, socket) { 1079 | errorProcessedRequests ++; 1080 | socket.end('HTTP/1.1 400 Bad Request\r\n\r\n'); 1081 | }); 1082 | httpServer.on('error', function (error) { 1083 | errorProcessedRequests ++; 1084 | cannotListen(error); 1085 | }); 1086 | proxyServer = new httpProxy.createProxyServer(proxyServerOptions); 1087 | proxyServer.on('error', proxyErrorHandler); 1088 | proxyServer.on('proxyReq', proxyRequestRewrite); 1089 | proxyServer.on('proxyRes', proxyResponseRewrite); 1090 | proxyServer.on('end', proxyResponseComplete); 1091 | 1092 | // Integration tests require a fully parsed configuration and a started server, so they were delayed until this point. 1093 | if (waitingToRunIntegrationTests || Configuration.isTestMode()) { 1094 | __runIntegrationTests(); 1095 | if ( ! Configuration.isTestMode()) { 1096 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Integration tests complete shutting down from test', null)); 1097 | process.exit(); 1098 | } 1099 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Integration tests complete starting server', null)); 1100 | } 1101 | 1102 | // Begin listening for client connections 1103 | try { 1104 | httpServer.listen(configuration.port); 1105 | } catch (exception) { 1106 | cannotListen(exception); 1107 | } 1108 | } else { 1109 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Proxy server not created', null)); 1110 | } 1111 | } catch (exception) { 1112 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Proxy server startup exception', {exception: exception.toLocaleString()})); 1113 | } 1114 | } 1115 | 1116 | /** 1117 | * When loading the configuration fails we end up here with a reason message. We terminate the app. 1118 | * @param reason {Error} A message indicating why the configuration failed. 1119 | */ 1120 | function cannotStartServer(reason) { 1121 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Server not started invalid config', {reason: reason.message})); 1122 | process.exit(); 1123 | } 1124 | 1125 | /** 1126 | * Start up fails when listening on the socket fails with a reason message. We terminate the app. 1127 | * @param reason {Error} A message indicating why listen failed. 1128 | */ 1129 | function cannotListen(reason) { 1130 | QuickLogger.logErrorEvent(Configuration.getStringTableEntry('Server not started due to', {reason: reason.message})); 1131 | process.exit(); 1132 | } 1133 | 1134 | /** 1135 | * Perform any actions when the app is terminated. 1136 | * @param options 1137 | * @param error 1138 | */ 1139 | function exitHandler (options, error) { 1140 | QuickLogger.logEventImmediately(QuickLogger.LOGLEVEL.INFO.value, Configuration.getStringTableEntry('Stopping server via', {reason: options.reason})); 1141 | if (rateMeter != null) { 1142 | rateMeter.stop(); 1143 | rateMeter = null; 1144 | } 1145 | if (error) { 1146 | console.log(error.stack); 1147 | } 1148 | if (options.exit) { 1149 | if (proxyServer != null) { 1150 | proxyServer.close(); 1151 | } 1152 | process.exit(); 1153 | } 1154 | } 1155 | 1156 | /** 1157 | * Set up the node process exit handlers and any other node integration we require. 1158 | * @param process 1159 | */ 1160 | function configProcessHandlers(process) { 1161 | process.stdin.resume(); // so the program will not close instantly 1162 | 1163 | // Set handler for app shutdown event 1164 | process.on('exit', exitHandler.bind(null, {reason: "normal exit"})); 1165 | process.on('SIGINT', exitHandler.bind(null, {exit: true, reason: "app terminated via SIGINT"})); 1166 | process.on('uncaughtException', exitHandler.bind(null, {exit: true, reason: "uncaught exception"})); 1167 | } 1168 | 1169 | /** 1170 | * Run any tests that require our server is up and running. Waits for the server to be up and running 1171 | * before scheduling the tests. These tests are here because the functions were not exported and not 1172 | * accessible to the unit/integration test object. 1173 | */ 1174 | function runIntegrationTests() { 1175 | if (configurationComplete) { 1176 | __runIntegrationTests(); 1177 | QuickLogger.logInfoEvent(Configuration.getStringTableEntry('Integration tests complete runIntegrationTests', null)); 1178 | process.exit(); 1179 | } else { 1180 | waitingToRunIntegrationTests = true; 1181 | } 1182 | } 1183 | 1184 | function __runIntegrationTests() { 1185 | var testStr, 1186 | targetStr, 1187 | result, 1188 | serverUrlInfo, 1189 | urlParts, 1190 | token; 1191 | 1192 | waitingToRunIntegrationTests = false; 1193 | console.log("TTTTT Starting ProxyJS integration tests "); 1194 | 1195 | QuickLogger.logInfoEvent('This is an Info level event'); 1196 | QuickLogger.logWarnEvent('This is a Warning level event'); 1197 | QuickLogger.logErrorEvent('This is an Error level event'); 1198 | 1199 | testStr = '/proxy/geo.arcgis.com/ArcGIS/rest/info/'; 1200 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1201 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1202 | 1203 | testStr = '/proxy/http/geo.arcgis.com/ArcGIS/rest/info/'; 1204 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1205 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1206 | 1207 | testStr = '/proxy/https/geo.arcgis.com/ArcGIS/rest/info/'; 1208 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1209 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1210 | 1211 | testStr = '/proxy/*/geo.arcgis.com/ArcGIS/rest/info/'; 1212 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1213 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1214 | 1215 | testStr = '/proxy?geo.arcgis.com/ArcGIS/rest/info/'; 1216 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1217 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1218 | 1219 | testStr = '/proxy?http/geo.arcgis.com/ArcGIS/rest/info/'; 1220 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1221 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1222 | 1223 | testStr = '/proxy?http://geo.arcgis.com/ArcGIS/rest/info/'; 1224 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1225 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1226 | 1227 | testStr = '/proxy&geo.arcgis.com/ArcGIS/rest/info/'; 1228 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1229 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1230 | 1231 | testStr = '/proxy&http/geo.arcgis.com/ArcGIS/rest/info/'; 1232 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1233 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1234 | 1235 | testStr = '/proxy&http://geo.arcgis.com/ArcGIS/rest/info/'; 1236 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1237 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1238 | 1239 | testStr = ''; 1240 | result = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1241 | console.log('parseURLRequest url=' + testStr + ' result=' + JSON.stringify(result)); 1242 | 1243 | testStr = "server.gateway.com"; // should match *.gateway.com 1244 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1245 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1246 | 1247 | testStr = "www.gateway.com"; // should match *.gateway.com 1248 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1249 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1250 | 1251 | testStr = "https://www.customer.com/gateway"; // should match www.customer.com 1252 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1253 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1254 | 1255 | testStr = "https://www.brindle.com/gateway"; // should match *://*/gateway 1256 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1257 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1258 | 1259 | testStr = "https://www.esri.com/1/2/3"; // should match https://* 1260 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1261 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1262 | 1263 | testStr = "http://www.esri.com/1/2/3"; // should NOT match https://* 1264 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1265 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1266 | 1267 | testStr = "*"; // should not match anything 1268 | targetStr = UrlFlexParser.validatedReferrerFromReferrer(testStr, configuration.allowedReferrers); 1269 | console.log('validatedReferrerFromReferrer referrer=' + testStr + ' result=' + targetStr); 1270 | 1271 | testStr = 'application/json'; 1272 | result = isContentTypeJSON(testStr); 1273 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1274 | 1275 | testStr = 'APPLICATION/json'; 1276 | result = isContentTypeJSON(testStr); 1277 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1278 | 1279 | testStr = 'application/Json'; 1280 | result = isContentTypeJSON(testStr); 1281 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1282 | 1283 | testStr = 'text/plain'; 1284 | result = isContentTypeJSON(testStr); 1285 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1286 | 1287 | testStr = 'xxx/json'; 1288 | result = isContentTypeJSON(testStr); 1289 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1290 | 1291 | testStr = 'text/xml'; 1292 | result = isContentTypeJSON(testStr); 1293 | console.log('isContentTypeJSON ' + testStr + ' result=' + (result ? 'true' : 'false')); 1294 | 1295 | targetStr = 'Loading configuration from'; 1296 | testStr = 'this is a test file name'; 1297 | result = Configuration.getStringTableEntry(targetStr, testStr); // known regression for non-object parameter 1298 | QuickLogger.logInfoEvent('getStringTableEntry result=' + result); 1299 | 1300 | targetStr = 'Loading configuration from'; 1301 | testStr = 'this is a test file name'; 1302 | result = Configuration.getStringTableEntry(targetStr, {file: testStr}); 1303 | QuickLogger.logInfoEvent('getStringTableEntry result=' + result); 1304 | 1305 | httpRequestPromiseResponse("www.enginesis.com", "/index.php", "POST", false, {fn: "ESRBTypeList", site_id: 100, response: "json", user_id: 9999}).then(function(responseBody) { 1306 | result = responseBody; 1307 | console.log('httpRequestPromiseResponse POST ' + result); 1308 | }, function(error) { 1309 | console.log('httpRequestPromiseResponse POST error ' + error.message); 1310 | }); 1311 | 1312 | httpRequestPromiseResponse("www.enginesis.com", "/index.php", "GET", false, {fn: "ESRBTypeList", site_id: 100, response: "json", user_id: 9999}).then(function(responseBody) { 1313 | result = responseBody; 1314 | console.log('httpRequestPromiseResponse GET ' + result); 1315 | }, function(error) { 1316 | console.log('httpRequestPromiseResponse GET error ' + error.message); 1317 | }); 1318 | 1319 | testStr = '/proxy/http://route.arcgis.com/arcgis/rest/services/World/ClosestFacility/NAServer/ClosestFacility_World/solveClosestFacility'; 1320 | urlParts = UrlFlexParser.parseURLRequest(testStr, configuration.listenURI); 1321 | serverUrlInfo = getServerUrlInfo(urlParts); 1322 | getTokenEndpointFromURL(serverUrlInfo.url).then( 1323 | function(endpoint) { 1324 | console.log('getTokenEndpointFromURL got ' + endpoint); 1325 | }, 1326 | function(error) { 1327 | console.log('getTokenEndpointFromURL fails with ' + error.message); 1328 | } 1329 | ); 1330 | 1331 | testStr = 'http://developers.arcgis.com'; 1332 | token = null; 1333 | getNewTokenFromUserNamePasswordLogin(testStr, serverUrlInfo).then( 1334 | function(tokenResponse) { 1335 | token = tokenResponse; 1336 | console.log('getNewTokenFromUserNamePasswordLogin got ' + token); 1337 | }, 1338 | function(error) { 1339 | console.log('getNewTokenFromUserNamePasswordLogin fails with ' + error.message); 1340 | } 1341 | ); 1342 | 1343 | if (token !== null) { 1344 | exchangePortalTokenForServerToken(token, serverURLInfo).then( 1345 | function (tokenResponse) { 1346 | token = tokenResponse; 1347 | console.log('exchangePortalTokenForServerToken got ' + token); 1348 | }, 1349 | function (error) { 1350 | console.log('exchangePortalTokenForServerToken fails with ' + error.message); 1351 | } 1352 | ); 1353 | } else { 1354 | console.log('exchangePortalTokenForServerToken test not run because we do not have a short-lived token to test with'); 1355 | } 1356 | 1357 | // getNewTokenIfCredentialsAreSpecified(serverURLInfo, requestUrl); 1358 | // userLogin succeeds 1359 | // userLogin fails 1360 | // appLogin succeeds 1361 | // appLogin fails 1362 | // serverURLInfo does not specify credentials (error test) 1363 | 1364 | console.log("TTTTT Completed ProxyJS integration tests "); 1365 | } 1366 | 1367 | function loadConfigThenStart() { 1368 | configProcessHandlers(process); 1369 | Configuration.loadConfigurationFile('').then(startServer, cannotStartServer); 1370 | } 1371 | 1372 | exports.ArcGISProxyIntegrationTest = runIntegrationTests; 1373 | 1374 | loadConfigThenStart(); 1375 | -------------------------------------------------------------------------------- /bin/test.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Project unit test suite. 3 | */ 4 | 5 | const QuickLogger = require('./QuickLogger'); 6 | const ProjectUtilities = require('./ProjectUtilities'); 7 | const UrlFlexParser = require('./UrlFlexParser'); 8 | const ProxyJS = require('./proxy'); 9 | 10 | 11 | function unitTests () { 12 | var testStr, 13 | targetStr, 14 | result; 15 | 16 | console.log('TTTTT Local unit tests start:'); 17 | 18 | testStr = 'http://server.com/application1'; 19 | targetStr = 'http://'; 20 | result = ProjectUtilities.startsWith(testStr, targetStr); 21 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 22 | 23 | testStr = 'http://server.com/application1'; 24 | targetStr = 'https://'; 25 | result = ProjectUtilities.startsWith(testStr, targetStr); 26 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 27 | 28 | testStr = 'http://server.com/application1'; 29 | targetStr = ''; 30 | result = ProjectUtilities.startsWith(testStr, targetStr); 31 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 32 | 33 | testStr = ''; 34 | targetStr = ''; 35 | result = ProjectUtilities.startsWith(testStr, targetStr); 36 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 37 | 38 | testStr = ''; 39 | targetStr = 'http://'; 40 | result = ProjectUtilities.startsWith(testStr, targetStr); 41 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 42 | 43 | testStr = 'http://server.com/application1'; 44 | targetStr = 'http://server.com/application1'; 45 | result = ProjectUtilities.startsWith(testStr, targetStr); 46 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 47 | 48 | testStr = 'http://server.com/application1'; 49 | targetStr = 'http://server.com/application1-xxx'; 50 | result = ProjectUtilities.startsWith(testStr, targetStr); 51 | console.log('startsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 52 | 53 | 54 | testStr = 'http://server.com/application1'; 55 | targetStr = 'http://'; 56 | result = ProjectUtilities.endsWith(testStr, targetStr); 57 | console.log('endsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 58 | 59 | testStr = 'http://server.com/application1'; 60 | targetStr = 'on1'; 61 | result = ProjectUtilities.endsWith(testStr, targetStr); 62 | console.log('endsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 63 | 64 | testStr = 'http://server.com/application1'; 65 | targetStr = ''; 66 | result = ProjectUtilities.endsWith(testStr, targetStr); 67 | console.log('endsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 68 | 69 | testStr = ''; 70 | targetStr = 'http:'; 71 | result = ProjectUtilities.endsWith(testStr, targetStr); 72 | console.log('endsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 73 | 74 | testStr = 'http://server.com/application1'; 75 | targetStr = 'http://server.com/application1'; 76 | result = ProjectUtilities.endsWith(testStr, targetStr); 77 | console.log('endsWith subject=' + testStr + ' needle=' + targetStr + ' result=' + (result ? 'true' : 'false')); 78 | 79 | 80 | result = ProjectUtilities.formatMillisecondsToHHMMSS(new Date()); 81 | console.log('formatMillisecondsToHHMMSS result=' + result); 82 | 83 | 84 | var configuration = { 85 | mustMatch: true, 86 | useHTTPS: true, 87 | basePath: '/1/2/3/index.html' 88 | }; 89 | 90 | targetStr = 'mustMatch'; 91 | result = ProjectUtilities.isPropertySet(configuration, targetStr); 92 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result ? 'true' : 'false')); 93 | 94 | targetStr = 'xyz'; 95 | result = ProjectUtilities.isPropertySet(configuration, targetStr); 96 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result ? 'true' : 'false')); 97 | 98 | targetStr = 'useHttps'; 99 | result = ProjectUtilities.isPropertySet(configuration, targetStr); 100 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result ? 'true' : 'false')); 101 | 102 | targetStr = 'useHTTPS'; 103 | result = ProjectUtilities.isPropertySet(configuration, targetStr); 104 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result ? 'true' : 'false')); 105 | 106 | targetStr = 'basePath'; 107 | result = ProjectUtilities.isPropertySet(configuration, targetStr); 108 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result ? 'true' : 'false')); 109 | 110 | 111 | targetStr = 'mustMatch'; 112 | result = ProjectUtilities.getIfPropertySet(configuration, targetStr, null); 113 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result === null ? 'null' : result)); 114 | 115 | targetStr = 'xyz'; 116 | result = ProjectUtilities.getIfPropertySet(configuration, targetStr, null); 117 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result === null ? 'null' : result)); 118 | 119 | targetStr = 'basePath'; 120 | result = ProjectUtilities.getIfPropertySet(configuration, targetStr, null); 121 | console.log('isPropertySet(configuration, ' + targetStr + ') result=' + (result === null ? 'null' : result)); 122 | 123 | 124 | testStr = '*://*.esri.com/'; 125 | result = UrlFlexParser.parseAndFixURLParts(testStr); 126 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 127 | 128 | testStr = 'http://*.esri.com/*'; 129 | result = UrlFlexParser.parseAndFixURLParts(testStr); 130 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 131 | 132 | testStr = '*://*/gateway/proxy/this-is-my-key/'; 133 | result = UrlFlexParser.parseAndFixURLParts(testStr); 134 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 135 | 136 | testStr = 'http://www.esri.com/1/2/3?q=123&y=0'; 137 | result = UrlFlexParser.parseAndFixURLParts(testStr); 138 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 139 | 140 | testStr = 'http://developers.arcgis.esri.com/1/2/3/'; 141 | result = UrlFlexParser.parseAndFixURLParts(testStr); 142 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 143 | 144 | testStr = 'https://developers.arcgis.esri.com/1/2/3/'; 145 | result = UrlFlexParser.parseAndFixURLParts(testStr); 146 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 147 | 148 | testStr = ''; 149 | result = UrlFlexParser.parseAndFixURLParts(testStr); 150 | console.log('parseAndFixURLParts url=' + testStr + ' result=' + JSON.stringify(result)); 151 | 152 | 153 | testStr = 'www.here.com'; 154 | targetStr = 'xyz'; 155 | result = UrlFlexParser.combinePath(testStr, targetStr); 156 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 157 | 158 | testStr = 'http://www.here.com/'; 159 | targetStr = '/xyz'; 160 | result = UrlFlexParser.combinePath(testStr, targetStr); 161 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 162 | 163 | testStr = 'http://www.here.com////'; 164 | targetStr = '////xyz'; 165 | result = UrlFlexParser.combinePath(testStr, targetStr); 166 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 167 | 168 | testStr = ''; 169 | targetStr = ''; 170 | result = UrlFlexParser.combinePath(testStr, targetStr); 171 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 172 | 173 | testStr = '/'; 174 | targetStr = '/'; 175 | result = UrlFlexParser.combinePath(testStr, targetStr); 176 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 177 | 178 | testStr = null; 179 | targetStr = null; 180 | result = UrlFlexParser.combinePath(testStr, targetStr); 181 | console.log('combinePath(' + testStr + ', ' + targetStr + ') result=' + (result === null ? 'null' : result)); 182 | 183 | 184 | testStr = 'server.com'; 185 | targetStr = 'server.com'; 186 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 187 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 188 | 189 | testStr = 'www.server.com'; 190 | targetStr = 'service.server.com'; 191 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 192 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 193 | 194 | testStr = '*.server.com'; 195 | targetStr = 'www.server.com'; 196 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 197 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 198 | 199 | testStr = 'www.xyz.server.com'; 200 | targetStr = 'service.xyz.server.com'; 201 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 202 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 203 | 204 | testStr = 'www.sdjfh.server.com'; 205 | targetStr = 'www.jsadfoij.server.com'; 206 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 207 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 208 | 209 | testStr = 'www.*.server.com'; 210 | targetStr = 'www.jsadfoij.server.com'; 211 | result = UrlFlexParser.testDomainsMatch(testStr, targetStr); 212 | console.log('testDomainsMatch domain=' + testStr + ' domain=' + targetStr + ' result=' + (result ? 'true' : 'false')); 213 | 214 | 215 | testStr = '*'; 216 | targetStr = '*'; 217 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 218 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 219 | 220 | testStr = '*'; 221 | targetStr = 'http'; 222 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 223 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 224 | 225 | testStr = '*'; 226 | targetStr = 'https'; 227 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 228 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 229 | 230 | testStr = 'http'; 231 | targetStr = '*'; 232 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 233 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 234 | 235 | testStr = 'https'; 236 | targetStr = 'http'; 237 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 238 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 239 | 240 | testStr = 'file'; 241 | targetStr = 'http'; 242 | result = UrlFlexParser.testProtocolsMatch(testStr, targetStr); 243 | console.log('testProtocolsMatch p1=' + testStr + ' p2=' + targetStr + ' result=' + (result ? 'true' : 'false')); 244 | 245 | 246 | targetStr = 'application/vnd.ogc.wms_xml'; 247 | testStr = 'text/xml'; 248 | result = '123application/vnd.ogc.wms_xml;charset: utf8;'.replace(targetStr, testStr); 249 | console.log('string.replace p1=' + targetStr + ' p2=' + testStr + ' result=' + result); 250 | 251 | targetStr = 'application/vnd.ogc.wms_xml'; 252 | testStr = 'text/xml'; 253 | result = 'application/vnd.ogc.wms_xml;charset: utf8;'.replace(targetStr, testStr); 254 | console.log('string.replace p1=' + targetStr + ' p2=' + testStr + ' result=' + result); 255 | 256 | targetStr = 'application/vnd.ogc.wms_xml'; 257 | testStr = 'text/xml'; 258 | result = 'ation/vnd.ogc.wms_xml;charset: utf8;'.replace(targetStr, testStr); 259 | console.log('string.replace p1=' + targetStr + ' p2=' + testStr + ' result=' + result); 260 | 261 | testStr = '?token=3426874628764872638476287634&x=121&y=22&f=json&user=Harry Bartell&sammy=davis'; 262 | result = ProjectUtilities.queryStringToObject(testStr); 263 | console.log('queryStringToObject for ' + testStr + ' result=' + JSON.stringify(result)); 264 | 265 | testStr = 'token=3426874628764872638476287634&x="121"&y=22&f=json&user=Harry Bartell&sammy=davis'; 266 | result = ProjectUtilities.queryStringToObject(testStr); 267 | console.log('queryStringToObject for ' + testStr + ' result=' + JSON.stringify(result)); 268 | 269 | targetStr = ProjectUtilities.objectToQueryString(result); 270 | console.log('objectToQueryString for ' + JSON.stringify(result) + ' result=' + targetStr); 271 | 272 | testStr = '?token=3426874628764872638476287634&x=121&y=22&f=json&user=Harry Bartell&sammy=davis'; 273 | targetStr = 'token'; 274 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 275 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 276 | 277 | testStr = '?y=22&f=json&user=Harry Bartell&sammy=davis&token=3426874628764872638476287634&x=121'; 278 | targetStr = 'token'; 279 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 280 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 281 | 282 | testStr = '?y=22&f=json&user=Harry Bartell&sammy=davis&token=3426874628764872638476287634'; 283 | targetStr = 'token'; 284 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 285 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 286 | 287 | testStr = 'token=3426874628764872638476287634'; 288 | targetStr = 'token'; 289 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 290 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 291 | 292 | testStr = 'token="3426874628764872638476287634"'; 293 | targetStr = 'token'; 294 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 295 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 296 | 297 | testStr = '?y=22&f=json&user=Harry Bartell&sammy=davis'; 298 | targetStr = 'token'; 299 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 300 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 301 | 302 | testStr = '"token":"3426874628764872638476287634"'; 303 | targetStr = 'token'; 304 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 305 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 306 | 307 | testStr = '"token": "3426874628764872638476287634"'; 308 | targetStr = 'token'; 309 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 310 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 311 | 312 | testStr = '{"token": "3426874628764872638476287634"}'; 313 | targetStr = 'token'; 314 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 315 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 316 | 317 | testStr = '{"error": 498,\n"token": "3426874628764872638476287634"\n"f": "json"}'; 318 | targetStr = 'token'; 319 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 320 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 321 | 322 | testStr = '{"error":498,\n"token":"3426874628764872638476287634"\n"f":"json"}'; 323 | targetStr = 'token'; 324 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 325 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 326 | 327 | testStr = '{"error":498,\n"f":"json"}'; 328 | targetStr = 'token'; 329 | result = ProjectUtilities.findTokenInString(testStr, targetStr); 330 | console.log('findTokenInString for ' + testStr + ' "' + targetStr + '" result=' + result); 331 | 332 | testStr = 'this-file-name.json'; 333 | result = ProjectUtilities.isFileTypeJson(testStr); 334 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 335 | testStr = 'this-file-name.xml'; 336 | result = ProjectUtilities.isFileTypeJson(testStr); 337 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 338 | testStr = ''; 339 | result = ProjectUtilities.isFileTypeJson(testStr); 340 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 341 | testStr = 'this-file-name'; 342 | result = ProjectUtilities.isFileTypeJson(testStr); 343 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 344 | testStr = 'json-JSON-name.JSON'; 345 | result = ProjectUtilities.isFileTypeJson(testStr); 346 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 347 | testStr = 'json-JSON-name-json'; 348 | result = ProjectUtilities.isFileTypeJson(testStr); 349 | console.log('isFileTypeJson for ' + testStr + ' result=' + (result ? 'true' : 'false')); 350 | 351 | console.log('TTTTT Local unit tests complete:'); 352 | 353 | if (ProxyJS && ProxyJS.ArcGISProxyIntegrationTest) { 354 | ProxyJS.ArcGISProxyIntegrationTest(); // <== actually just queues the integration test: it cannot start until after the server is started. 355 | } 356 | } 357 | 358 | unitTests(); 359 | -------------------------------------------------------------------------------- /conf/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "ProxyConfig": { 3 | "language": "en", 4 | "port": 3692, 5 | "useHTTPS": false, 6 | "mustMatch": true, 7 | "logLevel": "ALL", 8 | "logToConsole": true, 9 | "logFile": "arcgis-proxy-node.log", 10 | "logFilePath": "./", 11 | "listenURI": ["/proxy", "/sproxy"], 12 | "pingPath": "/ping", 13 | "statusPath": "/status", 14 | "staticFilePath": "../assets", 15 | "allowedReferrers": "*" 16 | }, 17 | "serverUrls": [ 18 | { 19 | "url": "http://services.arcgisonline.com", 20 | "matchAll": false 21 | }, 22 | { 23 | "url": "http://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer/Geoenrichment/enrich?f=json", 24 | "matchAll": false 25 | }, 26 | { 27 | "url": "demo.arcgis.com/ArcGIS/rest/info/", 28 | "hostRedirect": "https://services.arcgisonline.com", 29 | "rateLimit": 120, 30 | "rateLimitPeriod": 1, 31 | "matchAll": true 32 | }, 33 | { 34 | "url": "demo.arcgis.com", 35 | "hostRedirect": "https://services.arcgisonline.com/ArcGIS/rest/info/", 36 | "rateLimit": 120, 37 | "rateLimitPeriod": 1, 38 | "matchAll": true 39 | }, 40 | { 41 | "url": "http://geocode.arcgis.com/arcgis/rest/services/Locators/ESRI_Geocode_USA/GeocodeServer/suggest", 42 | "rateLimit": 120, 43 | "rateLimitPeriod": 1, 44 | "matchAll": true 45 | }, 46 | { 47 | "url": "route.arcgis.com", 48 | "hostRedirect": "http://route.arcgis.com/arcgis/rest/services/World/Route/NAServer/Route_World", 49 | "oauth2Endpoint": "https://www.arcgis.com/sharing/oauth2", 50 | "username": "username", 51 | "password": "password", 52 | "clientId": "6Xo1d-example-9Kn2", 53 | "clientSecret": "5a5d50-example-c867b6efcf969bdcc6a2", 54 | "rateLimit": 120, 55 | "rateLimitPeriod": 1, 56 | "matchAll": true 57 | } 58 | ] 59 | } -------------------------------------------------------------------------------- /conf/config.xml: -------------------------------------------------------------------------------- 1 | 2 | 15 | 16 | 18 | 21 | 26 | 31 | 36 | 46 | 47 | 48 | -------------------------------------------------------------------------------- /conf/en.json: -------------------------------------------------------------------------------- 1 | { 2 | "Configuration file not valid": "Configuration file not valid, check log or error console for more information.", 3 | "Invalid configuration file format": "!!! Invalid configuration file format {error} !!!", 4 | "No URI to listen for": "No URI was set to listen for. Indicate a URI path on your server, for example /proxy.", 5 | "You must configure serverUrls": "You must configure serverUrls.", 6 | "You must configure one serverUrl": "You must configure serverUrls for at least one service.", 7 | "Error in server URL definition": "Error(s) in the server URL definitions for {url}: {error}", 8 | "You should configure at least one referrer": "You should configure allowedReferrers to at least one referrer, use ['*'] to accept all connections. Defaulting to ['*'].", 9 | "No logging level requested": "No logging level requested, logging level set to ERROR.", 10 | "Undefined logging level": "Undefined logging level {level} requested, logging level set to ERROR.", 11 | "OAuth requires clientId, clientSecret, oauth2Endpoint": "When using OAuth a setting for clientId, clientSecret, and oauth2Endpoint must all be provided. At least one is missing.", 12 | "Must provide username/password": "When using username/password both must be provided. At least one is missing.", 13 | "Missing HTTPS proxy configuration": "HTTPS proxy was requested but the necessary files [httpsPfxFile or httpsKeyFile, httpsCertificateFile] were not defined in configuration.", 14 | "Loading configuration from": "Loading configuration from {file}", 15 | "Invalid static file path": "Invalid or inaccessible static file path {path}, this service will not be available.", 16 | "Proxy server not created": "Proxy server was not created, probably an OS system or memory issue.", 17 | "Proxy server startup exception": "Proxy server encountered an exception and is not able to start: {exception}.", 18 | "Integration tests complete startServer": "Integration tests completed from startServer.", 19 | "Integration tests complete runIntegrationTests": "Integration tests completed from runIntegrationTests.", 20 | "Stopping server via": "Stopping server via {reason}.", 21 | "Server not started due to": "!!! Server not started due to {reason} !!!", 22 | "Server not started invalid config": "!!! Server not started due to invalid configuration {reason} !!!", 23 | "Resource not accessible": "Resource not accessible.", 24 | "Proxy error 500": "Proxy error {error}.", 25 | "Proxy request error": "Proxy request error {code}: {description}", 26 | "proxyResponseError caught error": "proxyResponseError caught error {code}: {description} on {target} status={status}", 27 | "proxyRequestRewrite alter request before service": "proxyRequestRewrite opportunity to alter request before contacting service.", 28 | "Resource not found": "Resource {url} not found.", 29 | "Referrer not allowed": "Referrer {referrer} not allowed.", 30 | "Invalid request 403": "Invalid request: could not parse request as a valid request.", 31 | "New request from": "---- New request from {referrer} for {path} ----", 32 | "Request error with info": "Request error: {error} ({code}) for {url}", 33 | "RateMeter failed on": "RateMeter failed on {url} from {referrer}: {error}.", 34 | "Metered resource status failed": "This is a metered resource but the server failed to determine the meter status of this resource.", 35 | "RateMeter blocking access to": "RateMeter exceeded, blocking access to {url} from {referrer}.", 36 | "Metered requests exceeded": "This is a metered resource, number of requests have exceeded the rate limit interval.", 37 | "Status request from": "Status request from {referrer}.", 38 | "System error processing request": "System error processing request: {message}.", 39 | "Resource Proxy Status title": "Resource Proxy Status", 40 | "Empty row": "** empty **", 41 | "Unable to transform to token endpoint": "Unable to transform {url} or {tokenUrl} into a token endpoint URL.", 42 | "Unable to transform to usable URL": "Unable to transform {url} or {tokenUrl} into a usable URL.", 43 | "Username and password must be set": "Both username and password must be set to enable user login.", 44 | "Service is secured by": "Service is secured by {oauth2Endpoint}: getting new token...", 45 | "Service requires user login": "Service requires user login.", 46 | "App login could not get a token": "App login could not get a token from the server response of {response}.", 47 | "User login could not get a token": "User login could not get a token from the server response of {response}.", 48 | "No method to authenticate": "serverURLInfo {url} does not specify a method to authenticate.", 49 | "Could not get a token from server response": "Error: could not get a token from the server response of {response}.", 50 | "Internal error": "Internal error: proxy server is not operating.", 51 | "Proxy has not been set up for": "Request from {referrer}, proxy has not been set up for {path}.", 52 | "Proxy has not been set up for extra": "Make sure there is a serverUrl in the configuration file that matches {path}.", 53 | "Could not generate a new token": "We could not generate a new token for you.", 54 | "unexpected value for parameterOverride": "Unexpected value for parameterOverride: {value}. Assuming 'referrer'.", 55 | "Transform url to token endpoint": "Transformed {url} to {tokenEndpoint} in order to get token" 56 | } -------------------------------------------------------------------------------- /conf/es.json: -------------------------------------------------------------------------------- 1 | { 2 | "Configuration file not valid": "Archivo de configuración no válido, registro de comprobación o consola de errores para obtener más información.", 3 | "Invalid configuration file format": "!!! Formato de archivo de configuración no válido. {error} !!!", 4 | "No URI to listen for": "No URI was set to listen for. Indicate a URI path on your server, for example /proxy.", 5 | "You must configure serverUrls": "You must configure serverUrls.", 6 | "You must configure one serverUrl": "You must configure serverUrls for at least one service.", 7 | "Error in server URL definition": "Error(s) in the server URL definitions for {url}: {error}", 8 | "You should configure at least one referrer": "You should configure allowedReferrers to at least one referrer, use ['*'] to accept all connections. Defaulting to ['*'].", 9 | "No logging level requested": "No logging level requested, logging level set to ERROR.", 10 | "Undefined logging level": "Undefined logging level {level} requested, logging level set to ERROR.", 11 | "OAuth requires clientId, clientSecret, oauth2Endpoint": "When using OAuth a setting for clientId, clientSecret, and oauth2Endpoint must all be provided. At least one is missing.", 12 | "Must provide username/password": "When using username/password both must be provided. At least one is missing.", 13 | "Missing HTTPS proxy configuration": "HTTPS proxy was requested but the necessary files [httpsPfxFile or httpsKeyFile, httpsCertificateFile] were not defined in configuration.", 14 | "Loading configuration from": "Cargando la configuración desde {file}", 15 | "Invalid static file path": "Invalid or inaccessible static file path {path}, this service will not be available.", 16 | "Proxy server not created": "Proxy server was not created, probably an OS system or memory issue.", 17 | "Proxy server startup exception": "Proxy server encountered an exception and is not able to start: {exception}.", 18 | "Integration tests complete startServer": "Integration tests completed from startServer.", 19 | "Integration tests complete runIntegrationTests": "Integration tests completed from runIntegrationTests.", 20 | "Stopping server via": "Detención del servidor mediante {reason}.", 21 | "Server not started due to": "!!! Server not started due to {reason} !!!", 22 | "Server not started invalid config": "!!! El servidor no se inició debido a una configuración no válida {reason} !!!", 23 | "Resource not accessible": "Recurso no accesible.", 24 | "Proxy error 500": "Proxy error {error}.", 25 | "Proxy request error": "Proxy request error {code}: {description}", 26 | "proxyResponseError caught error": "proxyResponseError caught error {code}: {description} on {target} status={status}", 27 | "proxyRequestRewrite alter request before service": "proxyRequestRewrite opportunity to alter request before contacting service.", 28 | "Resource not found": "Resource {url} not found.", 29 | "Referrer not allowed": "Referrer {referrer} not allowed.", 30 | "Invalid request 403": "Solicitud no válida: no se pudo analizar la solicitud como una solicitud válida.", 31 | "New request from": "---- Nueva solicitud de {referrer} para {path} ----", 32 | "Request error with info": "Solicitud de error: {error} ({code}) de {url}", 33 | "RateMeter failed on": "RateMeter failed on {url} from {referrer}: {error}.", 34 | "Metered resource status failed": "This is a metered resource but the server failed to determine the meter status of this resource.", 35 | "RateMeter blocking access to": "RateMeter exceeded, blocking access to {url} from {referrer}.", 36 | "Metered requests exceeded": "This is a metered resource, number of requests have exceeded the rate limit interval.", 37 | "Status request from": "Solicitud de estado de {referrer}.", 38 | "System error processing request": "Solicitud de procesamiento de error del sistema: {message}.", 39 | "Resource Proxy Status title": "Estado del proxy de recursos", 40 | "Empty row": "** vacío **", 41 | "Unable to transform to token endpoint": "Unable to transform {url} or {tokenUrl} into a token endpoint URL.", 42 | "Unable to transform to usable URL": "Unable to transform {url} or {tokenUrl} into a usable URL.", 43 | "Username and password must be set": "Tanto el nombre de usuario como la contraseña deben configurarse para habilitar el inicio de sesión de usuario.", 44 | "Service is secured by": "Service is secured by {oauth2Endpoint}: getting new token...", 45 | "Service requires user login": "El servicio requiere acceso de usuario.", 46 | "App login could not get a token": "App login could not get a token from the server response of {response}.", 47 | "User login could not get a token": "User login could not get a token from the server response of {response}.", 48 | "No method to authenticate": "serverURLInfo {url} does not specify a method to authenticate.", 49 | "Could not get a token from server response": "Error: could not get a token from the server response of {response}.", 50 | "Internal error": "Internal error: proxy server is not operating.", 51 | "Proxy has not been set up for": "Request from {referrer}, proxy has not been set up for {path}.", 52 | "Proxy has not been set up for extra": "Make sure there is a serverUrl in the configuration file that matches {path}.", 53 | "Could not generate a new token": "No pudimos generar un nuevo token para ti.", 54 | "unexpected value for parameterOverride": "Unexpected value for parameterOverride: {value}. Assuming 'referrer'.", 55 | "Transform url to token endpoint": "Transformed {url} to {tokenEndpoint} in order to get token" 56 | } 57 | -------------------------------------------------------------------------------- /notes.md: -------------------------------------------------------------------------------- 1 | # Notes 2 | 3 | ## Generating a token for application 4 | 5 | You need: 6 | 7 | client-id: tdb85Rcx5sESf9p6 8 | client-secret: 83e0d77a2f264ef0b81c80abef9c61d1 9 | 10 | ``` 11 | https://www.arcgis.com/sharing/oauth2/token?grant_type=client_credentials&f=json&client_id={client-id}&client_secret={client-secret} 12 | ``` 13 | 14 | Response: 15 | 16 | ``` 17 | { 18 | "access_token":"H9u5TwbUXyCHCjbsTX-0wKG6ya_tuyBl23QeYdcQtbhormiaiOEAZny4EjUxekMId6tnq9esrqxDAMOrt4VeYhABuUdocKM0oH589yJGwE9CPz-NLdb0avHVbb2u0SNG620JQuXHzJgsCXrge0YQmA..", 19 | "expires_in":7200 20 | } 21 | ``` 22 | 23 | ## Generating a token for named user 24 | 25 | You need: 26 | 27 | user-id: 28 | password: 29 | 30 | POST to 31 | 32 | ``` 33 | https://www.arcgis.com/sharing/oauth2/authorize 34 | client_id={client-id} 35 | response_type=code 36 | expiration=7200 37 | redirect_uri={} 38 | ``` 39 | 40 | Response: 41 | 42 | ``` 43 | { 44 | "access_token":"H9u5TwbUXyCHCjbsTX-0wKG6ya_tuyBl23QeYdcQtbhormiaiOEAZny4EjUxekMId6tnq9esrqxDAMOrt4VeYhABuUdocKM0oH589yJGwE9CPz-NLdb0avHVbb2u0SNG620JQuXHzJgsCXrge0YQmA..", 45 | "expires_in":7200 46 | } 47 | ``` 48 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "arcgis-resource-proxy", 3 | "description": "ArcGIS Resource Proxy", 4 | "version": "0.1.5", 5 | "author": "Esri Runtime", 6 | "contributors": [ 7 | "john foster " 8 | ], 9 | "bugs": { 10 | "url": "https://github.com/ArcGIS/arcgis-resource-proxy/issues" 11 | }, 12 | "repository": { 13 | "type": "git", 14 | "url": "https://github.com/ArcGIS/arcgis-resource-proxy" 15 | }, 16 | "homepage": "https://github.com/ArcGIS/arcgis-for-developers", 17 | "keywords": [ 18 | "proxy", 19 | "node", 20 | "javascript", 21 | "esri", 22 | "arcgis", 23 | "developers" 24 | ], 25 | "license": "Apache-2.0", 26 | "main": "bin/proxy.js", 27 | "private": false, 28 | "dependencies": { 29 | "body-parser": "latest", 30 | "bufferhelper": "latest", 31 | "connect": "latest", 32 | "http-proxy": "latest", 33 | "load-json-file": "^2.0.0", 34 | "node-static": "latest", 35 | "os": "latest", 36 | "path.join": "latest", 37 | "sqlite3": "latest", 38 | "xml2js": "^0.4.17" 39 | }, 40 | "devDependencies": { 41 | }, 42 | "scripts": { 43 | "start": "node bin/proxy.js", 44 | "test": "node bin/test.js" 45 | } 46 | } 47 | --------------------------------------------------------------------------------