├── .gitignore ├── LICENSE ├── README-SAR.md ├── README.md ├── examples ├── apps │ ├── api-gateway.yaml │ └── scheduled-event.yaml ├── cloudrace.js └── visitgoogle.js ├── layer ├── fetch-binaries.sh └── lib │ ├── libORBit-2.so.0 │ └── libgconf-2.so.4 ├── src ├── event-api.json ├── event.json ├── index.js ├── lib │ ├── api-handler.js │ ├── chromium.js │ ├── sandbox.js │ └── visit-page.js ├── package-lock.json ├── package.json ├── scripts │ ├── fetch-dependencies.sh │ └── invoke.sh └── tests │ └── .gitkeep └── template.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | terraform.tfstate 3 | terraform.tfstate.backup 4 | lambda_function.zip 5 | build/ 6 | bin/ 7 | node_modules 8 | packaged.yaml 9 | 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | ISC License (ISC) 2 | Copyright 2018 Clay Smith 3 | 4 | Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. 5 | 6 | THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 7 | -------------------------------------------------------------------------------- /README-SAR.md: -------------------------------------------------------------------------------- 1 | ## lambdium 2 | ### serverless application repository quickstart 3 | 4 | Lambdium allows you to run a Selenium Webdriver script written in Javascript inside of an AWS Lambda function bundled with a special version of Google Chrome (headless chrome). 5 | 6 | For example, this script uses Selenium to automate visiting the google.com homepage and print the title to the console: 7 | 8 | ``` 9 | console.log('About to visit google.com...'); 10 | $browser.get('http://www.google.com/ncr'); 11 | $browser.wait($driver.until.titleIs('Google'), 1000); 12 | $browser.getTitle().then(function(title) { 13 | console.log("title is: " + title); 14 | }); 15 | console.log('Finished running script!'); 16 | ``` 17 | 18 | After running this script inside of of the AWS Lambda function running lambdium, you can look at Cloudwatch logs to read the output of the script: 19 | 20 | ``` 21 | 23:37:332018-03-12T23:37:33.682Z 567babb5-264e-11e8-ae20-e37b20048a93 About to visit google.com... 22 | 23:37:33 2018-03-12T23:37:33.682Z 567babb5-264e-11e8-ae20-e37b20048a93 Finished running script! 23 | 23:37:36 2018-03-12T23:37:36.860Z 567babb5-264e-11e8-ae20-e37b20048a93 title is: Google 24 | ``` 25 | 26 | ### running your first selenium script 27 | 28 | lambdium requires you to invoke it using a special event trigger: a JSON document with a single property called `Base64Script`, which is a Base64-encoded selenium script written in Javascript. The example which visits google.com looks like this: 29 | 30 | ``` 31 | { 32 | "Base64Script": "Ly8gU2FtcGxlIHNlbGVuaW11bS13ZWJkcml2ZXIgc2NyaXB0IHRoYXQgdmlzaXRzIGdvb2dsZS5jb20KLy8gVGhpcyB1c2VzIHRoZSBzZWxlbml1bS13ZWJkcml2ZXIgMy40IHBhY2thZ2UuCi8vIERvY3M6IGh0dHBzOi8vc2VsZW5pdW1ocS5naXRodWIuaW8vc2VsZW5pdW0vZG9jcy9hcGkvamF2YXNjcmlwdC9tb2R1bGUvc2VsZW5pdW0td2ViZHJpdmVyL2luZGV4Lmh0bWwKLy8gJGJyb3dzZXIgPSB3ZWJkcml2ZXIgc2Vzc2lvbgovLyAkZHJpdmVyID0gZHJpdmVyIGxpYnJhcmllcwovLyBjb25zb2xlLmxvZyB3aWxsIG91dHB1dCB0byBBV1MgTGFtYmRhIGxvZ3MgKHZpYSBDbG91ZHdhdGNoKQoKY29uc29sZS5sb2coJ0Fib3V0IHRvIHZpc2l0IGdvb2dsZS5jb20uLi4nKTsKJGJyb3dzZXIuZ2V0KCdodHRwOi8vd3d3Lmdvb2dsZS5jb20vbmNyJyk7CiRicm93c2VyLmZpbmRFbGVtZW50KCRkcml2ZXIuQnkubmFtZSgnYnRuSycpKS5jbGljaygpOwokYnJvd3Nlci53YWl0KCRkcml2ZXIudW50aWwudGl0bGVJcygnR29vZ2xlJyksIDEwMDApOwokYnJvd3Nlci5nZXRUaXRsZSgpLnRoZW4oZnVuY3Rpb24odGl0bGUpIHsKICAgIGNvbnNvbGUubG9nKCJ0aXRsZSBpczogIiArIHRpdGxlKTsKfSk7CmNvbnNvbGUubG9nKCdGaW5pc2hlZCBydW5uaW5nIHNjcmlwdCEnKTs=" 33 | } 34 | ``` 35 | 36 | To encode another script, you can paste the contents of a script in an online converter like [https://www.base64encode.org/](https://www.base64encode.org/) or using the following commands, which are also available as a simple shell script in `scripts/invoke.sh`: 37 | 38 | ``` 39 | SELENIUM_SCRIPT=/path/to/your/script 40 | BASE64_ENCODED=`cat $SELENIUM_SCRIPT | openssl base64` 41 | PAYLOAD_STRING='{"Base64Script": "'$BASE64_ENCODED'"}' 42 | echo $PAYLOAD_STRING 43 | ``` 44 | 45 | With the payload constructed, you can then invoke the function directly in the AWS Console by defining a test event or using the AWS CLI with the payload written to a file defined in the $PAYLOAD_FILE enviornment variable. Replace the function name with the name of the function deployed by the serverless application repository (it should be `lambdium` by default). 46 | 47 | ``` 48 | aws lambda invoke --invocation-type RequestResponse --function-name lambdium --payload file://$PAYLOAD_FILE --log-type Tail $OUTPUT_FILE 49 | ``` 50 | 51 | ### troubleshooting 52 | 53 | Errors and failures will be written to Cloudwatch. The `DEBUG_ENV` environment variable outputs additional information about the filesystem and inputted Base64 script. 54 | 55 | The maximum payload size is 4kb and the default timeout is 10 seconds, so this won't work with very large scripts or scripts that run a long time. 56 | 57 | More details are available on the official Github page, https://github.com/smithclay/lambdium. 58 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## lambdium 2 | ### headless chrome + selenium webdriver in AWS Lambda 3 | 4 | **Lambdium uses Selenium Webdriver with [Headless Chromium](https://developers.google.com/web/updates/2017/04/headless-chrome) to run Webdriver scripts written in JavaScript on AWS Lambda.** Since this project was created, AWS now offers this as a completely managed service called [Device Farm Desktop Browser Testing](https://docs.aws.amazon.com/devicefarm/latest/testgrid/what-is-testgrid.html). 5 | 6 | You can use this AWS Lambda function by itself, bundled with your own application as a standalone AWS Lambda layer, or with other AWS services to: 7 | 8 | * Run many concurrent selenium scripts at the same time without worrying about the infrastructure 9 | * Run execute a selenium script via an HTTP call using API Gateway ([example app](/examples/apps/api-gateway.yaml)) 10 | * Configure Cloudwatch events to run a script on a schedule ([example app](/examples/apps/scheduled-event.yaml)) 11 | * Integrate selenium tests running in Chrome into different event-driven workflows (like CodeDeploy checks, webhooks, or uploads to an S3 bucket) 12 | 13 | Since this Lambda function is written using node.js, you can run almost any script written for [selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver). Example scripts can be found in the `examples` directory. 14 | 15 | This uses a special version of Chrome (headless chromium) from the [serverless-chrome](https://github.com/adieuadieu/serverless-chrome) project. 16 | 17 | *This is highly experimental and not all chromedriver functions will work. Check [issues](https://github.com/smithclay/lambdium/issues) for known issues.* 18 | 19 | #### Requirements and setup for local development 20 | 21 | *This project is on the [AWS Serverless Application Repository](https://serverlessrepo.aws.amazon.com), allowing you to install it in your AWS account with one click. Install in your AWS account [here](https://serverlessrepo.aws.amazon.com/#/applications/arn:aws:serverlessrepo:us-east-1:156280089524:applications~lambdium).* Quickstart instructions are in the [`README-SAR.md` file](https://github.com/smithclay/lambdium/blob/master/README-SAR.md). 22 | 23 | * An AWS Account 24 | * The [AWS SAM Local](https://github.com/awslabs/aws-sam-local) running functions locally with the [Serverless Application Model](https://github.com/awslabs/serverless-application-model) (see: `template.yaml`, install: `npm install -g aws-sam-local`) 25 | * node.js + npm 26 | * `modclean` npm modules for reducing function size 27 | * Bash 28 | 29 | #### Local development setup 30 | 31 | ##### 1. Fetching large binary dependencies 32 | 33 | The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. [Marco Lüthy](https://github.com/adieuadieu) has an excellent post on Medium about how he built chromium for for AWS Lambda [here](https://medium.com/@marco.luethy/running-headless-chrome-on-aws-lambda-fa82ad33a9eb). 34 | 35 | ```sh 36 | $ ./layer/fetch-binaries.sh 37 | ``` 38 | 39 | ##### 2. Building 40 | 41 | This is now handled by the `sam build` command. In the root of this project, run: 42 | 43 | ```sh 44 | $ sam build 45 | ``` 46 | 47 | ##### 3. Running locally with SAM Local 48 | 49 | SAM Local can run this function on your computer inside a Docker container that acts like AWS Lambda. To run the function with an example event trigger that uses selenium to use headless chromium to visit `google.com`, run this: 50 | 51 | ```sh 52 | $ sam local invoke Lambdium -e event.json 53 | ``` 54 | 55 | ### Deploying the function to AWS 56 | 57 | To deploy the function to your AWS account, you'll need to have followed the instructions above to fetch dependencies. Running it locally with SAM local and the test event (in `event.json`) is a good idea to verify everything works correctly before running it in the cloud. 58 | 59 | #### 1. Creating a S3 bucket for the function deployment 60 | 61 | This will create a file called `packaged.yaml` you can use with Cloudformation to deploy the function. 62 | 63 | You need to create a S3 bucket configured on your AWS account to upload the packed function files. For example: 64 | 65 | ```sh 66 | $ export LAMBDA_BUCKET_NAME=lambdium-upload-bucket 67 | ``` 68 | 69 | #### 2. Packaging the function for Cloudformation using SAM 70 | 71 | ```sh 72 | $ sam package --s3-bucket $LAMBDA_BUCKET_NAME > packaged.yaml 73 | ``` 74 | 75 | #### 3. Deploying the package using SAM 76 | 77 | This will create the function using Cloudformation after packaging it is complete. 78 | 79 | ```sh 80 | $ sam deploy --template-file ./packaged.yaml --stack-name <> --capabilities CAPABILITY_IAM 81 | ``` 82 | 83 | If set, the optional `DEBUG_ENV` environment variable will log additional information to Cloudwatch. 84 | 85 | ### Running the function 86 | 87 | Post-deploy, you can have AWS Lambda run a selenium script. There's an example of a selenium-webdriver simple script in the `examples/` directory that the Lambda function can now run. 88 | 89 | Expected JSON input for the function event trigger is: `{"Base64Script": ""}` (this can also be provided as an environment variable named `BASE64_SCRIPT`). 90 | 91 | To run the example Selenium script, you can use the example with the AWS CLI in the `scripts` directory. It takes care of base64 encoding the file and assumes the function name is `lambdium` running in `us-west-2`: 92 | 93 | ```sh 94 | $ scripts/invoke.sh 95 | ``` 96 | 97 | To use your own `selenium-webdriver` script: 98 | 99 | ```sh 100 | $ scripts/invoke.sh ~/Desktop/my-script.js 101 | ``` 102 | 103 | #### Related projects 104 | * [serverless-chrome](https://github.com/adieuadieu/serverless-chrome) 105 | * [How to Get Headless Chrome on Lambda by Marco Lüthy](https://medium.com/@marco.luethy/running-headless-chrome-on-aws-lambda-fa82ad33a9eb) 106 | * [Getting Started with Headless Chrome](https://developers.google.com/web/updates/2017/04/headless-chrome) 107 | * [Selenium Webdriver 3.0 Docs](https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index.html) 108 | -------------------------------------------------------------------------------- /examples/apps/api-gateway.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion : '2010-09-09' 2 | Transform: AWS::Serverless-2016-10-31 3 | Description: selenium with headless chromium 4 | Resources: 5 | Lambdium: 6 | Type: AWS::Serverless::Function 7 | Properties: 8 | Handler: index.postApiGatewayHandler 9 | Runtime: nodejs6.10 10 | FunctionName: lambdium 11 | Description: headless chromium running selenium 12 | # This needs to be fairly large: chromium needs a lot of memory 13 | MemorySize: 1156 14 | Timeout: 20 15 | Environment: 16 | Variables: 17 | CLEAR_TMP: "true" 18 | # packaged lambdium archive @ v0.2 19 | CodeUri: s3://lambdium-upload-bucket/690dd891bba09e8520a70cf68af7d904 20 | Events: 21 | RunScript: 22 | Properties: 23 | Method: POST 24 | Path: '/runScript' 25 | RestApiId: !Ref Api 26 | Type: Api 27 | # POST a selenium script file to this endpoint to execute the script: 28 | # curl -v -F "script=@examples/visitgoogle.js" <> 29 | Api: 30 | Type: AWS::Serverless::Api 31 | Properties: 32 | Name: RunScriptAPI 33 | StageName: Prod 34 | DefinitionBody: 35 | swagger: "2.0" 36 | schemes: 37 | - "https" 38 | paths: 39 | '/runScript': 40 | post: 41 | responses: {} 42 | x-amazon-apigateway-integration: 43 | uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Lambdium.Arn}/invocations 44 | passthroughBehavior: "when_no_match" 45 | httpMethod: "POST" 46 | type: aws_proxy 47 | x-amazon-apigateway-binary-media-types: 48 | - "*/*" -------------------------------------------------------------------------------- /examples/apps/scheduled-event.yaml: -------------------------------------------------------------------------------- 1 | 2 | # Visits the PAGE_URL every minute using Cloudwatch Events 3 | AWSTemplateFormatVersion: '2010-09-09' 4 | Transform: AWS::Serverless-2016-10-31 5 | Description: lambdium scheduled script example 6 | Resources: 7 | SyntheticTrafficFunction: 8 | Type: AWS::Serverless::Function 9 | Properties: 10 | # packaged lambdium archive @ v0.1.2 11 | CodeUri: s3://lambdium-upload-bucket/4fcc083b4f8302daf6c68d19001d8c8d 12 | Runtime: nodejs6.10 13 | Handler: index.handler 14 | MemorySize: 1154 15 | Timeout: 25 16 | Environment: 17 | Variables: 18 | PAGE_URL: "http://www.google.com" 19 | Events: 20 | CheckWebsiteScheduledEvent: 21 | Type: Schedule 22 | Properties: 23 | # Must follow this syntax: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html 24 | Schedule: rate(1 minute) 25 | # To specify a script directly, use the Input parameter and remove the PAGE_URL environment variable: 26 | # Input: '{"Base64Script": "JGJyb3dzZXIuZ2V0KCdodHR..."}' -------------------------------------------------------------------------------- /examples/cloudrace.js: -------------------------------------------------------------------------------- 1 | // Interact with a React SPA app. 2 | // This uses the selenium-webdriver 3.4 package. 3 | // Docs: https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index.html 4 | 5 | $browser.get('https://d3q2ykjqmzt1zl.cloudfront.net/').then(function(){ 6 | return $browser.findElement($driver.By.id('play-race')).then(function(element){ 7 | return element.click().then(function(){ 8 | $browser.wait(function() { 9 | return $driver.until.elementLocated($driver.By.xpath("//*[contains(text(),'#1')]")); 10 | }, 10000).then(function() { 11 | console.log('Race finished!'); 12 | $browser.close(); 13 | }); 14 | }); 15 | }); 16 | }); -------------------------------------------------------------------------------- /examples/visitgoogle.js: -------------------------------------------------------------------------------- 1 | // Sample selenimum-webdriver script that visits google.com 2 | // This uses the selenium-webdriver 3.4 package. 3 | // Docs: https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index.html 4 | // $browser = webdriver session 5 | // $driver = driver libraries 6 | // console.log will output to AWS Lambda logs (via Cloudwatch) 7 | 8 | console.log('About to visit google.com...'); 9 | $browser.get('http://www.google.com/ncr'); 10 | $browser.findElement($driver.By.name('btnK')).click(); 11 | $browser.wait($driver.until.titleIs('Google'), 1000); 12 | $browser.getTitle().then(function(title) { 13 | console.log("title is: " + title); 14 | console.log('Finished running script!'); 15 | }); 16 | -------------------------------------------------------------------------------- /layer/fetch-binaries.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | mkdir -p bin 4 | 5 | # Get headless shell 6 | curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-35/stable-headless-chromium-amazonlinux-2017-03.zip > chromeheadless.zip 7 | unzip chromeheadless.zip -d bin/ 8 | rm chromeheadless.zip 9 | 10 | # Get Chromedriver 11 | curl -SL https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip> chromedriver.zip 12 | unzip chromedriver.zip -d bin/ 13 | rm chromedriver.zip 14 | -------------------------------------------------------------------------------- /layer/lib/libORBit-2.so.0: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smithclay/lambdium/f11dd302c282e2a28e17e4b9f5c3ae7cbbe1f507/layer/lib/libORBit-2.so.0 -------------------------------------------------------------------------------- /layer/lib/libgconf-2.so.4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smithclay/lambdium/f11dd302c282e2a28e17e4b9f5c3ae7cbbe1f507/layer/lib/libgconf-2.so.4 -------------------------------------------------------------------------------- /src/event-api.json: -------------------------------------------------------------------------------- 1 | { 2 | "body": "LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS03NjgyM2JlYzc1MjY0MGQ0DQpDb250ZW50LURpc3Bvc2l0aW9uOiBmb3JtLWRhdGE7IG5hbWU9InNjcmlwdCI7IGZpbGVuYW1lPSJ2aXNpdGdvb2dsZS5qcyINCkNvbnRlbnQtVHlwZTogYXBwbGljYXRpb24vb2N0ZXQtc3RyZWFtDQoNCi8vIFNhbXBsZSBzZWxlbmltdW0td2ViZHJpdmVyIHNjcmlwdCB0aGF0IHZpc2l0cyBnb29nbGUuY29tCi8vIFRoaXMgdXNlcyB0aGUgc2VsZW5pdW0td2ViZHJpdmVyIDMuNCBwYWNrYWdlLgovLyBEb2NzOiBodHRwczovL3NlbGVuaXVtaHEuZ2l0aHViLmlvL3NlbGVuaXVtL2RvY3MvYXBpL2phdmFzY3JpcHQvbW9kdWxlL3NlbGVuaXVtLXdlYmRyaXZlci9pbmRleC5odG1sCi8vICRicm93c2VyID0gd2ViZHJpdmVyIHNlc3Npb24KLy8gJGRyaXZlciA9IGRyaXZlciBsaWJyYXJpZXMKLy8gY29uc29sZS5sb2cgd2lsbCBvdXRwdXQgdG8gQVdTIExhbWJkYSBsb2dzICh2aWEgQ2xvdWR3YXRjaCkKCmNvbnNvbGUubG9nKCdBYm91dCB0byB2aXNpdCBnb29nbGUuY29tLi4uJyk7CiRicm93c2VyLmdldCgnaHR0cDovL3d3dy5nb29nbGUuY29tL25jcicpOwokYnJvd3Nlci5maW5kRWxlbWVudCgkZHJpdmVyLkJ5Lm5hbWUoJ2J0bksnKSkuY2xpY2soKTsKJGJyb3dzZXIud2FpdCgkZHJpdmVyLnVudGlsLnRpdGxlSXMoJ0dvb2dsZScpLCAxMDAwKTsKJGJyb3dzZXIuZ2V0VGl0bGUoKS50aGVuKGZ1bmN0aW9uKHRpdGxlKSB7CiAgICBjb25zb2xlLmxvZygidGl0bGUgaXM6ICIgKyB0aXRsZSk7CiAgICBjb25zb2xlLmxvZygnRmluaXNoZWQgcnVubmluZyBzY3JpcHQhJyk7Cn0pOwoNCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tNzY4MjNiZWM3NTI2NDBkNC0tDQo=", 3 | "resource": "/{proxy+}", 4 | "requestContext": { 5 | "resourceId": "123456", 6 | "apiId": "1234567890", 7 | "resourcePath": "/{proxy+}", 8 | "httpMethod": "POST", 9 | "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", 10 | "accountId": "123456789012", 11 | "identity": { 12 | "apiKey": null, 13 | "userArn": null, 14 | "cognitoAuthenticationType": null, 15 | "caller": null, 16 | "userAgent": "Custom User Agent String", 17 | "user": null, 18 | "cognitoIdentityPoolId": null, 19 | "cognitoIdentityId": null, 20 | "cognitoAuthenticationProvider": null, 21 | "sourceIp": "127.0.0.1", 22 | "accountId": null 23 | }, 24 | "stage": "prod" 25 | }, 26 | "queryStringParameters": { 27 | "foo": "bar" 28 | }, 29 | "headers": { 30 | "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", 31 | "Accept-Language": "en-US,en;q=0.8", 32 | "CloudFront-Is-Desktop-Viewer": "true", 33 | "CloudFront-Is-SmartTV-Viewer": "false", 34 | "CloudFront-Is-Mobile-Viewer": "false", 35 | "X-Forwarded-For": "127.0.0.1, 127.0.0.2", 36 | "CloudFront-Viewer-Country": "US", 37 | "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", 38 | "Upgrade-Insecure-Requests": "1", 39 | "X-Forwarded-Port": "443", 40 | "Host": "1234567890.execute-api.us-east-1.amazonaws.com", 41 | "X-Forwarded-Proto": "https", 42 | "X-Amz-Cf-Id": "aaaaaaaaaae3VYQb9jd-nvCd-de396Uhbp027Y2JvkCPNLmGJHqlaA==", 43 | "CloudFront-Is-Tablet-Viewer": "false", 44 | "Cache-Control": "max-age=0", 45 | "Content-Type": "multipart/form-data; boundary=------------------------76823bec752640d4", 46 | "User-Agent": "Custom User Agent String", 47 | "CloudFront-Forwarded-Proto": "https", 48 | "Accept-Encoding": "gzip, deflate, sdch" 49 | }, 50 | "pathParameters": { 51 | "proxy": "/examplepath" 52 | }, 53 | "httpMethod": "POST", 54 | "stageVariables": { 55 | "baz": "qux" 56 | }, 57 | "isBase64Encoded": true, 58 | "path": "/examplepath" 59 | } -------------------------------------------------------------------------------- /src/event.json: -------------------------------------------------------------------------------- 1 | { 2 | "Base64Script": "Ly8gU2FtcGxlIHNlbGVuaW11bS13ZWJkcml2ZXIgc2NyaXB0IHRoYXQgdmlzaXRzIGdvb2dsZS5jb20KLy8gVGhpcyB1c2VzIHRoZSBzZWxlbml1bS13ZWJkcml2ZXIgMy40IHBhY2thZ2UuCi8vIERvY3M6IGh0dHBzOi8vc2VsZW5pdW1ocS5naXRodWIuaW8vc2VsZW5pdW0vZG9jcy9hcGkvamF2YXNjcmlwdC9tb2R1bGUvc2VsZW5pdW0td2ViZHJpdmVyL2luZGV4Lmh0bWwKLy8gJGJyb3dzZXIgPSB3ZWJkcml2ZXIgc2Vzc2lvbgovLyAkZHJpdmVyID0gZHJpdmVyIGxpYnJhcmllcwovLyBjb25zb2xlLmxvZyB3aWxsIG91dHB1dCB0byBBV1MgTGFtYmRhIGxvZ3MgKHZpYSBDbG91ZHdhdGNoKQoKY29uc29sZS5sb2coJ0Fib3V0IHRvIHZpc2l0IGdvb2dsZS5jb20uLi4nKTsKJGJyb3dzZXIuZ2V0KCdodHRwOi8vd3d3Lmdvb2dsZS5jb20vbmNyJyk7CiRicm93c2VyLmZpbmRFbGVtZW50KCRkcml2ZXIuQnkubmFtZSgnYnRuSycpKS5jbGljaygpOwokYnJvd3Nlci53YWl0KCRkcml2ZXIudW50aWwudGl0bGVJcygnR29vZ2xlJyksIDEwMDApOwokYnJvd3Nlci5nZXRUaXRsZSgpLnRoZW4oZnVuY3Rpb24odGl0bGUpIHsKICAgIGNvbnNvbGUubG9nKCJ0aXRsZSBpczogIiArIHRpdGxlKTsKfSk7CmNvbnNvbGUubG9nKCdGaW5pc2hlZCBydW5uaW5nIHNjcmlwdCEnKTs=" 3 | } -------------------------------------------------------------------------------- /src/index.js: -------------------------------------------------------------------------------- 1 | 2 | const chromium = require('./lib/chromium'); 3 | const sandbox = require('./lib/sandbox'); 4 | const log = require('lambda-log'); 5 | const apiHandler = require('./lib/api-handler'); 6 | 7 | if (process.env.DEBUG_ENV || process.env.SAM_LOCAL) { 8 | log.config.debug = true; 9 | log.config.dev = true; 10 | } 11 | 12 | log.info('Loading function'); 13 | 14 | // Create new reusable session (spawns chromium and webdriver) 15 | if (!process.env.CLEAN_SESSIONS) { 16 | $browser = chromium.createSession(); 17 | } 18 | 19 | // Handler for POST events from API gateway 20 | // curl -v -F "script=@examples/visitgoogle.js" <> 21 | exports.postApiGatewayHandler = apiHandler; 22 | 23 | // Default function event handler 24 | // Accepts events: 25 | // * {"Base64Script": "<>"} 26 | // * {"pageUrl": "<>"} 27 | // Accepts environment variables: 28 | // * BASE64_SCRIPT: encoded selenium script 29 | // * PAGE_URL: URI to visit 30 | exports.handler = (event, context, callback) => { 31 | $browser = sandbox.initBrowser(event, context); 32 | 33 | var opts = sandbox.buildOptions(event, $browser); 34 | 35 | sandbox.executeScript(opts, function(err) { 36 | if (process.env.LOG_DEBUG) { 37 | log.debug(child.execSync('ps aux').toString()); 38 | log.debug(child.execSync('cat /tmp/chromedriver.log').toString()) 39 | } 40 | if (err) { 41 | log.error(err); 42 | return callback(err, null); 43 | } 44 | 45 | callback(null, 'Finished executing script'); 46 | }); 47 | }; 48 | -------------------------------------------------------------------------------- /src/lib/api-handler.js: -------------------------------------------------------------------------------- 1 | const busboy = require('busboy'); 2 | const path = require('path'); 3 | const os = require('os'); 4 | 5 | const sandbox = require('./sandbox'); 6 | 7 | module.exports = function(event, context, callback) { 8 | $browser = sandbox.initBrowser(event, context); 9 | var errorMessage = ''; 10 | const response = { 11 | statusCode: 200, 12 | headers: { 13 | "Content-Type": 'application/text', 14 | "X-Error": errorMessage || null 15 | }, 16 | body: '', 17 | isBase64Encoded: false 18 | }; 19 | var body = event.body; 20 | if (event.isBase64Encoded) { 21 | body = Buffer.from(event.body, 'base64').toString('utf8'); 22 | } 23 | var scriptFile = new Buffer(0) 24 | 25 | 26 | const SCRIPT_FIELDNAME = 'script'; 27 | 28 | var contentType = event.headers['Content-Type'] || event.headers['content-type']; 29 | var bb = new busboy({ headers: { 'content-type': contentType }}); 30 | var result = {}; 31 | bb.on('file', function (fieldname, file, filename, encoding, mimetype) { 32 | file.on('data', data => { 33 | result.file = data; 34 | }); 35 | 36 | file.on('end', () => { 37 | result.filename = filename; 38 | result.contentType = mimetype; 39 | }); 40 | }) 41 | .on('finish', () => { 42 | 43 | // Execute uploaded script 44 | var scriptText = result.file.toString(); 45 | var opts = sandbox.buildOptions(event, $browser); 46 | opts.scriptText = scriptText; 47 | 48 | sandbox.executeScript(opts, function(err, output) { 49 | if (err) { 50 | response.headers['X-Error'] = err; 51 | response.body = err; 52 | response.statusCode = 500; 53 | return callback(null, response); 54 | } 55 | response.body = output; 56 | callback(null, response); 57 | }); 58 | }) 59 | .on('error', err => { 60 | response.headers['X-Error'] = err; 61 | response.body = err; 62 | response.statusCode = 500; 63 | callback(null, response); 64 | }); 65 | 66 | bb.end(body); 67 | }; -------------------------------------------------------------------------------- /src/lib/chromium.js: -------------------------------------------------------------------------------- 1 | const child = require('child_process'); 2 | const os = require('os'); 3 | const path = require('path'); 4 | const chrome = require('selenium-webdriver/chrome'); 5 | const webdriver = require('selenium-webdriver'); 6 | 7 | const CHROME_REMOTE_DEBUGGING_PORT = 9222; 8 | 9 | // TODO: Override the user agent? 10 | // http://peter.sh/experiments/chromium-command-line-switches/ 11 | const defaultChromeFlags = [ 12 | '--headless', // Redundant? 13 | //`--remote-debugging-port=${CHROME_REMOTE_DEBUGGING_PORT}`, 14 | 15 | '--disable-gpu', // TODO: should we do this? 16 | '--window-size=1280x1696', // Letter size 17 | '--no-sandbox', 18 | '--user-data-dir=/tmp/user-data', 19 | '--hide-scrollbars', 20 | '--enable-logging', 21 | '--log-level=0', 22 | '--v=99', 23 | '--single-process', 24 | '--data-path=/tmp/data-path', 25 | 26 | '--ignore-certificate-errors', // Dangerous? 27 | 28 | // '--no-zygote', // Disables the use of a zygote process for forking child processes. Instead, child processes will be forked and exec'd directly. Note that --no-sandbox should also be used together with this flag because the sandbox needs the zygote to work. 29 | 30 | '--homedir=/tmp', 31 | // '--media-cache-size=0', 32 | // '--disable-lru-snapshot-cache', 33 | // '--disable-setuid-sandbox', 34 | // '--disk-cache-size=0', 35 | '--disk-cache-dir=/tmp/cache-dir', 36 | // '--use-simple-cache-backend', 37 | // '--enable-low-end-device-mode', 38 | 39 | // '--trace-startup=*,disabled-by-default-memory-infra', 40 | //'--trace-startup=*', 41 | ]; 42 | 43 | const HEADLESS_CHROME_PATH = '/opt/bin/headless-chromium'; 44 | const CHROMEDRIVER_PATH = '/opt/bin/chromedriver'; 45 | exports.createSession = function() { 46 | var service; 47 | if (process.env.LOG_DEBUG || process.env.SAM_LOCAL) { 48 | service = new chrome.ServiceBuilder(CHROMEDRIVER_PATH) 49 | .loggingTo('/tmp/chromedriver.log') 50 | .build(); 51 | } else { 52 | service = new chrome.ServiceBuilder(CHROMEDRIVER_PATH) 53 | .build(); 54 | } 55 | 56 | const options = new chrome.Options(); 57 | 58 | const logPrefs = new webdriver.logging.Preferences(); 59 | logPrefs.setLevel(webdriver.logging.Type.PERFORMANCE, webdriver.logging.Level.ALL); 60 | options.setLoggingPrefs(logPrefs); 61 | 62 | options.setPerfLoggingPrefs({ enableNetwork: true, enablePage: true }); 63 | options.setChromeBinaryPath(HEADLESS_CHROME_PATH); 64 | options.addArguments(defaultChromeFlags); 65 | return chrome.Driver.createSession(options, service); 66 | } 67 | 68 | -------------------------------------------------------------------------------- /src/lib/sandbox.js: -------------------------------------------------------------------------------- 1 | const vm = require('vm'); 2 | const log = require('lambda-log'); 3 | const chromium = require('./chromium'); 4 | const webdriver = require('selenium-webdriver'); 5 | 6 | exports.initBrowser = function(event, context) { 7 | context.callbackWaitsForEmptyEventLoop = false; 8 | 9 | if (process.env.CLEAN_SESSIONS) { 10 | log.info('attempting to clear /tmp directory') 11 | log.info(child.execSync('rm -rf /tmp/core*').toString()); 12 | } 13 | 14 | log.info(`Received event: ${JSON.stringify(event, null, 2)}`); 15 | 16 | // Creates a new session on each event (instead of reusing for performance benefits) 17 | if (process.env.CLEAN_SESSIONS) { 18 | $browser = chromium.createSession(); 19 | } 20 | return $browser; 21 | }; 22 | 23 | exports.buildOptions = (event, browser) => { 24 | var opts = opts = { 25 | browser: $browser, 26 | driver: webdriver 27 | }; 28 | 29 | const inputParam = event.Base64Script || process.env.BASE64_SCRIPT; 30 | if (typeof inputParam !== 'string') { 31 | opts.pageUrl = event.pageUrl || process.env.PAGE_URL; 32 | return opts; 33 | } 34 | 35 | var inputBuffer = Buffer.from(inputParam, 'base64').toString('utf8'); 36 | opts.scriptText = inputBuffer; 37 | 38 | return opts; 39 | }; 40 | 41 | exports.executeScript = function(opts = {}, cb) { 42 | const browser = opts.browser; 43 | const driver = opts.driver; 44 | var output = ''; 45 | var scriptText = opts.scriptText; 46 | 47 | // Just visit a web page if a script isn't specified 48 | if (opts.pageUrl && scriptText === undefined) { 49 | scriptText = require('fs').readFileSync(require('path').join(__dirname, 'visit-page.js'), 'utf8').toString(); 50 | } 51 | log.info(`Executing script "${scriptText}"`); 52 | 53 | if (typeof scriptText !== 'string') { 54 | return cb('Error: no url or script found to execute.'); 55 | } 56 | 57 | var consoleWrapper = { 58 | log: function(){ 59 | var args = Array.prototype.slice.call(arguments); 60 | args.unshift('[lambdium-selenium]'); 61 | output = `${output}\n${args.join(' ')}`; 62 | console.log.apply(console, args); 63 | } 64 | }; 65 | 66 | // Create Sandbox VM 67 | const sandbox = { 68 | '$browser': browser, 69 | '$driver': driver, 70 | '$pageUrl': opts.pageUrl, 71 | 'console': consoleWrapper 72 | }; 73 | 74 | const script = new vm.Script(scriptText); 75 | // TODO: Set timeout options for VM context 76 | const scriptContext = new vm.createContext(sandbox); 77 | try { 78 | script.runInContext(scriptContext); 79 | } catch (e) { 80 | log.error(`[script error] ${e}`); 81 | return cb(e, null); 82 | } 83 | 84 | // https://github.com/GoogleChrome/puppeteer/issues/1825#issuecomment-372241101 85 | // Reuse existing session, likely some edge cases around this... 86 | if (process.env.CLEAN_SESSIONS) { 87 | browser.quit().then(function() { 88 | cb(null, output); 89 | }); 90 | } else { 91 | browser.manage().deleteAllCookies().then(function() { 92 | return browser.get('about:blank').then(function() { 93 | cb(null, output); 94 | }); 95 | }).catch(function(err) { 96 | cb(err, output); 97 | }); 98 | } 99 | } 100 | -------------------------------------------------------------------------------- /src/lib/visit-page.js: -------------------------------------------------------------------------------- 1 | // This is the default lambdium webdriver script: it just visits a page url. 2 | console.log($pageUrl); 3 | $browser.get($pageUrl).then(function() { 4 | console.log('visited page', $pageUrl); 5 | }).catch(function() { 6 | console.log('error visiting page', $pageUrl); 7 | }); 8 | -------------------------------------------------------------------------------- /src/package-lock.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "lambdium", 3 | "version": "0.2.0", 4 | "lockfileVersion": 1, 5 | "requires": true, 6 | "dependencies": { 7 | "balanced-match": { 8 | "version": "0.4.2", 9 | "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-0.4.2.tgz", 10 | "integrity": "sha1-yz8+PHMtwPAe5wtAPzAuYddwmDg=" 11 | }, 12 | "brace-expansion": { 13 | "version": "1.1.7", 14 | "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.7.tgz", 15 | "integrity": "sha1-Pv/DxQ4ABTH7cg6v+A8K6O8jz1k=", 16 | "requires": { 17 | "balanced-match": "^0.4.1", 18 | "concat-map": "0.0.1" 19 | } 20 | }, 21 | "busboy": { 22 | "version": "0.2.14", 23 | "resolved": "https://registry.npmjs.org/busboy/-/busboy-0.2.14.tgz", 24 | "integrity": "sha1-bCpiLvz0fFe7vh4qnDetNseSVFM=", 25 | "requires": { 26 | "dicer": "0.2.5", 27 | "readable-stream": "1.1.x" 28 | }, 29 | "dependencies": { 30 | "isarray": { 31 | "version": "0.0.1", 32 | "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", 33 | "integrity": "sha1-ihis/Kmo9Bd+Cav8YDiTmwXR7t8=" 34 | }, 35 | "readable-stream": { 36 | "version": "1.1.14", 37 | "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz", 38 | "integrity": "sha1-fPTFTvZI44EwhMY23SB54WbAgdk=", 39 | "requires": { 40 | "core-util-is": "~1.0.0", 41 | "inherits": "~2.0.1", 42 | "isarray": "0.0.1", 43 | "string_decoder": "~0.10.x" 44 | } 45 | } 46 | } 47 | }, 48 | "concat-map": { 49 | "version": "0.0.1", 50 | "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz", 51 | "integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=" 52 | }, 53 | "core-js": { 54 | "version": "2.3.0", 55 | "resolved": "https://registry.npmjs.org/core-js/-/core-js-2.3.0.tgz", 56 | "integrity": "sha1-+rg/uwstjchfpjbEudNMdUIMbWU=" 57 | }, 58 | "core-util-is": { 59 | "version": "1.0.2", 60 | "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz", 61 | "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac=" 62 | }, 63 | "dicer": { 64 | "version": "0.2.5", 65 | "resolved": "https://registry.npmjs.org/dicer/-/dicer-0.2.5.tgz", 66 | "integrity": "sha1-WZbAhrszIYyBLAkL3cCc0S+stw8=", 67 | "requires": { 68 | "readable-stream": "1.1.x", 69 | "streamsearch": "0.1.2" 70 | }, 71 | "dependencies": { 72 | "isarray": { 73 | "version": "0.0.1", 74 | "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", 75 | "integrity": "sha1-ihis/Kmo9Bd+Cav8YDiTmwXR7t8=" 76 | }, 77 | "readable-stream": { 78 | "version": "1.1.14", 79 | "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz", 80 | "integrity": "sha1-fPTFTvZI44EwhMY23SB54WbAgdk=", 81 | "requires": { 82 | "core-util-is": "~1.0.0", 83 | "inherits": "~2.0.1", 84 | "isarray": "0.0.1", 85 | "string_decoder": "~0.10.x" 86 | } 87 | } 88 | } 89 | }, 90 | "es6-promise": { 91 | "version": "3.0.2", 92 | "resolved": "https://registry.npmjs.org/es6-promise/-/es6-promise-3.0.2.tgz", 93 | "integrity": "sha1-AQ1YWEI6XxGJeWZfRkhqlcbuK7Y=" 94 | }, 95 | "fs.realpath": { 96 | "version": "1.0.0", 97 | "resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz", 98 | "integrity": "sha1-FQStJSMVjKpA20onh8sBQRmU6k8=" 99 | }, 100 | "glob": { 101 | "version": "7.1.2", 102 | "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.2.tgz", 103 | "integrity": "sha512-MJTUg1kjuLeQCJ+ccE4Vpa6kKVXkPYJ2mOCQyUuKLcLQsdrMCpBPUi8qVE6+YuaJkozeA9NusTAw3hLr8Xe5EQ==", 104 | "requires": { 105 | "fs.realpath": "^1.0.0", 106 | "inflight": "^1.0.4", 107 | "inherits": "2", 108 | "minimatch": "^3.0.4", 109 | "once": "^1.3.0", 110 | "path-is-absolute": "^1.0.0" 111 | } 112 | }, 113 | "immediate": { 114 | "version": "3.0.6", 115 | "resolved": "https://registry.npmjs.org/immediate/-/immediate-3.0.6.tgz", 116 | "integrity": "sha1-nbHb0Pr43m++D13V5Wu2BigN5ps=" 117 | }, 118 | "inflight": { 119 | "version": "1.0.6", 120 | "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", 121 | "integrity": "sha1-Sb1jMdfQLQwJvJEKEHW6gWW1bfk=", 122 | "requires": { 123 | "once": "^1.3.0", 124 | "wrappy": "1" 125 | } 126 | }, 127 | "inherits": { 128 | "version": "2.0.3", 129 | "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz", 130 | "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=" 131 | }, 132 | "isarray": { 133 | "version": "1.0.0", 134 | "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", 135 | "integrity": "sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE=" 136 | }, 137 | "jszip": { 138 | "version": "3.1.5", 139 | "resolved": "https://registry.npmjs.org/jszip/-/jszip-3.1.5.tgz", 140 | "integrity": "sha512-5W8NUaFRFRqTOL7ZDDrx5qWHJyBXy6velVudIzQUSoqAAYqzSh2Z7/m0Rf1QbmQJccegD0r+YZxBjzqoBiEeJQ==", 141 | "requires": { 142 | "core-js": "~2.3.0", 143 | "es6-promise": "~3.0.2", 144 | "lie": "~3.1.0", 145 | "pako": "~1.0.2", 146 | "readable-stream": "~2.0.6" 147 | } 148 | }, 149 | "lambda-log": { 150 | "version": "1.3.0", 151 | "resolved": "https://registry.npmjs.org/lambda-log/-/lambda-log-1.3.0.tgz", 152 | "integrity": "sha1-b2YnzkOjLhT1WN826htcGUMGuV0=" 153 | }, 154 | "lie": { 155 | "version": "3.1.1", 156 | "resolved": "https://registry.npmjs.org/lie/-/lie-3.1.1.tgz", 157 | "integrity": "sha1-mkNrLMd0bKWd56QfpGmz77dr2H4=", 158 | "requires": { 159 | "immediate": "~3.0.5" 160 | } 161 | }, 162 | "lodash": { 163 | "version": "4.17.11", 164 | "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.11.tgz", 165 | "integrity": "sha512-cQKh8igo5QUhZ7lg38DYWAxMvjSAKG0A8wGSVimP07SIUEK2UO+arSRKbRZWtelMtN5V0Hkwh5ryOto/SshYIg==" 166 | }, 167 | "minimatch": { 168 | "version": "3.0.4", 169 | "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz", 170 | "integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==", 171 | "requires": { 172 | "brace-expansion": "^1.1.7" 173 | } 174 | }, 175 | "once": { 176 | "version": "1.4.0", 177 | "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", 178 | "integrity": "sha1-WDsap3WWHUsROsF9nFC6753Xa9E=", 179 | "requires": { 180 | "wrappy": "1" 181 | } 182 | }, 183 | "os-tmpdir": { 184 | "version": "1.0.2", 185 | "resolved": "https://registry.npmjs.org/os-tmpdir/-/os-tmpdir-1.0.2.tgz", 186 | "integrity": "sha1-u+Z0BseaqFxc/sdm/lc0VV36EnQ=" 187 | }, 188 | "pako": { 189 | "version": "1.0.6", 190 | "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.6.tgz", 191 | "integrity": "sha512-lQe48YPsMJAig+yngZ87Lus+NF+3mtu7DVOBu6b/gHO1YpKwIj5AWjZ/TOS7i46HD/UixzWb1zeWDZfGZ3iYcg==" 192 | }, 193 | "path-is-absolute": { 194 | "version": "1.0.1", 195 | "resolved": "https://registry.npmjs.org/path-is-absolute/-/path-is-absolute-1.0.1.tgz", 196 | "integrity": "sha1-F0uSaHNVNP+8es5r9TpanhtcX18=" 197 | }, 198 | "process-nextick-args": { 199 | "version": "1.0.7", 200 | "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-1.0.7.tgz", 201 | "integrity": "sha1-FQ4gt1ZZCtP5EJPyWk8q2L/zC6M=" 202 | }, 203 | "readable-stream": { 204 | "version": "2.0.6", 205 | "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.0.6.tgz", 206 | "integrity": "sha1-j5A0HmilPMySh4jaz80Rs265t44=", 207 | "requires": { 208 | "core-util-is": "~1.0.0", 209 | "inherits": "~2.0.1", 210 | "isarray": "~1.0.0", 211 | "process-nextick-args": "~1.0.6", 212 | "string_decoder": "~0.10.x", 213 | "util-deprecate": "~1.0.1" 214 | } 215 | }, 216 | "rimraf": { 217 | "version": "2.6.1", 218 | "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.6.1.tgz", 219 | "integrity": "sha1-wjOOxkPfeht/5cVPqG9XQopV8z0=", 220 | "requires": { 221 | "glob": "^7.0.5" 222 | } 223 | }, 224 | "sax": { 225 | "version": "1.2.2", 226 | "resolved": "https://registry.npmjs.org/sax/-/sax-1.2.2.tgz", 227 | "integrity": "sha1-/YYxojvHgmvvXYcb24c3jJVkeCg=" 228 | }, 229 | "selenium-webdriver": { 230 | "version": "3.6.0", 231 | "resolved": "https://registry.npmjs.org/selenium-webdriver/-/selenium-webdriver-3.6.0.tgz", 232 | "integrity": "sha512-WH7Aldse+2P5bbFBO4Gle/nuQOdVwpHMTL6raL3uuBj/vPG07k6uzt3aiahu352ONBr5xXh0hDlM3LhtXPOC4Q==", 233 | "requires": { 234 | "jszip": "^3.1.3", 235 | "rimraf": "^2.5.4", 236 | "tmp": "0.0.30", 237 | "xml2js": "^0.4.17" 238 | } 239 | }, 240 | "streamsearch": { 241 | "version": "0.1.2", 242 | "resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-0.1.2.tgz", 243 | "integrity": "sha1-gIudDlb8Jz2Am6VzOOkpkZoanxo=" 244 | }, 245 | "string_decoder": { 246 | "version": "0.10.31", 247 | "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", 248 | "integrity": "sha1-YuIDvEF2bGwoyfyEMB2rHFMQ+pQ=" 249 | }, 250 | "tmp": { 251 | "version": "0.0.30", 252 | "resolved": "https://registry.npmjs.org/tmp/-/tmp-0.0.30.tgz", 253 | "integrity": "sha1-ckGdSovn1s51FI/YsyTlk6cRwu0=", 254 | "requires": { 255 | "os-tmpdir": "~1.0.1" 256 | } 257 | }, 258 | "util-deprecate": { 259 | "version": "1.0.2", 260 | "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", 261 | "integrity": "sha1-RQ1Nyfpw3nMnYvvS1KKJgUGaDM8=" 262 | }, 263 | "wrappy": { 264 | "version": "1.0.2", 265 | "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", 266 | "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=" 267 | }, 268 | "xml2js": { 269 | "version": "0.4.17", 270 | "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.4.17.tgz", 271 | "integrity": "sha1-F76T6q4/O3eTWceVtBlwWogX6Gg=", 272 | "requires": { 273 | "sax": ">=0.6.0", 274 | "xmlbuilder": "^4.1.0" 275 | } 276 | }, 277 | "xmlbuilder": { 278 | "version": "4.2.1", 279 | "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-4.2.1.tgz", 280 | "integrity": "sha1-qlijBBoGb5DqoWwvU4n/GfP0YaU=", 281 | "requires": { 282 | "lodash": "^4.0.0" 283 | } 284 | } 285 | } 286 | } 287 | -------------------------------------------------------------------------------- /src/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "lambdium", 3 | "version": "0.2.0", 4 | "description": "headless chromium in lambda prototype", 5 | "main": "index.js", 6 | "scripts": { 7 | "test": "echo \"Error: no test specified\" && exit 1" 8 | }, 9 | "author": "Clay Smith", 10 | "license": "ISC", 11 | "dependencies": { 12 | "busboy": "^0.2.14", 13 | "lambda-log": "^1.3.0", 14 | "selenium-webdriver": "^3.6.0" 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /src/scripts/fetch-dependencies.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Install dependencies 4 | npm install 5 | mkdir -p bin 6 | 7 | if ! [ -x "$(command -v modclean)" ]; then 8 | echo 'Error: modclean is not installed. To install: npm i -g modclean' >&2 9 | exit 1 10 | fi 11 | 12 | # Reduce size of node_modules directory 13 | modclean --patterns="default:*" 14 | 15 | # Get headless shell 16 | curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-35/stable-headless-chromium-amazonlinux-2017-03.zip > chromeheadless.zip 17 | unzip chromeheadless.zip -d bin/ 18 | rm chromeheadless.zip 19 | 20 | # Get Chromedriver 21 | curl -SL https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip> chromedriver.zip 22 | unzip chromedriver.zip -d bin/ 23 | rm chromedriver.zip 24 | -------------------------------------------------------------------------------- /src/scripts/invoke.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | DEFAULT_SCRIPT="examples/visitgoogle.js" 3 | SELENIUM_SCRIPT=${1:-$DEFAULT_SCRIPT} 4 | 5 | OUTPUT_FILE=$(mktemp) 6 | PAYLOAD_FILE=$(mktemp) 7 | 8 | echo "Preparing $SELENIUM_SCRIPT for execution..." 9 | 10 | BASE64_ENCODED=`cat $SELENIUM_SCRIPT | openssl base64` 11 | echo $BASE64_ENCODED 12 | echo 'Invoking function...' 13 | 14 | PAYLOAD_STRING='{"Base64Script": "'$BASE64_ENCODED'"}' 15 | echo $PAYLOAD_STRING > $PAYLOAD_FILE 16 | 17 | aws lambda invoke --invocation-type RequestResponse --function-name lambdium --payload file://$PAYLOAD_FILE --region us-west-2 --log-type Tail $OUTPUT_FILE 18 | 19 | cat $OUTPUT_FILE 20 | 21 | rm $PAYLOAD_FILE 22 | rm $OUTPUT_FILE 23 | -------------------------------------------------------------------------------- /src/tests/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smithclay/lambdium/f11dd302c282e2a28e17e4b9f5c3ae7cbbe1f507/src/tests/.gitkeep -------------------------------------------------------------------------------- /template.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion : '2010-09-09' 2 | Transform: AWS::Serverless-2016-10-31 3 | Description: selenium with headless chromium 4 | Resources: 5 | DeploymentPermission: 6 | Type: "AWS::Lambda::LayerVersionPermission" 7 | Properties: 8 | Action: lambda:GetLayerVersion 9 | LayerVersionArn: !Ref ChromiumLayer 10 | Principal: '*' 11 | 12 | ChromiumLayer: 13 | Type: AWS::Serverless::LayerVersion 14 | Properties: 15 | LayerName: chromium-selenium-layer 16 | Description: Headless Chromium and Selenium WebDriver 17 | ContentUri: ./layer 18 | CompatibleRuntimes: 19 | - nodejs8.10 20 | - python3.7 21 | - python2.7 22 | - go1.x 23 | - java8 24 | LicenseInfo: 'MIT' 25 | RetentionPolicy: Retain 26 | 27 | Lambdium: 28 | Type: AWS::Serverless::Function 29 | Properties: 30 | Handler: index.handler 31 | Runtime: nodejs8.10 32 | FunctionName: lambdium 33 | Description: headless chromium running selenium 34 | # This needs to be fairly large: chromium needs a lot of memory 35 | MemorySize: 1156 36 | Timeout: 20 37 | Layers: 38 | - !Ref ChromiumLayer 39 | Environment: 40 | Variables: 41 | CLEAR_TMP: "true" 42 | CodeUri: ./src 43 | --------------------------------------------------------------------------------