Skip to content

switch to SAM local from terraform #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 25, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ lambda_function.zip
build/
bin/
node_modules
packaged.yaml

22 changes: 0 additions & 22 deletions Makefile

This file was deleted.

61 changes: 43 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,50 +18,75 @@ Since this Lambda function is written using node.js, you can run almost any scri
#### Requirements

* An AWS Account
* [Terraform](https://terraform.io) (optional but highly recommended for function creation and deploy)
* [AWS SAM Local](https://github.com/awslabs/aws-sam-local) AWS tool for running functions locally [Serverless Application Model](https://github.com/awslabs/serverless-application-model)
* node.js + npm
* Bash
* `make`

#### Installing `modclean`
#### Fetching dependencies

Lambdium uses the `modclean` NPM package to reduce the size of the `node_modules` directory. To install it, run:
The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. [Marco Lüthy](https://github.com/adieuadieu) has an excellent post on Medium about how he built chromium for for AWS Lambda [here](https://medium.com/@marco.luethy/running-headless-chrome-on-aws-lambda-fa82ad33a9eb).

```sh
npm i -g modclean
$ ./scripts/fetch-dependencies.sh
```

#### Fetching dependencies
#### Running locally with SAM

The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. [Marco Lüthy](https://github.com/adieuadieu) has an excellent post on Medium about how he built chromium for for AWS Lambda [here](https://medium.com/@marco.luethy/running-headless-chrome-on-aws-lambda-fa82ad33a9eb).
```sh
$ sam local invoke Lambdium -e event.json
```

### Deploying to

#### Creating a bucket for the function deployment

This will create a file called `packaged.yaml` you can use with Cloudformation to deploy the function.

You need to have an S3 bucket configured on your AWS account to upload the packed function files. For example:

```sh
$ ./scripts/fetch-dependencies.sh
$ export LAMBDA_BUCKET_NAME=lambdium-upload-bucket
```

##### Reducing function size for performance (and faster uploads!)

It's a good idea to clean the `node_modules` directory before packaging to make the function size significantly smaller (making the function run faster!). You can do this using the `modclean` package:

To install it:

```sh
$ npm i -g modclean
```

Then, run:

```sh
$ modclean --patterns="default:*"
```

#### Building Lambda `.zip` archive
##### Packaging the function for Cloudformation using SAM

```sh
$ make lambda
$ sam package --template-file template.yaml --s3-bucket $LAMBDA_BUCKET_NAME --output-template-file packaged.yaml
```

#### Creating and Deploying Using Terraform
#### Creating and Deploying Using SAM

This will create the function using Terraform with all required permissions.
This will create the function using Cloudformation after packaging it is complete.

```sh
$ make deploy
$ sam deploy --template-file ./packaged.yaml --stack-name <<your-cloudformation-stack-name>> --capabilities CAPABILITY_IAM
```

The optional `DEBUG_ENV` environment variable will log additional information to Cloudwatch. The `PATH` environment variable points to the `bin` directory in this project—this is required in order to launch the `chromedriver` process.
If set, the optional `DEBUG_ENV` environment variable will log additional information to Cloudwatch.

## Usage
## Invoking the function

If dependencies are installed and `make deploy` succeeds you can have Lambda run a script. There's an example of a selenium-webdriver simple script in the `examples/` directory that the Lambda function can now run.
Post-deploy, you can have lambda run a Webdriver script. There's an example of a selenium-webdriver simple script in the `examples/` directory that the Lambda function can now run.

Expected JSON input for the function: `{"Base64Script": "<Base64 Encoding of Selenium Script>"}` (this can also be provided as an environment variable named `BASE64_SCRIPT`).
Expected JSON input for the function is: `{"Base64Script": "<Base64 Encoding of Selenium Script>"}` (this can also be provided as an environment variable named `BASE64_SCRIPT`).

To run the example Selenium script, you can use the example with the AWS CLI in the `scripts` directory. It takes care of base64 encoding the file:
To run the example Selenium script, you can use the example with the AWS CLI in the `scripts` directory. It takes care of base64 encoding the file and assumes the function name is `lambdium` running in `us-west-2`:

```sh
$ scripts/invoke.sh
Expand Down
3 changes: 3 additions & 0 deletions event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"Base64Script": "Ly8gU2FtcGxlIHNlbGVuaW11bS13ZWJkcml2ZXIgc2NyaXB0IHRoYXQgdmlzaXRzIGdvb2dsZS5jb20KLy8gVGhpcyB1c2VzIHRoZSBzZWxlbml1bS13ZWJkcml2ZXIgMy40IHBhY2thZ2UuCi8vIERvY3M6IGh0dHBzOi8vc2VsZW5pdW1ocS5naXRodWIuaW8vc2VsZW5pdW0vZG9jcy9hcGkvamF2YXNjcmlwdC9tb2R1bGUvc2VsZW5pdW0td2ViZHJpdmVyL2luZGV4Lmh0bWwKLy8gJGJyb3dzZXIgPSB3ZWJkcml2ZXIgc2Vzc2lvbgovLyAkZHJpdmVyID0gZHJpdmVyIGxpYnJhcmllcwovLyBjb25zb2xlLmxvZyB3aWxsIG91dHB1dCB0byBBV1MgTGFtYmRhIGxvZ3MgKHZpYSBDbG91ZHdhdGNoKQoKY29uc29sZS5sb2coJ0Fib3V0IHRvIHZpc2l0IGdvb2dsZS5jb20uLi4nKTsKJGJyb3dzZXIuZ2V0KCdodHRwOi8vd3d3Lmdvb2dsZS5jb20vbmNyJyk7CiRicm93c2VyLmZpbmRFbGVtZW50KCRkcml2ZXIuQnkubmFtZSgnYnRuSycpKS5jbGljaygpOwokYnJvd3Nlci53YWl0KCRkcml2ZXIudW50aWwudGl0bGVJcygnR29vZ2xlJyksIDEwMDApOwokYnJvd3Nlci5nZXRUaXRsZSgpLnRoZW4oZnVuY3Rpb24odGl0bGUpIHsKICAgIGNvbnNvbGUubG9nKCJ0aXRsZSBpczogIiArIHRpdGxlKTsKfSk7CmNvbnNvbGUubG9nKCdGaW5pc2hlZCBydW5uaW5nIHNjcmlwdCEnKTs="
}
2 changes: 1 addition & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ console.log('Loading function');

exports.handler = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;

if (process.env.CLEAR_TMP) {
log('attempting to clear /tmp directory')
log(child.execSync('rm -rf /tmp/core*').toString());
Expand Down
82 changes: 0 additions & 82 deletions lambda.tf

This file was deleted.

4 changes: 2 additions & 2 deletions lib/chromium.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ const defaultChromeFlags = [
];

const HEADLESS_CHROME_PATH = 'bin/headless_shell';

const CHROMEDRIVER_PATH = '/var/task/bin/chromedriver';
exports.createSession = function() {
const service = new chrome.ServiceBuilder()
const service = new chrome.ServiceBuilder(CHROMEDRIVER_PATH)
.enableVerboseLogging()
.build();

Expand Down
64 changes: 53 additions & 11 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "lambdium",
"version": "0.0.2",
"version": "0.1.0",
"description": "headless chromium in lambda prototype",
"main": "index.js",
"scripts": {
Expand Down
18 changes: 18 additions & 0 deletions template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: headless chromium running selenium
Resources:
Lambdium:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
Runtime: nodejs6.10
FunctionName: lambdium
Description: headless chromium running selenium
MemorySize: 1024
Timeout: 10
Environment:
Variables:
DEBUG_ENV: "true"
CLEAR_TMP: "true"
CodeUri: .