Merge pull request #24 from smithclay/layers

Add support for AWS Lambda Layers

Author: smithclay

README.md

@@ -1,9 +1,9 @@
 ## lambdium
 ### headless chrome + selenium webdriver in AWS Lambda
 
-**Lambdium allows you to run a Selenium Webdriver script written in Javascript inside of an AWS Lambda function bundled with [Headless Chromium](https://developers.google.com/web/updates/2017/04/headless-chrome).**
+**Lambdium uses Selenium Webdriver with [Headless Chromium](https://developers.google.com/web/updates/2017/04/headless-chrome) to run Webdriver scripts written in JavaScript on AWS Lambda.**
 
-You can use this AWS Lambda function by itself or with other AWS services to:
+You can use this AWS Lambda function by itself, bundled with your own application as a standalone AWS Lambda layer, or with other AWS services to:
 
 * Run many concurrent selenium scripts at the same time without worrying about the infrastructure
 * Run execute a selenium script via an HTTP call using API Gateway ([example app](/examples/apps/api-gateway.yaml))
@@ -33,27 +33,17 @@ This uses a special version of Chrome (headless chromium) from the [serverless-c
 The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. [Marco Lüthy](https://github.com/adieuadieu) has an excellent post on Medium about how he built chromium for for AWS Lambda [here](https://medium.com/@marco.luethy/running-headless-chrome-on-aws-lambda-fa82ad33a9eb). 
 
 ```sh
-    $ ./scripts/fetch-dependencies.sh
+    $ ./layer/fetch-binaries.sh
 ```
 
-##### 2. Cleaning up the `node_modules` directory to reduce function size
+##### 2. Building
 
-It's a good idea to clean the `node_modules` directory before packaging to make the function size significantly smaller (making the function run faster!). You can do this using the `modclean` package:
-
-To install it:
-
-```sh
-    $ npm i -g modclean
-```
-
-Then, run: 
+This is now handled by the `sam build` command. In the root of this project, run:
 
 ```sh
-    $ modclean --patterns="default:*"
+    $ sam build
 ```
 
-Follow the prompts and choose 'Y' to remove extraneous files from `node_modules`.
-
 ##### 3. Running locally with SAM Local
 
 SAM Local can run this function on your computer inside a Docker container that acts like AWS Lambda. To run the function with an example event trigger that uses selenium to use headless chromium to visit `google.com`, run this:
@@ -79,7 +69,7 @@ You need to create a S3 bucket configured on your AWS account to upload the pack
 #### 2. Packaging the function for Cloudformation using SAM
 
 ```sh
-    $ sam package --template-file template.yaml --s3-bucket $LAMBDA_BUCKET_NAME --output-template-file packaged.yaml
+    $ sam package --s3-bucket $LAMBDA_BUCKET_NAME > packaged.yaml
 ```
 
 #### 3. Deploying the package using SAM

layer/fetch-binaries.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+mkdir -p bin
+
+# Get headless shell
+curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-35/stable-headless-chromium-amazonlinux-2017-03.zip > chromeheadless.zip
+unzip chromeheadless.zip -d bin/
+rm chromeheadless.zip
+
+# Get Chromedriver
+curl -SL https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip> chromedriver.zip
+unzip chromedriver.zip -d bin/
+rm chromedriver.zip

lib/libORBit-2.so.0 renamed to layer/lib/libORBit-2.so.0

@@ -40,8 +40,8 @@ const defaultChromeFlags = [
   //'--trace-startup=*',
 ];
 
-const HEADLESS_CHROME_PATH = 'bin/headless-chromium';
-const CHROMEDRIVER_PATH = '/var/task/bin/chromedriver';
+const HEADLESS_CHROME_PATH = '/opt/bin/headless-chromium';
+const CHROMEDRIVER_PATH = '/opt/bin/chromedriver';
 exports.createSession = function() {
   var service;
   if (process.env.LOG_DEBUG || process.env.SAM_LOCAL) {
@@ -60,7 +60,7 @@ exports.createSession = function() {
   options.setLoggingPrefs(logPrefs);
 
   options.setPerfLoggingPrefs({ enableNetwork: true, enablePage: true });
-  options.setChromeBinaryPath(path.join(process.env.LAMBDA_TASK_ROOT, HEADLESS_CHROME_PATH));
+  options.setChromeBinaryPath(HEADLESS_CHROME_PATH);
   options.addArguments(defaultChromeFlags);
   return chrome.Driver.createSession(options, service);
 }

lib/libgconf-2.so.4 renamed to layer/lib/libgconf-2.so.4

@@ -2,17 +2,41 @@ AWSTemplateFormatVersion : '2010-09-09'
 Transform: AWS::Serverless-2016-10-31
 Description: selenium with headless chromium
 Resources:
+  DeploymentPermission:
+    Type: "AWS::Lambda::LayerVersionPermission"
+    Properties:
+      Action: lambda:GetLayerVersion
+      LayerVersionArn: !Ref ChromiumLayer
+      Principal: '*'
+
+  ChromiumLayer:
+    Type: AWS::Serverless::LayerVersion
+    Properties:
+      LayerName: chromium-selenium-layer
+      Description: Headless Chromium and Selenium WebDriver
+      ContentUri: ./layer
+      CompatibleRuntimes:
+        - nodejs8.10
+        - python3.7
+        - python2.7
+        - go1.x
+        - java8
+      LicenseInfo: 'MIT'
+      RetentionPolicy: Retain
+
   Lambdium:
     Type: AWS::Serverless::Function
     Properties:
       Handler: index.handler
-      Runtime: nodejs6.10
+      Runtime: nodejs8.10
       FunctionName: lambdium
       Description: headless chromium running selenium
       # This needs to be fairly large: chromium needs a lot of memory
       MemorySize: 1156
       Timeout: 20
+      Layers:
+        - !Ref ChromiumLayer
       Environment:
         Variables:
           CLEAR_TMP: "true"
-      CodeUri: .
+      CodeUri: ./src