Environment dev, stage, prod variables configuration example

[jgardezi]

Hi,

Can you anyone provide an examples for Development, Staging and Production variables using this documentation https://serverless.com/framework/docs/providers/aws/guide/variables/.

Kind regards,
Javed Gardezi

[nikgraf]

Hey @jgardezi, what's your use-case? You just want to have different variables depending on the stage?

[jgardezi]

@nikgraf yes that is my use case. Need different variables depending on the stage. If possible can you please post example.

[nikgraf]

might take me a while to get to it, currently catching up with issues and PRs

[oliverlloyd]

As I understand it, I think the OP wants to be able to deploy to, say, stage test, and have that automatically change the value for certain ENV VARs.

@nikgraf Do you mind if I try and answer this and you tell me if I'm close or not?

So:
My function calls some endpoint to get some info. I have Dev, Test and Prod endpoints like:

dev.infoprovider.com
test.infoprovider.com
prod.infoprovider.com

If I use the config below will that automatically route requests based on the stage / environment?

service: my-function-stuff

frameworkVersion: ">=1.2.0 <2.0.0"

provider:
  name: aws
  runtime: nodejs4.3
  environment:
    ENDPOINT: "${opt:stage}.infoprovider.com"

functions:
  getInfo:
    handler: handler.getInfo
...

In general, this seems a pretty standard use case but it's an area where naming semantics confuse things and it can take a while before you get to the 'aha' moment so, yeah, a simple example would reduce the time-to-aha for a lot of people.

[quantuminformation]

Theres a lot of examples using env variables and opt, but the Readme's don't cover those steps. Could be some improvements made this way to : https://github.com/serverless/examples/tree/master/aws-node-rest-api-with-dynamodb

[jgardezi]

Below are the steps I have done to solve this.

1- Create env.yml file. The structure of this file looks like this.
`
// env.yml
local:
host: localhost
port: 5432
name: mydb
user: root
password: secret

dev:
host: db.myserver.com
port: 5432
name: mydb
user: root
password: secret
`

In the env.yml file there are two environment local and dev. local is testing on local development environment variables. dev is for the cloud hosting environment.

2- To reference env.yml in serverless.yml so that we deploy microservice based on the env variables i.e. sls deploy --s local or sls deploy --s dev

// serverless.yml provider: stage: local environment: ${file(./env.yml):${opt:stage, self:provider.stage}}

That's all we need, to configure the multi-env. I have customised it further so that we don't need to run sls deploy --s local. The environment picks automatically local and all we need is sls deploy command.

3- Lastly, if you need to refer the variables in .js files all you need to do is call them like
// PostgreSQL database connection parameters: var config = { host: process.env.host, port: process.env.port, database: process.env.name, user: process.env.user, password: process.env.password, };

I hope above solves lot of issues.

Kind regards,
Javed Gardezi

[sandeeprao]

I am trying to replicate the above example to populate some of the variable values for a function that are extracted from the environment variables but I am getting an error.
Example:
serverless.yml(partial file)
environment: ${file(./env.yml):${opt:stage, self:provider.stage}}

iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:ListBucket"
- "s3:GetObject"
- "s3:PutObject"
Resource: arn:aws:s3:::${file(./env.yml):${opt:stage}.s3.report.input.path}
// Resource: arn:aws:s3:::${env:s3.report.input.path} (Even this format has the same issue)

Error:Trying to populate non string value into a string for
variable ${env:s3.report.input.path}. Please make sure
the value of the property is a string.

I have the s3.report.input.path in env.yml file
dev:
s3.report.input.path: 'path_value'

Any help in resolving would be very helpful.
[[update]] edited ..env.yml path to .env.yml

[rupakg]

@sandeeprao where is the .env.yml located? I see you ref. it once at the top using ./env.yml and then another time using ../env.yml

Can you check the path?

[sandeeprao]

Serverless ane env.yml are both located in the same folder. I modified both to have the same path but it's the same issue.

[kalote]

What i did for this was:

• used serverless-secrets-plugin
• created a secrets.${opt.stage}.yml (file in .gitignore)
• encrypted the secrets.${opt.stage}.yml (file that in github)

With this configuration, i can have multiple env (${opt.stage} comes from serverless.yml provider.stage info), secure env variables (like db user or pwd) and version control over the encrypted file for easy collaboration.

[ceefour]

I think using things like ${file(./env.yml):${opt:stage}.s3.report.input.path} and serverless-secrets-plugin is rather extraneous for something that's very common, this functionality should be in "core" with a recommended best practice. At the very least there should be a very easy and straightforward way to specify different environment variables (i.e. Database URI, access tokens etc.) per stage.

As of August 2017, serverless-secrets-plugin hasn't been updated for 9 months.

[dscis]

+1 for this

[jamesdixon]

have any recommended solutions comes about for this?

[bubba-h57]

I recommend self-referencing custom: variables within the serverles.yml paired with something like AWS Systems Manager Parameter Store (ssm). This simplifies things keeping the configuration in a single file, and still allows us to commit everything without worrying about leaking secure information.

service: example

provider:
  name: aws
  vpc:
    securityGroupIds:
      - sg-xxxxxxxx
    subnetIds:
      - subnet-xxxxxxxx
      - subnet-xxxxxxxx
      - subnet-xxxxxxxx
  runtime: python3.6
  stage: local  # or QA or PRD
  region: us-east-69
  memorySize: 512
  timeout: 60
  versionFunctions: false
  endpointType: REGIONAL
  environment:
    ONE: ${self:custom.one.${self:provider.stage}}
    TWO: ${self:custom.two.${self:provider.stage}}
    THREE: ${self:custom.three.${self:provider.stage}}
    FOUR: ${self:custom.four.${self:provider.stage}}
    FIVE: ${self:custom.five.${self:provider.stage}}
    SIX: ${self:custom.six.${self:provider.stage}}

functions:
  example-one:
      handler: handler.example_one
      name: example-one-${self:provider.stage}
      description: An example for setting variables based on stage deployed.

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
      dockerizePip: non-linux
      vendor: ../packages
  one:
    local: "one.local"
    qa: "one.qa"
    prd: ${ssm:/example/prd/one} #get from ssm
  two:
    local: "two.local"
    qa: "two.qa"
    prd: ${ssm:/example/prd/two} #get from ssm
  three:
    local: "three.local"
    qa: "three.qa"
    prd: ${ssm:/example/prd/three} #get from ssm
  four:
    local: "four.local"
    qa: "four.qa"
    prd: ${ssm:/example/prd/four} #get from ssm
  five:
    local: "five.local"
    qa: "five.qa"
    prd: ${ssm:/example/prd/five} #get from ssm
  six:
    local: "six.local"
    qa: "six.qa"
    prd: ${ssm:/example/prd/six} #get from ssm