sangeeta.io
May 1, 2019 • 11 min read

Environments with Express & Serverless

node  express  lambda  javascript 

I recently worked on a Node Express API that tracks all holidays in the United States in 2019.

During development, I used a MongoDB Docker container and ran the Express API locally. For production, I imported all my data to a free Mongo Atlas instance, and deployed the API on an AWS Lambda function. I went with this approach for the obvious benefits that AWS Lambda provides – only running your code when a user visits your site. Goodbye 24/7 server fees! 💸

During the process of deploying my API to Lambda using the Serverless framework, I realized three things:

  1. I needed to load my environment variables in my lambda function 😨
  2. I didn’t want my “prod” lambda endpoint to end with /dev/ (more on this later) 😕
  3. I needed to reference the lambda API Gateway URL from within my app 😬

As I worked through these, I realized that the common theme was simply managing multiple development environments! We can rewrite the above to better define our goals:

  1. Load app environment variables into lambda based on development environment
  2. Handle different environments in our Serverless config
  3. Export environment variables defined with Serverless back to app based on development environment

Starting to sound repetitive yet?

I solved this using the serverless-dotenv-plugin and this article. I’ll show you how I set up my serverless configuration with the plugin and some tips on what I learned along the way.

For demonstrative purposes, I’ve made a bare bones node express serverless app which you can refer to here. For the remainder of this article, I’ll be referencing this project.

The folder structure looks like this:

.
├── .gitignore
├── config
│   ├── .env
│   └── .env.production
├── handler.js
├── package-lock.json
├── package.json
├── serverless.yml
└── src
    ├── app.js
    └── index.js

Disclaimer: The following is not meant to be a tutorial, just an overview and some commentary along the way! ✨

src

The src directory holds our application code.

Our express app is defined within app.js, and we run it within index.js. Some reasons for doing this is to have the app easily accessible by itself for testing and passing it to serverless to handle requests.

// src/app.js
const express = require('express')
const app = express()

app.get('/', async (req, res, next) => {
  const env = process.env.NODE_ENV
  res.status(200).send(`Hello World! We're in ${env}!`)
})

module.exports = app
// src/index.js
const app = require('./app')
const port = process.env.PORT

app.listen(port, () => {
  console.log(`App listening on port ${port}`)
})

Another benefit is that our workflow sans serverless remains intact. We can still run package.json scripts like npm run dev. In this example, I use env-cmd to handle environment variables and nodemon to automatically restart the process.

{
    "scripts": {
        "dev": "env-cmd -f ./config/.env nodemon src/index.js"
    }
}

config

The config directory holds our .env files. In this example there’s two env files, .env for development and .env.production for production.

# config/.env

PORT=3000
NODE_ENV=development
DB_URL=your-dev-db-url
DB_NAME=your-dev-db-name

# serverless
STAGE=dev
REGION=us-east-1

handler.js

The handler script imports our app from src/ and wraps it for serverless use within serverless.yml.

const sls = require('serverless-http')
const app = require('./src/app')

module.exports.server = sls(app)

serverless.yml

Finally, we can tie everything together in the serverless.yml file to configure our lambda function.

Without loading our env files just yet, our serverless.yml would look like this

service: serverless-express


provider:
  name: aws
  runtime: nodejs8.10
  stage: dev
  region: us-east-1


functions:
  app:
    handler: handler.server
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true

But we need to manage our enviroments! To load our environment variables into our lambda function, we use the serverless-dotenv-plugin. In our serverless.yml file, we define a custom section and add a variable called dotenv to use the plugin.

This plugin supports a basePath option to point to a directory with multiple env files (our config directory). We access the variables we need for this configuration by ${env:SOME-VAR}.

service: serverless-express


custom:
  dotenv:
    basePath: ./config/


provider:
  name: aws
  runtime: nodejs8.10
  stage: ${env:STAGE}
  region: ${env:REGION}


functions:
  app:
    handler: handler.server
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true


plugins:
  - serverless-dotenv-plugin
Quick note about stages

Stages in the Serverless framework allows us to create different environments for our app. If the stage is not defined in the serverless.yml file, then Serverless will automatically default the stage to dev.

Typically, we want to test our lambda independently of our production environment, hence the reason to dynamically set the stage based on our development environment. When we do this, the Serverless framework creates a different API Gateway project and we get a different endpoint host.

To test our lambda app locally, we can use the serverless-offline plugin. Make sure to update the plugins section in the serverless.yml file.

plugins:
  - serverless-dotenv-plugin
  - serverless-offline

Deploying

To deploy our app and use our production environment variables we run the serverless deploy command with NODE_ENV. The serverless-dotenv-plugin will look for a file called .env.{ENV}. If we don’t pass the NODE_ENV argument, the plugin will look for a .env.development file or default to .env.

To deploy the production version of our app, we run NODE_ENV=production sls deploy or sls deploy --env production if the first doesn’t work.

To run the lambda offline, we run sls offine. This will use our development environment variables.

Gateway URL inside app

What if you need the API Gateway URL within your application? For example, I wanted my holiday API to return an endpoint to view a holiday’s details for each holiday.

{
  "holidays": [
    {
      "id": "some-id",
      "name": "New Year's Day",
      "type": "Federal Holiday",
      "date": "2019-01-01",
      "detailUrl": "https://l922n1th0f.execute-api.us-east-1.amazonaws.com/prod/holidays/some-id"
    }
  ]
}

To achieve this, we need to reconstruct the API Gateway URL with a combination of Serverless variables and CloudFormation (read this article) so that we can use it as an environment variable in our app like process.env.GATEWAY_URL. We will only need to modify the provider and custom sections of the serverless.yml file.

So far, those sections look like this:

custom:
  dotenv:
    basePath: ./config/

provider:
  name: aws
  runtime: nodejs8.10
  stage: ${env:STAGE}
  region: ${env:REGION}

We define an environment variable GATEWAY_URL in the provider section of our serverless.yml file. This variable will depend on what stage we’re using. For production, we’ll construct the url. For development, we’ll use localhost:3000. To do this elegantly, our provider GATEWAY_URL will reference a variable of the same name in the custom section under a specific stage.

custom:
  prod:
    GATEWAY_URL: { "Fn::Join" : ["", [ "https://", { "Ref" : "ApiGatewayRestApi" }, ".execute-api.${self:custom.region}.amazonaws.com/${self:custom.stage}" ] ]  }
  dev:
    GATEWAY_URL: "localhost:3000"

provider:
  GATEWAY_URL: ${self.custom.${self:custom.stage}.GATEWAY_URL}

You might notice that the custom prod GATEWAY_URL uses the stage and region values from the custom section itself.

Since we originally read those values from our env files with the dotenv plugin in the provider section, we can copy the references to the custom section whilst keeping the variables in the provider section. Then, in the provider we just reference their values from custom.

custom:
  stage: ${env:STAGE}
  region: ${env:REGION}

provider:
  stage: ${self:custom.stage}
  region: ${self:custom.region}

Now, we’re done! Here’s a look at the final serverless.yml:

service: serverless-express


custom:
  dotenv:
    basePath: ./config/
  stage: ${env:STAGE}
  region: ${env:REGION}
  prod:
    GATEWAY_URL: { "Fn::Join" : ["", [ "https://", { "Ref" : "ApiGatewayRestApi" }, ".execute-api.${self:custom.region}.amazonaws.com/${self:custom.stage}" ] ]  }
  dev:
    GATEWAY_URL: "http://localhost:3000"

provider:
  name: aws
  runtime: nodejs8.10
  stage: ${self:custom.stage}
  region: ${self:custom.region}
  environment:
    GATEWAY_URL: ${self:custom.${self:custom.stage}.GATEWAY_URL}

functions:
  app:
    handler: handler.server
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true

plugins:
  - serverless-offline
  - serverless-dotenv-plugin

Now, you can access the API Gateway URL from within your app as an environment variable. Remember that when developing locally without serverless-offline, process.env.GATEWAY_URL does not exist, so handle accordingly:

app.get('/', async (req, res) => {
  const gatewayUrl = process.env.GATEWAY_URL || 'localhost:3000'
  response = {
    users_list: `${gatewayUrl}/users`
  }
  res.status(200).send(response)
})
sangeeta.io

Sangeeta Jadoonanan's personal blog. All thoughts and opinions are solely mine, please do your own googling and only copy paste from highly upvoted StackOverflow answers 😏