4 Tips for Starting a Serverless Development Project

Name Your Root Directory Meaningfully

I know this sounds really basic, stick with me! You might be tempted to name the directory that contains all of your serverless code serverless. This is a bad idea. It is much more useful and easier to maintain projects if you name folders after what they do. To belabor the point, imagine if you had a non-serverless Python project that contained all of your API source code and you called the root directory python. Imagine then in the same project you need to add a module for data ingestion – you’ll also write it in Python. With this naming convention you’d have to put the data-ingestion code into the python directory, right? At this point, you’d probably find it helpful to put api and data_ingestion each in their own sub-directory. Remind me then what the purpose of the root python directory was? What a mess.

For example, a single service called api should look like this.

api
├── pyproject.toml
├── poetry.lock
├── README.md
├── widgets
│   ├── __init__.py
│   └── handler.py
├── users
│   ├── __init__.py
│   └── handler.py
├── package.json
├── pytest.ini
├── serverless.yml
├── tests
│   └── test_....py
└── yarn.lock

Naming the directory after what it does also has the benefit of affording multiple services in a single repository, for example data ingestion and api services with completely separate stacks. An example multi-service project might look like this.

my_project
├── api
│   ├── pyproject.toml
│   ├── poetry.lock
│   ├── README.md
│   ├── widgets
│   │   ├── __init__.py
│   │   └── handler.py
│   ├── users
│   │   ├── __init__.py
│   │   └── handler.py
│   ├── package.json
│   ├── pytest.ini
│   ├── serverless.yml
│   ├── tests
│   │   └── test_....py
│   └── yarn.lock
└── data_ingestion
    ├── pyproject.toml
    ├── poetry.lock
    ├── README.md
    ├── scheduled
    │   ├── __init__.py
    │   └── handler.py
    ├── package.json
    ├── pytest.ini
    ├── serverless.yml
    ├── tests
    │   └── test_....py
    └── yarn.lock

Isn’t that clean?

One Handler or Many?

Many. Each logical component should have its own directory and a handler.py. Within that file, you can have many handler functions. Each of those handler functions should start with the domain. For example, a function for adding users should be, users_add (not add_user).

The reason for this benefit might not be obvious until you consider the way functions are defined.

functions:
  users_add:
    handler: users.handler.users_add
    events:
      - http:
          path: users/
          method: post

When we define the handler this way, we see how this project layout neatly namespaces our handler code. This convention also has the added benefit of grouping related resources together in your cloudformation stack, bonus! Finally, we notice that the handler function and the lambda function are named the same, which makes navigating your code easier.

“WAIT”, you say, “Wouldn’t it be ‘neatly namespaced’ If we just had one handler.py? LOOK!”

functions:
  users_add:
    handler: handler.users_add
    events:
      - http:
          path: users/
          method: post

Glad you asked. Because you’re right. It is still namespaced. However, this single handler file will grow linearly with the size of our project, and eventually it will become difficult to wade through. Ouch.

A final note on handlers and namespaces. There is an equally good argument to having single-handler files for every function, and naming none of the files handler.py. Using this method, the file is named the same as the function, and each has a single handler function. Imagine the following:

├── users
│   ├── __init__.py
│   ├── users_add.py
│   ├── users_delete.py
│   ├── users_get.py
│   └── users_update.py

This seems good at first, almost better than my recommended approach!

The handler: users.users_add.handler namespace looks great. However, consider that since each of these handlers should be as thin as possible, that we’re cluttering up our codebase with essentially routing information as files. My guess is most projects would benefit from the first pattern over this one, but there are reasons this last pattern might fit.

Breaking Out Functions, Roles, Resources (Utilizing Interpolation)

Serverless can interpolate several kinds of variables and files, including json, and yaml. It is a best practice to break your functions, resources, and roles out into separate files. The benefit is an easier-to-read serverless.yml. Beyond that, you can combine environment and file interpolation in such a way that deploying to a development stage can deploy resources with a smaller footprint.

An example might look like this.

├── resources
│   ├── dev.yml
│   └── prod.yml
├── roles
│   ├── dev.yml
│   └── prod.yml
└── serverless.yml

That way you can do a real fancy thing, define your resources block in serverless.yml like this.

resources:
  - ${file(resources/${opt:stage, self:provider.stage}.yml)}

Taking a look at our resources/dev.yml resources, we leverage smaller provisioning.

Resources:
  UsersTable:
    Type: "AWS::DynamoDB::Table"
    Properties:
      AttributeDefinitions:
        - AttributeName: username
          AttributeType: S
      KeySchema:
        - AttributeName: username
          KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: 1
        WriteCapacityUnits: 1
      TableName: dev-table

Compare that to what we might want in production, by looking at prod.yml.

Resources:
  UsersTable:
    Type: "AWS::DynamoDB::Table"
    Properties:
      AttributeDefinitions:
        - AttributeName: username
          AttributeType: S
      KeySchema:
        - AttributeName: username
          KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: 100
        WriteCapacityUnits: 100
      TableName: prod-table

This will enable sls deploy -s dev to build a dev environment, and sls deploy -s prod to deploy prod-like resources, a very useful feature for CI/CD!

Use Dependency Management, Use a Bundling Plugin

Figuring out how to deploy a large serverless project with dependencies can be tricky. You might have success, but more likely than not you’ll want to reach for a packaging and bundling solution. Luckily, there are stable serverless plugins for popular languages. At Very, we use a good deal of Python and TypeScript, so we have found useful plugins in those ecosystems.

For Python projects, I use Poetry to manage my dependencies. You might use pipenv or plain-old pip + requirements.txt. No matter your choice, I recommend using the serverless-python-requirements plugin to take care of bundling up your dependencies. For Node projects I use serverless-webpack. No matter which you use, I’m sure you’ll find that they allow you to more closely match your usual development flow.

Future You

I have espoused many opinions in this blog. They come from my experiences with the serverless framework maintaining several projects over the last two years. You might agree with some of them, you might find some don’t work for you or your team. Whatever the case, I hope I’ve gotten the gears turning when thinking about the long-term maintenance of your serverless projects.