Deploying Django RESTful APIs as Serverless Applications with Zappa

TL;DR: In this article, you will see how to build and deploy a serverless Django API with Zappa.

Introduction

Serverless technology was developed from the idea of allowing developers to build and run applications without server management. Servers, of course, still exist, but developers and administrators don't need to manage them. This concept was heralded by AWS when it open-sourced its Serverless Application Model (SAM) framework. It then went on to release Chalice, a flask-like Python framework for building serverless applications. You can learn about building Chalice applications and APIs in this article.

With all the goodness that we are presented with, deploying serverless applications can be difficult when using frameworks like Django. Debugging in production environments can be hard too. Hence, there is a need for a way to deploy painlessly and also provide a means to debug and also monitor logs in production.

Zappa is an open-source Python tool created by Rich Jones of Gun.io for developers to create, deploy and manage serverless Python software on API Gateway and AWS Lambda infrastructure. Zappa allows you to carry out deployment with speed and ease in the most cost-efficient way possible.

In this article, you will learn how to develop serverless Django RESTful APIs, use Zappa to deploy them to AWS, and add authentication with Auth0.

Features and Benefits of Zappa

Some of the benefits provided by Zappa include:

• Automated deployment: With a single command, you can package and deploy your Python project as an AWS serverless app. You can also update and destroy your app with a single command in the different staging environments. You can even deploy your application in different AWS regions.
• No server maintenance: in compliance with the serverless ecosystem, Zappa takes away the maintenance of your servers from your list of tasks.
• Out-of-the-box Scalability: Zappa enables quick infrastructure scaling.
• Great Usability: Zappa provides a single settings file where you can do all your configuration as well as environment variables. It can be stored in JSON or YAML format.
• SSL certificates: Zappa provides a dedicated certify command which grants you an access to SSL certificates from different provides such as let's Encrypt and AWS

Using Zappa with Django

To use Zappa for your Django projects, ensure you have met the following prerequisites:

• Have a machine running Python 3.6, 3.7, or 3.8
• Can create a virtual environment with either venv or virtualenv
• Have an AWS account.
• Configured AWS credentials
• Have basic experience with Python and the Django framework

If you don't have any of the specified Python versions installed, you can download Python here. You can use Venv, Virtualenv, or Pyenv to create virtual environments. If you do not have an AWS account yet, you can sign up for free. You may follow these instructions to configure your AWS credentials.

Building a Django RESTful API

Companies need an employee management system to keep details about each employee. It could be in a form of web application where new staff could be added or removed. What's more, staff details should also be presented.

An employee would have attributes like:

• name
• email
• age
• salary
• location

Hence, in this section, you will create an API that could do the following actions:

• List all employees
• Show the details of an employee
• Add a new employee
• Edit the details of an employee
• Delete an employee

The endpoints could then be as follows:

• List all employees: GET /api/employees
• Show the details of an employee: GET /api/employees/{id}
• Add a new employee: POST /api/employees
• Edit the details of an employee: PUT /api/employees/{id}
• Delete an employee: DELETE api/employees/{id}

Installing Dependencies

Create a folder where the files of this project will reside. Name it django_zappa or any other name. Also, navigate to the new folder:

mkdir django_zappa && cd django_zappa

Create a new virtual environment and activate it:

python -m venv env
source env/bin/activate

Use pip to install the needed dependencies on the CLI as shown below:

pip install Django==2.1.9 djangorestframework==3.10.0 zappa

We used the Pip package manager that comes pre-installed with Python to install the following packages:

• Django: version 2.1.9 of the Django package. Installing this earlier version helps avoid an SQLite version conflict that could arise when deploying Zappa
• djangorestframework: the version 3.10.0 of the Django REST framework package for creating APIs
• zappa: Zappa package for deployment

Scaffolding the Project

Now, you will create the Django project and application. Use the django-admin command to create a new Django project called company:

django-admin startproject company

Then, create a new app called api inside the company parent directory. This app will contain the API code:

cd company
django-admin startapp api

Next, open the settings.py file of the Django project and add rest_framework and api to the list of installed apps.

# company/company/settings.py
...
INSTALLED_APPS = [
    ...
    'rest_framework',
    'api',
]
...

Creating the Model, Database Migrations and Serializer

Here, you will create a model for employee API that will determine which employee details will be saved in the database. Go to the models.py file inside the api application directory and add the following code:

# company/api/models.py

from django.db import models
class Employee(models.Model):
    name = models.CharField(max_length=225)
    email = models.EmailField()
    age = models.IntegerField()
    salary = models.CharField(max_length=60)
    location = models.CharField(max_length=125)

Next, you need to create migrations to seed the model into the database:

python manage.py migrations
python manage.py migrate

Then, you will create a serializer that will allow Django views to return an appropriate response to the users' requests. Create a new serializers.py file in the api directory and add the following code:

# company/api/serializers.py

from rest_framework import serializers
from api.models import Employee
class Serializer(serializers.ModelSerializer):

    class Meta:
        model = Employee
        fields = '__all__'

Creating Views and URLs

You need to make views that will handle the logic of the HTTP actions when users make requests to the API endpoints. The views will also interact with models via the serializer.

Go to the views.py file of the api directory and add the following code:

# company/api/views.py

from rest_framework import generics
from api.models import Employee
from api.serializers import EmployeeSerializer
class EmployeeList(generics.ListCreateAPIView):
    queryset = Employee.objects.all()
    serializer_class = EmployeeSerializer
class EmployeeDetail(generics.RetrieveUpdateDestroyAPIView):
    queryset = Employee.objects.all()
    serializer_class = EmployeeSerializer

The code above has 2 views. The first one, called EmployeeList lists all employees and also allows the creation of a new employee. The other view, EmployeeDetail, allows the creation, retrieve, update, and deletion of particular employees. It accepts a single model instance.

Next, create URLs for the views respectively. Navigate to the urls.py file in the project sub-directory and add the following code:

# company/api/urls.py
...
from django.urls import path
from api.views import EmployeeList, EmployeeDetail

urlpatterns = [
    ...
    path('api/employees/', EmployeeList.as_view()),
    path('api/employees/{id}', EmployeeDetail.as_view()),
]

In the code above, you added paths to the two views created earlier. The URLs were also defined as api/employees/ and api/employees/{id} respectively. The second URL accepts the id as an integer parameter, which is a primary key, int:pk. This is for the single model instance to be fetched when requests are made at the endpoint.

Now, the API has been successfully created. Next, Zappa will be used to deploy the API to AWS Lambda.

Testing Locally

Use the runserver command in your terminal to start the built-in Django server so you can access the API in your browser.

python manage.py runserver

Navigate to127.0.0.1:8000 in your browser. You should see a page that shows a list of accessible endpoints like this:

If you navigate to api/employees endpoint at 127.0.0.1:8000/api/employees, you will see the Django REST framework browsable API:

You have confirmed that the API works well in your local environment.

Deploying with Zappa

Before you deploy the Django API with Zappa, you have to initialize Zappa in the project:

zappa init

When you ran zappa init, you should get a command-line output that looks like the following:

███████╗ █████╗ ██████╗ ██████╗  █████╗
╚══███╔╝██╔══██╗██╔══██╗██╔══██╗██╔══██╗
  ███╔╝ ███████║██████╔╝██████╔╝███████║
 ███╔╝  ██╔══██║██╔═══╝ ██╔═══╝ ██╔══██║
███████╗██║  ██║██║     ██║     ██║  ██║
╚══════╝╚═╝  ╚═╝╚═╝     ╚═╝     ╚═╝  ╚═╝

Welcome to Zappa!

Zappa is a system for running server-less Python web applications on AWS Lambda and AWS API Gateway.
This init command will help you create and configure your new Zappa deployment.
Let's get started!

Your Zappa configuration can support multiple production stages, like 'dev', 'staging', and 'production'.
What do you want to call this environment (default 'dev'):

Follow the prompts to name your staging environment and private S3 bucket where the project files will be stored.

• It will prompt you to request whether you want to deploy globally or not.
• Then, it will detect your application type as Django.
• It will also locate the settings file as company.settings.
• Finally, it will create a zappa_settings.json file in your project directory.

By the end of the prompts, you should get an output like this:

Done! You can also deploy all by executing:

        $ zappa deploy --all

After that, you can update your application code with:

        $ zappa update --all

To learn more, check out our project page on GitHub here: https://github.com/Miserlou/Zappa
and stop by our Slack channel here: https://zappateam.slack.com

Enjoy!,
 ~ Team Zappa!

The zappa_settings.json file is crucial for your deployment because it contains the deployment settings.

{
    "dev": {
        "aws_region": "us-west-2",
        "django_settings": "company.settings",
        "profile_name": "default",
        "project_name": "company",
        "runtime": "python3.7",
        "s3_bucket": "zappa-xxxxxxxxx"
    }

The zappa_settings.json file may, however, contain the information for other regions if you choose global deployment while initializing Zappa.

You will deploy your project with the zappa deploy command where stage-name could be dev or any other stage name you use when initializing Zappa:

zappa deploy stage-name

Upon successful deployment, you should get a URL where you can access your API on the internet. It should look like this:

https://5vbm7e4hn2.execute-api.us-west-2.amazonaws.com/dev

Copy the URL generated for your application and add it to the list of ALLOWED_HOSTS inside the settings.py file of the project:

# company/company/settings.py
...
ALLOWED_HOSTS = ['127.0.0.1', '9dgf4q6wx8.execute-api.us-west-2.amazonaws.com',]
...

Then, update the deployment:

zappa update dev

Keep in mind that you need to update the deployment with the command above when making any changes in your project.

Serving Static Files

At this point, you may need to serve static files so that default Django styles can be active in the deployed stage.

Creating an S3 bucket

• Go to the Amazon S3 console and select Create bucket
• Give your bucket a unique name. The name must start with a lowercase character or number. It must not contain an uppercase character. The length should be between 3 and 63 characters.

Note that you can't change a bucket name after creating it.

• Select the AWS region where you want your bucket to be hosted.
• Under the Bucket settings for Block Public Access, make sure you uncheck Block all public access
• Note the name of the S3 bucket that you created

Go to the Permissions tab of your S3 bucket and navigate to the Cross-Origin resource sharing (CORS) section. Click the Edit button and add the following configuration:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET"],
    "AllowedOrigins": ["*"],
    "MaxAgeSeconds": 3000
  }
]

Configuring Django Settings for Handling Static Files

Install django-s3-storage library for Django to work with S3:

pip install django-s3-storage

Next, open the settings.py file and add the following:

# company/company/settings.py

import os
...

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

S3_BUCKET_NAME = "put-the-name-of-the-bucket-you-created-here"
STATICFILES_STORAGE = "django_s3_storage.storage.StaticS3Storage"
AWS_S3_BUCKET_NAME_STATIC = S3_BUCKET_NAME

# to serve the static files from your s3 bucket
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % S3_BUCKET_NAME
STATIC_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN

# if you hosted your static files on a custom domain
#AWS_S3_PUBLIC_URL_STATIC = "https://static.yourdomain.com/"

Now, update the deployment with the static files.

zappa update dev
zappa manage dev "collectstatic --noinput"

The browsable API should render with its styles appropriately like below:

Adding Authentication with Auth0

You need an Auth0 account to use Auth0 APIs for authenticating your Django APIs with Auth0. You can create an Auth0 account if you do not have one.

Click the Create API button on your Auth0 dashboard to create an API. Then, go to the Permissions tab of the newly created API to allow read access.

Now, install the libraries needed for authentication:

pip install cryptography==2.8 django-cors-headers==3.1.1 drf-jwt==1.13.3 pyjwt==1.7.1 requests==2.22.0

Next, open the settings.py file of the project and add the ModelBackend and the RemoteUserBackend to the list of Authentication backends.

# company/company/settings.py

AUTHENTICATION_BACKENDS = [
    'django.contrib.auth.backends.ModelBackend',
    'django.contrib.auth.backends.RemoteUserBackend',
]

The code above allows Django to link the Django users database with your Auth0 users database.

Next, update the middleware in the settings.py file as thus:

# company/company/settings.py

MIDDLEWARE = [
    ...
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.RemoteUserMiddleware',
]

The RemoteUserMiddleware middleware connects the user in the Auth0 Access Token to the user in the Django authentication system. The AuthenticationMiddleware handles authentication. Make sure you add the AuthenticationMiddleware after the SessionMiddleware to avoid an AssertionError.

Next, navigate to the api application directory and create a new file called utils.py. Then, add the following:

# company/api/utils.py

from django.contrib.auth import authenticate

def jwt_get_username_from_payload_handler(payload):
    username = payload.get('sub').replace('|', '.')
    authenticate(remote_user=username)
    return username

The code above consists of a function jwt_get_username_from_payload_handler that accepts the authentication payload, which is the Access Token. It then maps the sub field from the Access Token to the username variable. The authenticate method imported from the RemoteUserBackend creates a remote user in the Django authentication system. Then, a User object is returned for the username.

Next, navigate back to the settings.py file of the project and IsAuthenticated to the DEFAULT_PERMISSION_CLASSES. Also, add the JSONWebTokenAuthentication to the DEFAULT_AUTHENTICATION_CLASSESlike this:

# company/company/settings.py

REST_FRAMEWORK = {
    'DEFAULT_PERMISSION_CLASSES': (
        'rest_framework.permissions.IsAuthenticated',
    ),
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
        'rest_framework.authentication.SessionAuthentication',
        'rest_framework.authentication.BasicAuthentication',
    ),
}

In the code above, we have defined settings for permissions, i.e., what can be accessed after authentication. We also defined the type of authentication that we want as JWT.

Next, set the JWT_AUTH variable for the JWT authentication library in Django.

Add the following code to your settings.py file. Make sure you replace the JWT_AUDIENCE with your Auth0 API identifier and the JWT_ISSUER with your Auth0 domain.

# company/company/settings.py

JWT_AUTH = {
    'JWT_PAYLOAD_GET_USERNAME_HANDLER':
        'auth0authorization.utils.jwt_get_username_from_payload_handler',
    'JWT_DECODE_HANDLER':
        'auth0authorization.utils.jwt_decode_token',
    'JWT_ALGORITHM': 'RS256',
    'JWT_AUDIENCE': 'YOUR_AUTH0_API_IDENTIFIER',
    'JWT_ISSUER': 'https://YOUR_AUTH0__DOMAIN/',
    'JWT_AUTH_HEADER_PREFIX': 'Bearer',
}

Next, navigate to the utils.py file and add a function to get the JSON Web Token Key Sets (JWKS) from your dashboard to verify and decode Access Tokens.

# company/api/utils.py

...
import json
import jwt
import requests

def jwt_decode_token(token):
    header = jwt.get_unverified_header(token)
    jwks = requests.get('https://{}/.well-known/jwks.json'.format('YOUR_AUTH0__DOMAIN')).json()
    public_key = None
    for jwk in jwks['keys']:
        if jwk['kid'] == header['kid']:
            public_key = jwt.algorithms.RSAAlgorithm.from_jwk(json.dumps(jwk))

    if public_key is None:
        raise Exception('Public key not found.')

    issuer = 'https://{}/'.format('YOUR_AUTH0__DOMAIN')
    return jwt.decode(token, public_key, audience='YOUR_AUTH0_API_IDENTIFIER', issuer=issuer, algorithms=['RS256'])

Next, create two methods in the views.py file of the api application to check the scopes granted from the access_token:

# company/api/views.py

...
from functools import wraps
import jwt

from django.http import JsonResponse

def get_token_auth_header(request):
    """Obtains the Access Token from the Authorization Header
    """
    auth = request.META.get("HTTP_AUTHORIZATION", None)
    parts = auth.split()
    token = parts[1]

    return token

def requires_scope(required_scope):
    """Determines if the required scope is present in the Access Token
    Args:
        required_scope (str): The scope required to access the resource
    """
    def require_scope(f):
        @wraps(f)
        def decorated(*args, **kwargs):
            token = get_token_auth_header(args[0])
            decoded = jwt.decode(token, verify=False)
            if decoded.get("scope"):
                token_scopes = decoded["scope"].split()
                for token_scope in token_scopes:
                    if token_scope == required_scope:
                        return f(*args, **kwargs)
            response = JsonResponse({'message': 'You don\'t have access to this resource'})
            response.status_code = 403
            return response
        return decorated
    return require_scope

In the code above, the get_token_auth_header method gets the Access token from the authorization header while the requires_scope checks if the Access Token obtained contains the scope required to allow access to some specific parts of the application.

Now, you can protect endpoints by adding the authentication_classes and permission_classes decorators to methods that need authentication and permission scopes, respectively.

# company/api/views.py
 
from rest_framework import permissions
from rest_framework_jwt.authentication import JSONWebTokenAuthentication
...

rest_framework_jwt.authentication.JSONWebTokenAuthentication
class EmployeeList(generics.ListCreateAPIView):
    authentication_classes = [authentication.JSONWebTokenAuthentication]
    permission_classes = [permissions.IsAuthenticated]
    ...

Now, your application has Auth0 authentication enabled. Update the deployment for the authentication to be active:

zappa update dev

Conclusion

This tutorial has taken you through the process of building Django RESTful APIs and deploying them as serverless applications on AWS Lambda with Zappa. You also learned how to configure authentication using Auth0. Now, you can go on and use the knowledge gained in your projects building APIs and deploying serverless Python applications.

Let us have your suggestions and questions in the comments section below. Thanks.