Deploying to AWS Elastic Beanstalk using Docker: Challenges I Faced Being a New User

For the past two months, I’ve been working on a collaborative project with other data scientists and full-stack web developers. Our vision was to create a web application for users to compare cities across the United States for various features. Originally, our team was 10 members strong, however we finished with 4 main contributors to the project. Due to this, we decided to drop the Backend entirely and instead pass the data directly from the Data Science (DS) API to the Front-End.

In this blog post, I will show you how I deployed our DS-API to Amazon Web Services — Elastic Beanstalk (AWS — EB) using Docker.

First, I had to install Docker on my local machine. I struggled for three full days with my Docker Desktop installation on Windows Home. I followed these installation steps to no avail:

Windows Subsystem for Linux Installation guide Windows 10

Updating the WSL 2 Linux kernel

Install Docker Desktop on Windows

Install Docker Desktop on Windows Home

After this, I scavenged for articles to offer further assistance, and couldn’t find one with clear instructions that successfully aided my installation. Since our team was so behind, I unfortunately felt the need to bite the bullet and purchase Windows Pro for $99. If this is an issue you ran into, you might consider following suit. I also am making the decision to code on a Linux-based file system after concluding this project, which is another option if you don’t want to support Microsoft’s meddlesome monetization of virtualization technology. Once I purchased Windows Pro, the option to enable virtualization in my BIOS was enabled and I was a simple restart and update (~1 hr. wait) away from being able to host Linux-based containers and images on my machine.

Now, let’s get into how to properly lay out the Dockerfile, docker-compose.yml and Dockerrun.aws.json files for a successful first deployment!

Dockerfile

This Dockerfile contains the commands you would call on the command line to create an image. Docker will run all these commands in succession each time you run docker build. It should be within your project directory, but not inside the app directory (more on layout later).

The docker-compose.yml file contains the different services your app will utilize. For instance, I had to set up a PostgreSQL database with AWS-RDS (Relational Database Service) due to GitHub file-size issues and I had errors with connecting to the DB until I added a “DB” section to my services in this file. Here’s more on the YAML file and docker compose.

The last file I’ll touch on is the Dockerrun.aws.json file. This file describes how to deploy a remote Docker image as an Elastic Beanstalk application. It is necessary when working with images from a private repository. Just make sure you have your image name correct in this file before deploying!

Dockerrun.aws.json

Once You have these files setup, make sure you have your directories setup properly for building your first docker image.

Also, make sure you have your .env file with all your secret keys, passwords, and/or database information so when you can load them into your files properly without pushing them to GitHub.

Before you deploy your application to the Cloud, it’s important to make sure everything is running properly on your local machine to catch any bugs. First, make sure you clone your Git Repository and CD into the directory. Then, follow these steps for local Docker deployment:

$ docker-compose build$ docker-compose up

Once running these commands, Docker will create an image for your project and launch the project locally. You can check how it’s running by navigating to localhost:Container-Port in your browser. Mine was localhost:8000, but you can find the container port in the Dockerrun.aws.json file as shown above. It’s that easy to deploy your app locally with Docker!

Moving on, I received my AWS Access Keys from my Project Manager & Stakeholder and signed into my account. Make sure you are using an IAM User Account moving forward and not signed in as Root User; here’s the difference.

Next, I installed the AWS Command Line Interface and Configured AWS CLI by running the following commands in a CMD terminal:

$ aws configure
AWS Access Key ID [None]: AKIAIOSFODN7/EXAMPLEID
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/EXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: None
$ pip install pipx$ pipx install awsebcli

Follow the EB usage docs and these steps in your terminal to successfully deploy your application with Elastic Beanstalk:

$ docker build -f project/Dockerfile -t YOUR-DOCKER-HUB-ID/YOUR-IMAGE-NAME ./project$ docker login$ docker push YOUR-DOCKER-HUB-ID/YOUR-IMAGE-NAME

Then, use the EB CLI to deploy your image using these CLI steps:

$ git add --all$ git commit -m "Your commit message"$ eb init -p docker DECLARE-YOUR-APP-NAME-HERE --region us-east-1$ eb create DECLARE-YOUR-APP-NAME-HERE$ eb open

I ran into an issue with my machine not being able to locate my EB folder after installation. If you have a similar issue, locate your elastic beanstalk folder and add that folder path to your Windows Environment Variables. Here’s a 3-minute video showing how to add environment variables to PATH in windows.

To redeploy your app, replace the eb init and eb create steps with eb deploy:

$ git add --all$ git commit -m "Your commit message"$ git push origin branchname$ docker build -f project/Dockerfile -t YOUR-DOCKER-HUB-ID/YOUR-IMAGE-NAME ./project$ docker push YOUR-DOCKER-HUB-ID/YOUR-IMAGE-NAME$ eb deploy$ eb open

I ran into some problems when pushing to my branch, so your changes may need to be merged to main before redeploying with elastic beanstalk.

Once you can successfully deploy with Elastic Beanstalk, follow these steps to create a domain name for the application and enable HTTPS protocols. Once done, your Web Developers should be able to effortlessly access your Data Science API.

If you made it this far, feel free to check out our Driftly Data Science API here, and our deployed Web Application here.

Thank you so much for reading! I hope you learned something from this article or that it was able to help you solve a deployment issue you were having!

Here are my links to LinkedIn and GitHub if you’d like to connect. Have a great day 😊

https://www.linkedin.com/in/mtoce/

https://github.com/mtoce

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store