Introduction

This comprehensive guide establishes foundational knowledge for preparing code for deployment and execution across various environments, with AWS serving as the primary example. The principles discussed apply broadly to most cloud providers and custom server stacks.

“Note that I have chosen NodeJS as server platform, but this can be basically applied to any technology that runs on linux.”

The article covers two distinct scenarios:

  1. Permanent — Running NodeJS server on AWS EC2 instance
  2. Transient — Running computation-heavy tasks on EC2 instances that stop until needed again

What Is It Actually

Server: A standard web application listening on a specific port, exposing APIs and rendering HTML.

Task: Longer-running, computation-intensive operations involving data downloads, merging with streams, and database updates.

This section focuses on code and server deployment. Part two addresses task execution, scheduling, and ad-hoc triggering.

Before We Start

Security

  • Store access keys securely—never commit credentials to public repositories
  • Review AWS documentation thoroughly for Security Groups and IAM Role configurations
  • Misconfigurations can cause data leaks and security breaches
  • Never commit authentication data

Common Considerations

  • Set up entire system within the same AWS region

System Elements

  • Docker — virtualization
  • systemd — process management
  • AWS EC2 — compute infrastructure
  • AWS ECR — container image repository
  • AWS CloudWatch — logging

AWS Specifics

  • Security Group for EC2
  • EC2 access key (.pem file)
  • IAM access key ID and secret
  • IAM roles for EC2 and Lambda

Your Web Application

NodeJS

Basic server implementation in server.js:

var http = require('http');

console.log('Listening on 8080');
http.createServer(function (request, response) {
   response.writeHead(200, {
      'Content-Type': 'text/plain',
      'Access-Control-Allow-Origin' : '*'
   });
   response.end('Hello people!');
 }).listen(8080);

Docker

Docker containerizes applications, defining all system parameters and dependencies. Once packaged, the application runs anywhere Docker is installed with minimal overhead.

Create a Dockerfile in the same directory:

FROM node:boron
WORKDIR /usr/src/app
COPY . .
CMD ["node", "server.js"]

This configuration downloads a Node.js Docker image, sets the working directory, copies application files, and executes the server when the container starts.

First Results — Local Machine

Build and run locally:

docker build . -t node-app
docker run -p 8080:8080 node-app

This approach provides platform independence—Docker containerization enables running NodeJS code on any system supporting Docker.

“At this point you are platform agnostic, you can run your NodeJS code anywhere on the system that supports docker.”

Making It Available for Deployment

Create a container repository (name it node-test) and push the containerized server there. Steps for local or build machines:

aws ecr get-login --no-include-email --region <your_region> | /bin/bash
docker build -t node-test .
docker tag node-test <aws_account_id>.dkr.ecr.<region>.amazonaws.com/node-test
docker push <your ecr url>:latest

These commands authenticate to AWS ECR, build the Docker image, tag it appropriately, and push it to the repository.

Cloud Setup

Server (VM) Setup

Create an Ubuntu EC2 instance and download the .pem file for SSH access. Log in using:

chmod 400 access.pem
ssh -i access.pem ubuntu@<instance_url/ip>

Install Docker and AWS CLI:

sudo su
apt-get install docker.io awscli

Create ~/.aws/credentials:

[default]
aws_access_key_id = <your aws key id>
aws_secret_access_key = <your aws key>

This establishes the base infrastructure with minimal dependencies, allowing simple scaling of VM instances.

Systemd — Core of VM Setup

Systemd daemons automate server initialization, ensuring newly provisioned or restarted servers execute the same operations:

  • Pull Docker images from the repository
  • Run new Docker images

Create /etc/systemd/system/node-js.service:

[Unit]
Description=Docker NodeJS service
Requires=docker.service
After=syslog.target
[Service]
StandardOutput=syslog+console
KillMode=none
ExecStartPre=/bin/sh -c '/usr/bin/aws ecr get-login --region us-east-1 | /bin/bash'
ExecStartPre=-/usr/bin/docker kill nodejs-server
ExecStartPre=-/usr/bin/docker rm nodejs-server
ExecStartPre=/usr/bin/docker pull <your ecr repo>:latest
ExecStart=/usr/bin/docker run --log-driver=awslogs --log-opt awslogs-region=eu-central-1 --log-opt awslogs-group=nodejs --log-opt awslogs-stream=server \
--name nodejs-server <your ecr repo>:latest
ExecStop=/usr/bin/docker stop nodejs-server
[Install]
WantedBy=multi-user.target

This configuration:

  • Authenticates to AWS ECR (requires appropriate IAM Role permissions)
  • Removes previous Docker instances
  • Pulls the latest Docker image
  • Runs the new container

Bonus — Logging

The logging parameters (--log-driver=awslogs --log-opt awslogs-region=eu-central-1 --log-opt awslogs-group=nodejs --log-opt awslogs-stream=server) stream all Docker logs to AWS CloudWatch. This approach is cloud-provider specific and will differ with alternative solutions.

Create /etc/systemd/system/docker.service.d/aws-credentials.conf:

[Service]
Environment="AWS_ACCESS_KEY_ID=<your_aws_access_key_id>"
Environment="AWS_SECRET_ACCESS_KEY=<your_aws_secret_access_key>"

This configuration enables the Docker service to authenticate to AWS for CloudWatch logging. Add a log group named nodejs to CloudWatch to view logs from running instances.

The CI/CD Ability

The systemd approach enables continuous integration and delivery. Basic workflow progresses from:

Initial approach:

  • Complete feature development
  • Test locally on Docker
  • Build Docker image and push to ECR
  • SSH into remote machine and restart service

Improved approach with CI:

  • Code passes linting and unit tests
  • Docker image builds
  • System pushes image to ECR
  • CI triggers remote SSH command to restart systemd service
  • Systemd downloads image, removes previous version, and runs new container

Further On

Logical expansion paths include:

  • Use CI tools — Jenkins or GoCD easily integrate with this workflow
  • Explore scalability — Docker Swarm, Kubernetes, or Mesos may replace systemd for larger deployments
  • Use provisioning tools — Ansible, Puppet, or Chef automate infrastructure setup
  • Implement SSL — Deploy Elastic Load Balancer in front of instances

The next progression addresses running and automating tasks on AWS infrastructure.