Deploying applications with AWS ECS Cluster and ALB

Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster.Amazon ECS lets you launch and stop container-based applications with simple API calls, allows you to get the state of your cluster from a centralized service, and gives you access to many familiar Amazon EC2 features.Amazon ECS is a regional service that simplifies running application containers in a highly available manner across multiple Availability Zones within a Region.social_ECR_2.jpg

Amazon ECS is a regional service that simplifies running application containers in a highly available manner across multiple Availability Zones within a Region. You can create Amazon ECS clusters within a new or existing VPC. After a cluster is up and running, you can define task definitions and services that specify which Docker container images to run across your clusters. Container images are stored in and pulled from container registries, which may exist within or outside of your AWS infrastructure.

ALB and ECS Cluster Infrastructure

ecs-load-balancer.png

Containers and Images

To deploy applications on Amazon ECS, your application components must be architected to run in containers. A Docker container is a standardized unit of software development, containing everything that your software application needs to run: code, runtime, system tools, system libraries, etc. Containers are created from a read-only template called an image.

Images are typically built from a Dockerfile, a plain text file that specifies all of the components that are included in the container. These images are then stored in a registry from which they can be downloaded and run on your cluster.

Task Definitions

To prepare your application to run on Amazon ECS, you create a task definition. The task definition is a text file, in JSON format, that describes one or more containers, up to a maximum of ten, that form your application. It can be thought of as a blueprint for your application. Task definitions specify various parameters for your application.

Tasks and Scheduling

A task is the instantiation of a task definition within a cluster. After you have created a task definition for your application within Amazon ECS, you can specify the number of tasks that will run on your cluster.The Amazon ECS task scheduler is responsible for placing tasks within your cluster. There are several different scheduling options available. For example, you can define a service that runs and maintains a specified number of tasks simultaneously.

Container Agent

The container agent runs on each infrastructure resource within an Amazon ECS cluster. It sends information about the resource’s current running tasks and resource utilization to Amazon ECS, and starts and stops tasks whenever it receives a request from Amazon ECS.

Clusters

When you run tasks using Amazon ECS, you place them on a cluster, which is a logical grouping of resources. When using the Fargate launch type with tasks within your cluster, Amazon ECS manages your cluster resources. When using the EC2 launch type, then your clusters are a group of container instances you manage.

Amazon ECS Architecture

aws-ecs

Docker

Docker is a client-server application that can be installed on Linux, Windows, and MacOS and that allows you to run Docker containers. Containers are lightweight environments containing everything needed to run a specific application or part of an application. Multiple different containers can be run on one machine, so long as it has the Docker software installed.

Using Docker containers allows teams to have a consistent development environment by abstracting away the software, operating system, and hardware configuration into a standard building block that can be run on any machine.

Each container has exactly what it needs — for example, certain versions of a language or library — and no more than it needs. Multiple containers can be used for different parts of your application if you want, and they can be set up to communicate with each other when needed.

Create an ECS Cluster

Open the ECS option from Service menu.

ecs1

On the Clusters page, choose Create Cluster.

ecs2

For Select cluster compatibility, choose EC2 Linux + Networking–This choice takes you through the choices to launch a cluster of tasks using the EC2 launch type using Linux containers. The EC2 launch type allows you to run your containerized applications on a cluster of Amazon EC2 instances that you manage.

ecs3

For Cluster name, enter a name for your cluster.

As of now create a cluster with no resources, choose Create an empty cluster, and then click  Create.

ecs4

ecs5

Get a detailed view of the resources on your cluster.

ecs6

Create EC2 Instances and Attach to  ECS Cluster

Before we create an ec2 instance and add to cluster we need to create an IAM role.

IAM role is required for ECS agent to communicate to ECS service.

ecs7

Select the trusted entity as EC2 role for Elastic Container Service

ecs9

ecs10.JPG

Provide a role name and click on create role

ecs11

Create EC2 Instance 

Select the Amazon Machine Image (amzn-ami-2016.03.i-amazon-ecs-optimized ) under Community AMI section.

The Amazon ECS-optimized Amazon Linux AMIs are provided for you to use to launch your Amazon ECS container instances.

ecs12

Choose appropriate instance type and proceed to configure instance details.

Select the ecsInstanceRole IAM role value that you created for your container instances.

Also make sure, that you enable “Auto-assign Public IP”

ecs13

Configure your Amazon ECS container instance with user data, such as the agent environment variables from Amazon ECS Container Agent Configuration. Amazon EC2 user data scripts are executed only one time, when the instance is first launched.

Specify a By default, your container instance launches into your default cluster. To launch into a non-default cluster, choose the Advanced Details list. Then, paste the following script into the User data field, replacing your_cluster_name with the name of your cluster.

#!/bin/bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config

ecs14

On the Add Storage page, configure the storage for your container instance.

If you are using the Amazon ECS-optimized Amazon Linux 2 AMI, your instance a single 30 GiB volume configured, which is shared between the operating system and Docker.

If you are using the Amazon ECS-optimized AMI, your instance has two volumes configured. The Root volume is for the operating system’s use, and the second Amazon EBS volume (attached to /dev/xvdcz) is for Docker’s use.

You can optionally increase or decrease the volume sizes for your instance to meet your application needs.

ecs15

On the Add Tags page, specify tags by providing key and value combinations for the container instance.

Choose Next: Configure Security Group when you are done.

On the Configure Security Group page, use a security group to define firewall rules for your container instance. These rules specify which incoming network traffic is delivered to your container instance. All other traffic is ignored. Select or create a security group as follows, and then choose Review and Launch.

In Order to create dynamic port mapping we need to create a new security group.

Open all the TCP port range to the ALB load balancer group (eg: sg-07eb357c4482b3696)

ecs16

On the Review Instance Launch page, choose Launch.

In the Select an existing key pair or create a new key pair and launch the instance.

ecs17.JPG

After the EC2 Instance being launched, your EC2 instance will be automatically register as ECS container in this cluster.

To verify this navigate to select your cluster (webcls-1) and in ECS Instance tab you can see that the instance has automatically being reflected.

Also the Registered container instances will show the value as 1

ecs18

ecs19

Create Task Definitions

Open the Amazon ECS console.In the navigation pane, choose Task DefinitionsCreate new Task Definition.

ecs20

On the Select compatibility page, select the launch type that your task should use and choose Next step.

Here we will proceed using Using the EC2 launch type compatibility template

ecs21

For Task Definition Name, type a name for your task definition.

ecs22

(Optional) For Network Mode, choose the Docker network mode to use for the containers in your task. The default Docker network mode is bridge.

(Optional) For Task size, choose a value for Task memory (GB) and Task CPU (vCPU). Supported Task CPU (vCPU) values are between 128 CPU units (0.125 vCPUs) and 10240 CPU units (10 vCPUs).

Add Container Definition 

For each container in your task definition, complete the following steps.

  1. Choose Add container.

ecs23

2. Fill out each required field and any optional fields to use in your container definitions.

Image : The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. You can also specify other repositories with either repository-url/image:tag or repository-url/image@digest.

When a new task starts, the Amazon ECS container agent pulls the latest version of the specified image and tag for the container to use. However, subsequent updates to a repository image are not propagated to already running tasks.

Fill out the following fields:

Container Name: eg :httpdwebsrv
Image: httpd:2.4
Maximum memory (MB)*: 128

ecs24

Dynamic Port Mapping in ECS

Port mappings allow containers to access ports on the host container instance to send or receive traffic.

Container Port : The port number on the container that is bound to the user-specified or automatically assigned host port.

Host Port : The port number on the container instance to reserve for your container.

Select the host port 0. It will dynamically assign any available port when it runs a docker container.

ecs25

Advanced Container Definition Parameters

healthCheck :The health check command and associated configuration parameters for the container.

command :A string array representing the command that the container runs to determine if it is healthy.

interval :The time period in seconds between each health check execution. You may specify between 5 and 300 seconds. The default value is 30 seconds.

timeout: The time period in seconds to wait for a health check to succeed before it is considered a failure. You may specify between 2 and 60 seconds. The default value is 5 seconds.

retries :The number of times to retry a failed health check before the container is considered unhealthy. You may specify between 1 and 10 retries. The default value is three retries.

startPeriod: The optional grace period within which to provide containers time to bootstrap before failed health checks count towards the maximum number of retries.

Environment

cpu: The number of cpu units the Amazon ECS container agent will reserve for the container.

entryPoint : The entry point that is passed to the container. This parameter maps to Entrypoint in the Create a container section of the Docker Remote API and the –entrypoint option to docker run.

command : The command that is passed to the container.

Please Click here to download Sample html webpage Command  to pass in container

ecs26.JPG

After filling out these fields, click on ‘Add’ to close the window and then click on “Create” in order to save your task definition.

ecs27

ecs28.JPG

ecs49.JPG

Service Load Balancing

Your Amazon ECS service can optionally be configured to use Elastic Load Balancing to distribute traffic evenly across the tasks in your service.

Open the Amazon ECS console. In the Navigation pane choose Clusters option and select the cluster (eg:webcls-1)

Under the Service tab, click “Create”

ecs29.jpg

On the Configure service page, fill out the following parameters accordingly

Launch type:  service should run tasks on EC2

Cluster: Select the cluster in which to create your service.

Service name: Type a unique name for your service.

Service type: Select a scheduling strategy for your service.

Number of tasks: If you chose the REPLICA service type, type the number of tasks to launch and maintain on your cluster. ( eg : 1)

ecs30

Choose Next step and navigate to Step 2: Configure a Network.

ecs31.jpg

If you are not configuring your service to use a load balancer, you can choose None as the load balancer type and move on to the next section

If you have an available Elastic Load Balancing load balancer configured, you can attach it to your service with the following procedures

Health check grace period: Enter the period of time, in seconds, that the Amazon ECS service scheduler should ignore unhealthy Elastic Load Balancing target health checks after a task has first started.

For Load balancer type, choose the load balancer type to use with your service: Application Load Balancer

ecs32

For Select IAM role for service, choose Create new role to create a new role for your service, or select an existing IAM role to use for your service (by default, this is ecsServiceRole).

For ELB Name, choose the name of the load balancer to use with your service.

ecs33

For Container to load balance, choose the container and port combination from your task definition that your load balancer should distribute traffic to, and choose Add to load balancer.

For Listener port, choose the listener port and protocol of the listener that you created in creating an Application Load Balancer (if applicable)

For Target group name, choose the target group that you created in creating an Application Load Balancer (if applicable)

ecs34

We are not configuring  service to use a service discovery, so we can move on to the next section.

Your Amazon ECS service can optionally be configured to use Auto Scaling to adjust its desired count up or down in response to CloudWatch alarms.

Amazon ECS Service Auto Scaling supports the following types of scaling policies:

  • Target Tracking Scaling Policies : Increase or decrease the number of tasks that your service runs based on a target value for a specific metric.
  • Step Scaling Policies : Increase or decrease the number of tasks that your service runs based on a set of scaling adjustments, known as step adjustments, which vary based on the size of the alarm breach.

On the Set Auto Scaling page, select Configure Service Auto Scaling to adjust your service’s desired count.

For Minimum number of tasks, enter the lower limit of the number of tasks for Service Auto Scaling to use. Your service’s desired count is not automatically adjusted below this amount.

For Desired number of tasks, this field is pre-populated with the value that you entered earlier. You can change your service’s desired count at this time, but this value must be between the minimum and maximum number of tasks specified on this page.

For Maximum number of tasks, enter the upper limit of the number of tasks for Service Auto Scaling to use. Your service’s desired count is not automatically adjusted above this amount.

ecs36.JPG

For Scaling policy type, choose Step scaling.

For Execute policy when, select the CloudWatch alarm to use to scale your service up or down.

You can use an existing CloudWatch alarm that you have previously created, or you can choose to create a new alarm. The Create new alarm workflow allows you to create CloudWatch alarms that are based on the CPUUtilization and MemoryUtilization of the service that you are creating.

ecs37.JPG

ecs38.JPG

Choose Next step to proceed and navigate to  Review and Create Your Service.

ecs39.JPG

After you create a service, the target group ARN or load balancer name, container name, and container port specified in the service definition are immutable. You cannot add, remove, or change the load balancer configuration of an existing service. If you update the task definition for the service, the container name and container port that were specified when the service was created must remain in the task definition.

ecs40.JPG

ecs48.JPG

Verification 

Navigate to Events tab under Service option and verify whether the service has been in steady state.

ecs41

Navigate to Target Groups under Load Balancing and select the Targets tab .

Under the Registered targets Confirm whether the instance is in healthy state.

Also we can see that the ecs has assigned a dynamic port.

ecs42.jpg

Now let us just switch to the web browser and see, if the web site has been properly set up.

Copy the DNS name from Load Balancer and verify the web page.

ecs44.jpg

http://loadbalancer1-1067949394.us-east-1.elb.amazonaws.com

ecs43

Verify the docker container in instance level

Inside EC2 instance we can verify whether the instance is running using docker ps command

ecs45.JPG

Verify the load balancer using curl -v command

ecs46.JPG

List the container images

ecs47.JPG

Update ECS and Mount EBS Volume with Container

Attaching an Amazon EBS Volume to an Instance

In the navigation pane, choose Elastic Block StoreVolumes.Select an available volume and choose ActionsAttach Volume.

Once the volume is attached we can see the device in Block devices section of EC2 Instance.

Eg : /dev/sdf is the new volume attached here. Internally it is mounted as xvdf 

ecs50.jpg

Mount the new volume in EC2 Instance

Use the lsblk command to view your available disk devices and their mount points (if applicable) to help you determine the correct device name to use.

# lsblk

ecs51.JPG

Use the sudo file -s device command to list special information, such as file system type.

$ sudo file -s /dev/xvdf
/dev/xvdf: data

Use the following command to create a mount point directory for the volume. The mount point is where the volume is located in the file system tree and where you read and write files to after you mount the volume.

$ sudo mkdir /http_data

Use the following command to create an ext4 file system on the volume.

$ sudo mkfs -t ext4 /dev/xvdf

To mount this EBS volume on every system reboot, add an entry for the device to the /etc/fstab file.

$ sudo echo "/dev/xvdf /http_data ext4 defaults,nofail 0 2" >> /etc/fstab

ecs52.JPG

Updating ECS Cluster

Check the Existing ECS cluster, Select the “Tasks” and note down the task definition name, which is used by cluster to create Task.

ecs53

Navigate to Task Definitions and, select the latest version of task definition (httpdwebsrv) and click on “Create new revision”

ecs54.JPG

ecs55.JPG

Scroll down to volume option and click on “Add Volume” Fill Name and Source
path accordingly. (make sure the source path should exist in server)

If the source path does not exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported to the container.

ecs56.jpg

ecs57.jpg

ecs58.JPG

Select “Container Definition” and click on it.

ecs59.jpg

Under Container definition scroll to “STORAGE AND LOGGING” and select the
created volume in task definition as shown below. Make sure that you mention
container path.

ecs60.jpg

Update the container definition and create the Task Definition with new version.

ecs61.jpg

Update the newly created Task Definition in ECS Cluster services and save to by
keeping all other option unchanged.

ecs62.jpg

Change the Task Definition Revision value to 2 (latest)

ecs63.jpg

ecs64.jpg

After creating new container, validate with “docker inspect container-id” command.

$ docker inspect a389Ec6786

ecs65.jpg

Check the size of your docker-pool logical volume.

# sudo lvs

LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
docker-pool docker twi-aot--- 21.75g 4.17 15.80

Check the current available space in your volume group.

$ sudo vgs

Verify that docker info also recognizes the added storage space

$ docker info | grep "Data Space"

Reference

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html

ECS Basics

3 thoughts on “Deploying applications with AWS ECS Cluster and ALB

Add yours

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Powered by WordPress.com.

Up ↑

%d bloggers like this: