Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster.Amazon ECS lets you launch and stop container-based applications with simple API calls, allows you to get the state of your cluster from a centralized service, and gives you access to many familiar Amazon EC2 features.Amazon ECS is a regional service that simplifies running application containers in a highly available manner across multiple Availability Zones within a Region.
Amazon ECS is a regional service that simplifies running application containers in a highly available manner across multiple Availability Zones within a Region. You can create Amazon ECS clusters within a new or existing VPC. After a cluster is up and running, you can define task definitions and services that specify which Docker container images to run across your clusters. Container images are stored in and pulled from container registries, which may exist within or outside of your AWS infrastructure.
ALB and ECS Cluster Infrastructure
Containers and Images
To deploy applications on Amazon ECS, your application components must be architected to run in containers. A Docker container is a standardized unit of software development, containing everything that your software application needs to run: code, runtime, system tools, system libraries, etc. Containers are created from a read-only template called an image.
Images are typically built from a Dockerfile, a plain text file that specifies all of the components that are included in the container. These images are then stored in a registry from which they can be downloaded and run on your cluster.
To prepare your application to run on Amazon ECS, you create a task definition. The task definition is a text file, in JSON format, that describes one or more containers, up to a maximum of ten, that form your application. It can be thought of as a blueprint for your application. Task definitions specify various parameters for your application.
Tasks and Scheduling
A task is the instantiation of a task definition within a cluster. After you have created a task definition for your application within Amazon ECS, you can specify the number of tasks that will run on your cluster.The Amazon ECS task scheduler is responsible for placing tasks within your cluster. There are several different scheduling options available. For example, you can define a service that runs and maintains a specified number of tasks simultaneously.
The container agent runs on each infrastructure resource within an Amazon ECS cluster. It sends information about the resource’s current running tasks and resource utilization to Amazon ECS, and starts and stops tasks whenever it receives a request from Amazon ECS.
When you run tasks using Amazon ECS, you place them on a cluster, which is a logical grouping of resources. When using the Fargate launch type with tasks within your cluster, Amazon ECS manages your cluster resources. When using the EC2 launch type, then your clusters are a group of container instances you manage.
Amazon ECS Architecture
Docker is a client-server application that can be installed on Linux, Windows, and MacOS and that allows you to run Docker containers. Containers are lightweight environments containing everything needed to run a specific application or part of an application. Multiple different containers can be run on one machine, so long as it has the Docker software installed.
Using Docker containers allows teams to have a consistent development environment by abstracting away the software, operating system, and hardware configuration into a standard building block that can be run on any machine.
Each container has exactly what it needs — for example, certain versions of a language or library — and no more than it needs. Multiple containers can be used for different parts of your application if you want, and they can be set up to communicate with each other when needed.
Create an ECS Cluster
Open the ECS option from Service menu.
On the Clusters page, choose Create Cluster.
For Select cluster compatibility, choose EC2 Linux + Networking–This choice takes you through the choices to launch a cluster of tasks using the EC2 launch type using Linux containers. The EC2 launch type allows you to run your containerized applications on a cluster of Amazon EC2 instances that you manage.
For Cluster name, enter a name for your cluster.
As of now create a cluster with no resources, choose Create an empty cluster, and then click Create.
Get a detailed view of the resources on your cluster.
Create EC2 Instances and Attach to ECS Cluster
Before we create an ec2 instance and add to cluster we need to create an IAM role.
IAM role is required for ECS agent to communicate to ECS service.
Select the trusted entity as EC2 role for Elastic Container Service
Provide a role name and click on create role
Create EC2 Instance
Select the Amazon Machine Image (amzn-ami-2016.03.i-amazon-ecs-optimized ) under Community AMI section.
The Amazon ECS-optimized Amazon Linux AMIs are provided for you to use to launch your Amazon ECS container instances.
Choose appropriate instance type and proceed to configure instance details.
Select the ecsInstanceRole IAM role value that you created for your container instances.
Also make sure, that you enable “Auto-assign Public IP”
Configure your Amazon ECS container instance with user data, such as the agent environment variables from Amazon ECS Container Agent Configuration. Amazon EC2 user data scripts are executed only one time, when the instance is first launched.
Specify a By default, your container instance launches into your default cluster. To launch into a non-default cluster, choose the Advanced Details list. Then, paste the following script into the User data field, replacing your_cluster_name with the name of your cluster.
#!/bin/bash echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
On the Add Storage page, configure the storage for your container instance.
If you are using the Amazon ECS-optimized Amazon Linux 2 AMI, your instance a single 30 GiB volume configured, which is shared between the operating system and Docker.
If you are using the Amazon ECS-optimized AMI, your instance has two volumes configured. The Root volume is for the operating system’s use, and the second Amazon EBS volume (attached to /dev/xvdcz) is for Docker’s use.
You can optionally increase or decrease the volume sizes for your instance to meet your application needs.
On the Add Tags page, specify tags by providing key and value combinations for the container instance.
Choose Next: Configure Security Group when you are done.
On the Configure Security Group page, use a security group to define firewall rules for your container instance. These rules specify which incoming network traffic is delivered to your container instance. All other traffic is ignored. Select or create a security group as follows, and then choose Review and Launch.
In Order to create dynamic port mapping we need to create a new security group.
Open all the TCP port range to the ALB load balancer group (eg: sg-07eb357c4482b3696)
On the Review Instance Launch page, choose Launch.
In the Select an existing key pair or create a new key pair and launch the instance.
After the EC2 Instance being launched, your EC2 instance will be automatically register as ECS container in this cluster.
To verify this navigate to select your cluster (webcls-1) and in ECS Instance tab you can see that the instance has automatically being reflected.
Also the Registered container instances will show the value as 1
Create Task Definitions
Open the Amazon ECS console.In the navigation pane, choose Task Definitions, Create new Task Definition.
On the Select compatibility page, select the launch type that your task should use and choose Next step.
Here we will proceed using Using the EC2 launch type compatibility template
For Task Definition Name, type a name for your task definition.
(Optional) For Network Mode, choose the Docker network mode to use for the containers in your task. The default Docker network mode is bridge.
(Optional) For Task size, choose a value for Task memory (GB) and Task CPU (vCPU). Supported Task CPU (vCPU) values are between 128 CPU units (0.125 vCPUs) and 10240 CPU units (10 vCPUs).
Add Container Definition
For each container in your task definition, complete the following steps.
- Choose Add container.
2. Fill out each required field and any optional fields to use in your container definitions.
Image : The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. You can also specify other repositories with either repository-url/image:tag or repository-url/image@digest.
When a new task starts, the Amazon ECS container agent pulls the latest version of the specified image and tag for the container to use. However, subsequent updates to a repository image are not propagated to already running tasks.
Fill out the following fields:
Container Name: eg :httpdwebsrv
Maximum memory (MB)*: 128
Dynamic Port Mapping in ECS
Port mappings allow containers to access ports on the host container instance to send or receive traffic.
Container Port : The port number on the container that is bound to the user-specified or automatically assigned host port.
Host Port : The port number on the container instance to reserve for your container.
Select the host port 0. It will dynamically assign any available port when it runs a docker container.
Advanced Container Definition Parameters
healthCheck :The health check command and associated configuration parameters for the container.
command :A string array representing the command that the container runs to determine if it is healthy.
interval :The time period in seconds between each health check execution. You may specify between 5 and 300 seconds. The default value is 30 seconds.
timeout: The time period in seconds to wait for a health check to succeed before it is considered a failure. You may specify between 2 and 60 seconds. The default value is 5 seconds.
retries :The number of times to retry a failed health check before the container is considered unhealthy. You may specify between 1 and 10 retries. The default value is three retries.
startPeriod: The optional grace period within which to provide containers time to bootstrap before failed health checks count towards the maximum number of retries.
cpu: The number of cpu units the Amazon ECS container agent will reserve for the container.
entryPoint : The entry point that is passed to the container. This parameter maps to Entrypoint in the Create a container section of the Docker Remote API and the –entrypoint option to docker run.
command : The command that is passed to the container.
Please Click here to download Sample html webpage Command to pass in container
After filling out these fields, click on ‘Add’ to close the window and then click on “Create” in order to save your task definition.
Service Load Balancing
Your Amazon ECS service can optionally be configured to use Elastic Load Balancing to distribute traffic evenly across the tasks in your service.
Open the Amazon ECS console. In the Navigation pane choose Clusters option and select the cluster (eg:webcls-1)
Under the Service tab, click “Create”
On the Configure service page, fill out the following parameters accordingly
Launch type: service should run tasks on EC2
Cluster: Select the cluster in which to create your service.
Service name: Type a unique name for your service.
Service type: Select a scheduling strategy for your service.
Number of tasks: If you chose the REPLICA service type, type the number of tasks to launch and maintain on your cluster. ( eg : 1)
Choose Next step and navigate to Step 2: Configure a Network.
If you are not configuring your service to use a load balancer, you can choose None as the load balancer type and move on to the next section
If you have an available Elastic Load Balancing load balancer configured, you can attach it to your service with the following procedures
Health check grace period: Enter the period of time, in seconds, that the Amazon ECS service scheduler should ignore unhealthy Elastic Load Balancing target health checks after a task has first started.
For Load balancer type, choose the load balancer type to use with your service: Application Load Balancer
For Select IAM role for service, choose Create new role to create a new role for your service, or select an existing IAM role to use for your service (by default, this is ecsServiceRole).
For ELB Name, choose the name of the load balancer to use with your service.
For Container to load balance, choose the container and port combination from your task definition that your load balancer should distribute traffic to, and choose Add to load balancer.
For Listener port, choose the listener port and protocol of the listener that you created in creating an Application Load Balancer (if applicable)
For Target group name, choose the target group that you created in creating an Application Load Balancer (if applicable)
We are not configuring service to use a service discovery, so we can move on to the next section.
Your Amazon ECS service can optionally be configured to use Auto Scaling to adjust its desired count up or down in response to CloudWatch alarms.
Amazon ECS Service Auto Scaling supports the following types of scaling policies:
- Target Tracking Scaling Policies : Increase or decrease the number of tasks that your service runs based on a target value for a specific metric.
- Step Scaling Policies : Increase or decrease the number of tasks that your service runs based on a set of scaling adjustments, known as step adjustments, which vary based on the size of the alarm breach.
On the Set Auto Scaling page, select Configure Service Auto Scaling to adjust your service’s desired count.
For Minimum number of tasks, enter the lower limit of the number of tasks for Service Auto Scaling to use. Your service’s desired count is not automatically adjusted below this amount.
For Desired number of tasks, this field is pre-populated with the value that you entered earlier. You can change your service’s desired count at this time, but this value must be between the minimum and maximum number of tasks specified on this page.
For Maximum number of tasks, enter the upper limit of the number of tasks for Service Auto Scaling to use. Your service’s desired count is not automatically adjusted above this amount.
For Scaling policy type, choose Step scaling.
For Execute policy when, select the CloudWatch alarm to use to scale your service up or down.
You can use an existing CloudWatch alarm that you have previously created, or you can choose to create a new alarm. The Create new alarm workflow allows you to create CloudWatch alarms that are based on the CPUUtilization and MemoryUtilization of the service that you are creating.
Choose Next step to proceed and navigate to Review and Create Your Service.
After you create a service, the target group ARN or load balancer name, container name, and container port specified in the service definition are immutable. You cannot add, remove, or change the load balancer configuration of an existing service. If you update the task definition for the service, the container name and container port that were specified when the service was created must remain in the task definition.
Navigate to Events tab under Service option and verify whether the service has been in steady state.
Navigate to Target Groups under Load Balancing and select the Targets tab .
Under the Registered targets Confirm whether the instance is in healthy state.
Also we can see that the ecs has assigned a dynamic port.
Now let us just switch to the web browser and see, if the web site has been properly set up.
Copy the DNS name from Load Balancer and verify the web page.
Verify the docker container in instance level
Inside EC2 instance we can verify whether the instance is running using docker ps command
Verify the load balancer using curl -v command
List the container images
Update ECS and Mount EBS Volume with Container
Attaching an Amazon EBS Volume to an Instance
In the navigation pane, choose Elastic Block Store, Volumes.Select an available volume and choose Actions, Attach Volume.
Once the volume is attached we can see the device in Block devices section of EC2 Instance.
Mount the new volume in EC2 Instance
Use the lsblk command to view your available disk devices and their mount points (if applicable) to help you determine the correct device name to use.
Use the sudo file -s device command to list special information, such as file system type.
$ sudo file -s /dev/xvdf /dev/xvdf: data
Use the following command to create a mount point directory for the volume. The mount point is where the volume is located in the file system tree and where you read and write files to after you mount the volume.
$ sudo mkdir /http_data
Use the following command to create an ext4 file system on the volume.
$ sudo mkfs -t ext4 /dev/xvdf
To mount this EBS volume on every system reboot, add an entry for the device to the /etc/fstab file.
$ sudo echo "/dev/xvdf /http_data ext4 defaults,nofail 0 2" >> /etc/fstab
Updating ECS Cluster
Check the Existing ECS cluster, Select the “Tasks” and note down the task definition name, which is used by cluster to create Task.
Navigate to Task Definitions and, select the latest version of task definition (httpdwebsrv) and click on “Create new revision”
Scroll down to volume option and click on “Add Volume” Fill Name and Source
path accordingly. (make sure the source path should exist in server)
If the source path does not exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported to the container.
Select “Container Definition” and click on it.
Under Container definition scroll to “STORAGE AND LOGGING” and select the
created volume in task definition as shown below. Make sure that you mention
Update the container definition and create the Task Definition with new version.
Update the newly created Task Definition in ECS Cluster services and save to by
keeping all other option unchanged.
Change the Task Definition Revision value to 2 (latest)
After creating new container, validate with “docker inspect container-id” command.
$ docker inspect a389Ec6786
Check the size of your docker-pool logical volume.
# sudo lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool docker twi-aot--- 21.75g 4.17 15.80
Check the current available space in your volume group.
Verify that docker info also recognizes the added storage space
$ docker info | grep "Data Space"