What is the AMI Cleanup Pipeline?
The AMI Cleanup pipeline helps you clean up your old unused AMI and related snapshots.
Goals of AMI Cleanup Pipeline
- An Amazon Machine Image (AMI) is a packaged environment containing a software configuration and other parts required to set up an instance.
- As it contains the built solutions/artifacts, we end up creating AMIs for every new feature or release or even for testing the changes.
- Thus, to clean up the older AMIs, we can use this AMI Cleanup Pipeline.
Benefits of AMI Cleanup Pipeline
Key features of the AMI Cleanup Pipeline
AMI
An Amazon Machine Image (AMI) offers the information needed to launch an instance. You must define an AMI when you initiate an instance.
Amazon EBS Snapshots
You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots. Snapshots are incremental backups. This implies that only the blocks on the device that have changed after your latest snapshot are saved. It decreases the time needed to create the snapshot and saves a lot on storage costs by not duplicating data. Each snapshot contains all the information that is needed to restore your data (from the moment when the snapshot was taken) to a new EBS volume.
Our Solution Architecture with AMI Cleanup Pipeline
We integrated the AMI Cleanup Pipeline in our Devops-in-a-box platform to deliver better solutions to our customers.
Well, we are an Advanced AWS consultant now.
How we help one of our customers with the AMI Cleanup Pipeline?
Our customer’s app involved autoscaling-based deployment. The Autoscaling group helps us to scale out (add EC2 instances) to match an increased load and scale in (remove EC2 instances) to match the decreased load.
So whenever any instance goes down, the autoscaling group handles it quickly by launching another instance with the help of AMI.
Amazon Machine Image (AMI) provides the information required to launch an instance. You must specify an AMI before you launch an instance. Our customer’s app is an EC2-based application.
Let’s say in the app we want to add a new feature, so we will write code for it and create an AMI and we will push the code changes to development/production.
Again, we have a new feature to be added on top of the previous one, so we again repeat the same procedure. So for 10 new features, we created 10 new AMIs.
Also here we are not considering the AMIs that we created for testing purposes, so let’s say we created 5 AMIs for testing each new feature that itself creates 50 AMIs for 10 new features.
But we don’t need the AMIs that we created in the beginning, as they are of no use to us. If we don’t delete these old AMIs we will end up paying for AMIs, that are old and unused.
Instead of manually finding old and unused AMIs and de-registering them, in our client’s app we used AMI Cleanup Pipeline which de-registers all the old, unused AMIs automatically.
When you deregister an AMI, it doesn’t affect any instances that you’ve already initiated from the AMI or any snapshots created during the AMI creation process. You’ll carry on to incur usage costs for these instances and storage costs for the snapshot.
Therefore, you should terminate any instances and delete any snapshots that you’re finished with. The snapshots cannot be deleted before registering AMIs. For cleaning up the old unused AMIs we can use python or bash script.
In python, we have a Boto3 library that can be used to access/connect with AWS resources. We will execute a python program in our pipeline to de-register the AMIs.
Published By
Pranav Kakade
DevOps Champion
Pranav is an earnest developer who is passionate about big data. He loves to build pipelines that accelerate the delivery time of our solutions to the customers. He loves sports in his free time.
