Amazon Web Services (AWS) is the world’s leading cloud computing platform, providing hundreds of services covering compute, storage, networking, and platform as a service (PaaS) offerings such as managed databases and container orchestration.
Optimizing AWS costs involves monitoring and analyzing your usage, identifying opportunities to reduce costs, and implementing recommended actions.
Most Amazon services are based on compute resources provided by the Elastic Compute Cloud (EC2), so the cost of EC2 instances is probably a central factor in your AWS costs. Many optimization strategies involve reducing the cost of EC2 instances—for example by switching instances from on-demand pricing to lower-cost spot instances or reserved instances, or applying savings plans which can reduce costs across your Amazon account.
In this article:
The AWS cloud provides over 200 services. Cloud resources are dynamic and cost can be unpredictable and difficult to manage.
Here are the main causes of waste and high costs on AWS:
Here are five cost optimization pillars you can apply to most environments:
Here are several AWS cost optimization tools available free from Amazon:
To learn other ways to reduce your cloud costs, see our detailed guide to AWS cost savings
Image Source: AWS
The Cost Explorer interface enables you to view costs, usage and return on investment (ROI) for AWS services. The interface displays data for the past 13 months and can help you forecast your future spend. You can use the interface to create customized views, which can help further analyze your AWS costs and also identify certain areas for improvement. Additionally, the AWS Cost Explorer provides an API, which enables you to access data through your existing analytics tools.
Learn more in our detailed guide to the AWS Cost Explorer (coming soon)
Image Source: AWS
AWS Budgets can help you set and enforce budgets for each AWS service. When budgets are exceeded or reached, you can receive messages or emails from the Simple Notification Service (SNS). You can define an overall cost budget or connect a budget to certain data points, including data usage or the number of instances. The tool provides dashboard views similar to those generated by the Cost Explorer, displaying how each service is used compared to its budget.
Image Source: AWS
AWS Pricing Calculator enables you to estimate the cost of use cases on AWS. It lets you generate monthly cost estimates for all regions supported by a certain service. You can model a solution before building it, explore the pricing points and calculations of your estimate, as well as find available instance types and contract terms that meet your requirements. This can help you make informed decisions, plan your AWS cost and usage, as well as estimate the costs of setting up a new set of instances and services.
The following best practices can help you optimize and reduce your AWS costs.
To use the AWS Management Console, CLI or SDK, you must first choose a region. Most users base this decision on distance. However, the AWS region you choose also affects costs, latency and availability. Here are several aspects to consider when choosing the right AWS region for your project:
When accounting for all above aspects, you can prioritize them to choose the most important aspect and let it guide you as you choose a region.
When optimizing your AWS costs, you must pay attention to unused instances and shut them down. Here are several practices to consider:
The AWS Cost Explorer helps you visualize and manage the cost of Amazon services. The tool provides a Resource Optimization report, which shows idle or under-utilized EC2 instances. You can lower your costs by stopping or scaling down these instances.
Here are three tools to help you to stop wasting money on low utilization EC2 instances:
Amazon EC2 spot instances are an important option for reducing AWS costs. Spot instances provide savings of up to 90% compared to normal on-demand prices.
Spot instances are Amazon’s way to sell off spare EC2 capacity. They can significantly reduce Amazon EC2 costs by allowing you to request the same EC2 instances, when they are in low demand, at a significantly discounted price.
The downside of spot instances is that they are unreliable—if Amazon needs to reclaim the capacity for on-demand or reserved users, it terminates the instance with a two-minute warning. Amazon has recently introduced “rebalancing signals” which may provide earlier warning of spot instance termination, but this is not guaranteed. Another option to improve reliability is to run spot instances in an Auto Scaling Group (ASG) together with regular on-demand instances. This way at least some of the capacity in the group is guaranteed to remain available.
You can overcome these challenges and use spot instances even for mission critical or production workloads that cannot tolerate interruption with cloud optimization tools like Elastigroup from Spot by NetApp. Elastigroup provides AI-driven prediction of spot instance interruptions, and can perform automated rebalancing of workloads, to eliminate the “surprise factor” and risk of spot instance termination.
An ASG is a collection of Amazon EC2 instances, treated as a logical group for automatic scaling and management purposes. ASGs can take advantage of Amazon EC2 Auto Scaling features, such as health checks and custom scaling policies based on application metrics or preset schedules. You can dynamically add or remove EC2 instances from an ASG based on predetermined rules or dynamically, in response to application loads.
ASGs let you scale your EC2 fleets up or down as needed to conserve costs. You can view scaling activity by using the describe-scaling-activity CLI command, or the Auto Scaling Console. By optimizing scaling policies, you can reduce costs both when scaling up and down:
Here too, a tool like Elastigroup can help by exactly matching scaling events to application requirements, ensuring there are no wide margins that waste capacity. This can also save your team time, because there is no need to constantly monitor and fine tune auto scaling policies.
Learn more about using Elastigroup to optimize EC2 auto scaling
An Amazon Reserved Instance (RI) lets you commit to using an instance for periods of one or three years, providing a discount of up to 72%. There are several options you need to consider when committing to reserved instances:
Because RIs represent a long-term commitment, you may find yourself with unused reserved instances. This is why initial planning is important:
Here too, automated optimization tools can be useful. Eco from Spot by NetApp can help you automatically adjust your portfolio, to offload unused reserved instances, or relocate workloads from expensive on-demand instances.
Learn how to use Eco from Spot by NetApp to optimize the use of EC2 reserved instances
The Savings Plan is a flexible pricing model that lets users use EC2, Lambda, and Fargate at a lower cost, by committing to a continuous usage, measured in terms of USD per hour, for a period of 1 or 3 years. For example, a one-year Savings Plan with no upfront payment plan grants a discount of up to 54%.
Savings Plans apply to compute instances regardless of size, Auto Scaling Group, Availability Zone, or region. You can use AWS Cost Explorer to select the right options for your plan, based on an analysis of recent utilization.
Image Source: AWS
While Cost Explorer provides useful recommendations, your usage of Amazon services is dynamic, and there is a need to continuously manage and optimize your commitments. A cloud optimization tool like Eco constantly evaluates cloud usage and manages Savings Plans and reserved instance lifecycles automatically, to ensure you make the best use of long term commitments and achieve maximal discounts.
Learn how to use Eco from Spot by NetApp to optimize the use of EC2 savings plans
Monitoring can help you track actual usage and identify pricing patterns. AWS lets you monitor S3 usage via the S3 Analytics tool, which evaluates storage access patterns on a specific object dataset for a duration of 30 days or more.
The S3 Analytics tool offers recommendations that can help you leverage S3 Infrequently Accessed (S3 IA), AWS Glacier, or AWS Glacier Deep Archive for cost reduction.
You can then use lifecycle policies to automate the transfer of objects into a lower cost storage tier. Alternatively, you can use S3 Intelligent-Tiering for automatic analysis and transfer of objects with unknown or dynamic usage duration into a relevant storage tier.
In addition, a great way to save storage is to delete unused Elastic Block Storage (EBS) volumes - managed disk drives you can attach to EC2 instances. Even after the EC2 instance shuts down, EBS volumes can continue operating and incur costs.
Make sure teams are aware that when they use EBS volumes, they should select the option to Delete on termination. This will ensure that the EBS volume is deleted when the EC2 instance is terminated. If this was not done, look for EBS volumes that are marked as available—you can do this via Amazon CloudWatch or AWS Trusted Advisor, and clean them up automatically using a Lambda function.
When you terminate an EC2 instance, the associated EBS volume is also deleted by default. However, it's easy to forget that snapshots you created as backups of those EBS volumes are still in S3, and you're still paying an ongoing monthly fee to store them.
EBS backups are usually incremental, so each additional snapshot takes up limited storage space. However, if you perform frequent snapshots and have a high retention period, even this small incremental addition can add up over time.
Because most snapshots use data from the initial snapshot of the entire EBS volume, it is important to track down and delete the initial snapshot, if it is not needed. This can save much more storage space than deleting numerous incremental snapshots.
It is best to set up automated lifecycle management of EBS snapshots, via Amazon Data Lifecycle Manager, to ensure that you do not maintain snapshots for longer than needed.
Check your Elastic Load Balancing configuration to see which load balancers are not currently used. Every load balancer incurs ongoing costs. If your load balancer doesn't have any backend instances associated with it, or network traffic is very low, it is not being used effectively and is wasting resources.
You can use AWS Trusted Advisor to identify load balancers with a low number of requests (a good rule of thumb is less than 100 requests in the last 7 days). Reduce costs by removing idle load balancers—you can track overall data transfer costs with Cost Explorer.
Image Source: AWS
Another way to reduce costs, if you have high data transfer costs from EC2 to the public web, is to use Amazon CloudFront. CloudFront is a Content Delivery Network (CDN), which lets you cache web content in multiple edge locations around the world. This can significantly reduce the bandwidth required to serve spikes in traffic.
AWS Enterprise Billing Console is a new service that lets you manage chargebacks—billing Amazon services to units in your organization, or to third parties. Within an organization, this allows you to create accountability by billing each department or business unit, according to the cost of services they actually use.
The Enterprise Billing Console lets you allocate your costs across accounts, and use the concept of billing groups. Billing groups let you apply customized pricing plans to each department or business unit. Set up Cost and Usage (CUR) reports for each of your billing groups, and perform margin analysis to calculate savings for each group as a result of cost optimizations.
Another feature of the console is dashboard graphs, which offer default views of your overall spending, as well as month-to-date spend and service consumption by each group.
Spot by NetApp offers Eco for MSP which can be used by enterprises looking to easily manage cost, internal billing and showback for various business units.
Related content: Read our guide to AWS cost management (coming soon)
While AWS offers Savings Plans, RIs and spot instances for reducing EC2 cost, these all have inherent challenges:
Spot by NetApp addresses these challenges, allowing you to reliably use spot instances for production and mission-critical workloads as well as enjoy the long-term pricing of RIs without the risks of long-term commitment.
Key features of Spot by NetApp’s cloud financial management suite include:
Learn more about Spot by NetApp’s cloud financial management solutions
Complete access
for up to 20 instances