Using Elastigroup to Reduce Blockchain Mining Costs

Blockchain is the technology behind popular cryptocurrencies such as Bitcoin and Ethereum that can record transactions without the need for a trusted, central authority to ensure that transactions are verified and secure. Blockchain provides this by establishing a peer-to-peer network where each participant in the network has access to a shared ledger and by design,  transactions are immutable and independently verifiable. The downside to Blockchain mining is that it requires a lot of expensive computing power. However, Spotinst has figured out a way to help.

A way to reduce cloud computing costs is by using excess capacity instances with steep discounts known as Spot Instances. Spot Instance users can reduce compute costs up to 90% compared to on-demand instances. Spotinst Elastigroup can provide access to these flexible and cost-effective resources to quickly deploy and experiment with blockchain networks in minutes, and only pay only for what you use while utilizing cloud excess capacity resources effectively. In this post, I will go over how to get started with mining Ethereum and explain how Spotinst Elastigroup can manage the infrastructure and reduce costs.

What is Spotinst Elastigroup?

Elastigroup is an application scaling service by Spotinst designed to optimize performance and costs. Using predictive algorithms, Elastigroup reliably leverages Spot Instances and supports all major cloud providers such as AWS, Microsoft Azure, and Google Cloud while removing the risk and complexity, providing a simple orchestration and management at scale.

Spotinst Elastigroup predicts EC2 Spot markets behavior, capacity trends, pricing, and interruptions rate. Whenever there’s a risk of interruption, Elastigroup acts accordingly up to 15 minutes ahead of time, ensuring 100% availability. To play along with this blog post, you will need access to the Spotinst Console. If you do not have access yet, please sign up for a 14-day free trial.

Prerequisites

1. A Spotinst account.
2. An AWS account.
3. A VPC, Subnet, and imported IAM Policies per the Developers Guide for AWS Blockchain Templates.
4. Spotinst Ethereum CloudFormation Template.

Launching the CloudFormation Template

The Spotinst Ethereum CloudFormation template will automatically provision a new AWS Ethereum mining environment on Amazon ECS using Spot Instances managed by Elastigroup. Before importing the CloudFormation template, you will need to follow pages 8-15 of the Developers Guide for AWS Blockchain Templates to properly set up the VPC, Subnet, and IAM Policies. Cloud formation will fail to properly create the necessary resources if the instructions from the Developers Guide are done incorrectly

Please note that the CloudFormation template in this blog post is for the AWS Oregon region.

To deploy the template, access the CloudFormation module of the AWS console and click on Create Stack. Please make sure that you are in the Oregon Region to avoid issues.

Under Choose a template, click on Select a sample template and enter the following for the URL:

https://s3-us-west-2.amazonaws.com/spotinst-ethereum/ethereum-network.template.yaml

Click Next to continue.

You should now be on the Specify Details Page where the important deployment details are set. To get begin, enter any name you wish for the Stack Name.

Under Parameters, enter your Spotinst API Access Token and Account ID so Elastigroup can manage the CloudFormation deployment on Spot Instances. Under My Account in the Spotinst Console, you will be able to find your Account ID. To obtain your Spotinst token, click on the API section of My Account and generate a new token.

Cluster Orientation lets you decide if your Elastigroup should be focused more on cost-savings or availability.

A Balanced orientation is optimized for continuity and to have a cost-effective infrastructure. The AvailabilityOriented orientation is optimized for continuity and will abstain from replacing Spot Instances for as long as possible when interruptions occur or when cost optimizations are available. For CostOriented,  Elastigroup will take an aggressive approach to cost-savings by replacing Spot instances with less expensive instances when a less expensive market is available – even when no interruption is anticipated.

Spot Percentage is where you define the percentage of Spot Instances to use in this deployment. If you specify less than 100%, that percentage of your Elastigroup will consist of on-demand instances. It is best to leave it at 100% for maximum savings.

Under Elastigroup Autoscaling, you can configure the Elastigroup Autoscaler and set the minimum and maximum scaling capacity. If the Autoscaler is enabled, Elastigroup will deploy more ECS instances as needed.

Under EC2 configuration, choose your EC2 Key Pair. For Security Group and EC2 Instance Profile ARN, enter the values that were created per pages 8-15 of the Developers Guide for  AWS Blockchain Templates.

Under VPC Network Configuration, enter the VPC ID and the id of the subnet created per the Developers Guide for AWS Blockchain Templates.

For Application Load Balancer configuration (ECS only), enter the subnet ID from the previous step along with the Load balancer subnet ID created per pages 8-15 of the Developers Guide for  AWS Blockchain Templates. Note: ALB requires a minimum of two (2) subnets. Finally, enter the Security Group and IAM Role created per the Developers Guide.

Scroll down the page and click Next twice followed by Create. It typically takes 5-10 minutes for Cloudformation to create the stack.

Accessing the EthStats Web Console

The EthStats Console is a simple web interface that shows you the mining activities for each node. To view the EthStats Console, we will need to find the IP address of the web interface from CloudFormation.  When the stack creation is complete, click on the stack and then click on the Outputs tab on the bottom of the page.

Copy the Value for ExtEthStats into a new tab in your web browser. If successful, you should see the EthStats page.

Please be advised that for this AWS Ethereum use-case, a Bastion host (Proxy) was used to access the nodes and services of the ECS cluster. You can read more about it here.  For demonstration purposes, we allowed access from our office IP directly to the ECS instance.

With the EthStats page up and running, you will be able to see all of the mining activity across the cluster. With the Ethereum Stack up and running, let’s examine the newly created Elastigroup in the Spotinst Console.

In the original AWS template, accessing EthStats is only available through a Proxy server. In this demonstration, an ALB was added to allow remote access. Ethereum miners and clients statistics will be available when accessing the service internally and externally (through ALB). A Link to external EthStats can be found on the Output tab of Cloudformation; named ExtEthStats:

Viewing the Elastigroup

To find your CloudFormation deployment in Elastigroup, Navigate to the Spotinst Console and click on Elastigroups. In the search field, enter the name used when deploying the CloudFormation template and click on it.

This page lets us know how many instances are running and how long, and how much money Elastigroup saved versus using on-demand instances. After having this deployment running for 3 hours, I was able to save 67%.

Scrolling down the page, we can see all of the Ethereum Services and how many tasks are running:

Ethereum mining, in our example, is being performed by miners and are container based. To increase the number of miners in the cluster, increase the Desired tasks through the ECS cluster page.

Now, If you need to scale up the number of instances in this Elastigroup for more mining power to accommodate additional miners, click on Actions followed by Manage Capacity. Set the Target and Maximum values to 5 and click UPDATE.

After a few minutes, you can return to the Elastigroup and should see more instances running. The EthStats window should also reflect more instances running.  

Besides scaling up, Spotinst Elastigroup can also help you further reduce costs by automatically scaling down unwanted capacity by monitoring the cluster for idle instances. When an instance is found idle, Elastigroup will ensure that the running containers on the idle instance will find a new home on other instances using a process called Tetris Scaling.

Tetris scaling optimizes the task placement across the infrastructure of a cluster. When a new task starts on Amazon ECS, the ECS scheduler will try to find free capacity for it to run. When it is time to downscale or when a Spot interruption is imminent, the scheduler will drain the containers and reschedule them on other instances and terminate the idle instance.

Conclusion

In this post, I went over Blockchain and explained how to set up your own Ethereum Mining Deployment on AWS ECS using Spot instances managed by Elastigroup. Using Spotinst Elastigroup, I was able to reduce costs by 67% in only 3 hours and scale up the Ethereum Mining Cluster as needed. For more information on the capabilities of the Ethereum Deployment on AWS, please refer to the Developers Guide for AWS Blockchain Templates.