NEW Integration- Mesosphere DC/OS

We’re happy to announce our new Spotinst Elastigroup integration with Mesosphere DC/OS. Mesosphere DC/OS is based on the production-proven Apache Mesos distributed systems kernel and provides APIs for resource management and scheduling across entire data centers and cloud environments.

“Spot pricing on cloud instances is a great way to save money and scale clusters to meet demand—and is a natural fit for an elastic platform like DC/OS—but taking advantage of these pricing models can be tricky. By integrating its Elastigroup technology with DC/OS, SpotInst is making it even easier for users to overcome these obstacles and, hopefully, further reduce their monthly IaaS bills.

– Thomas Rampelberg, Product manager for DC/OS at Mesosphere

Benefits of using the Spotinst Mesosphere integration

1. Spotinst will automatically provision instances for you for the best availability and cost. You can even use scaling metrics or custom API scripts to automate this even further.
2. Running Spot instances manually on a production workload is unpredictable. You will need to carefully consider all scenarios in which your Spot instances can be interrupted and you also have to compensate for sudden market fluctuations.
   1. The Spotinst algorithm will predict when fluctuations in the market will occur and pre-emptively provision new instances for you well before the interruption.
3. The Spotinst API integration will automatically communicate with your DC/OS Master and send a connection draining signal to all instances affected. This ensures that new tasks are not added to servers that are about to go down. This feature sounds simple, but it makes the process of managing nodes completely automated!
4. You have both the availability your production workloads require and you get to save money.
5. Spotinst can provision a heterogeneous cluster for your entire DC/OS environment. Do you want to have exactly 30 vCPUs and don’t care about instances types? We can do that as well with our cluster weighting option.

How it Works

To start, let’s build a new DC/OS environment with the very convenient AWS Cloud Formation Template that can be found at the top of the installation guide here: https://dcos.io/docs/1.8/administration/installing/cloud/aws/ (if you have a cluster already you can skip down to step 7).

1. You’ll find a link to the cloud formation template at the top of the page. Select the region and click on “Launch Stack” link for “Single Master”.
   
2. You will need to fill in parameters for the template on the launch wizard like the key pair, slave node count, and OAuthEnabled. I kept everything to the bare minimum for simplicity sake.
3. The template will take about 10 minutes to deploy so go enjoy a nice cup of coffee or tea while you’re waiting.
4. Once the deployment has completed, you should now see the status change to “CREATE_COMPLETE” as you can see below. If you do not see this, make sure you are in the Cloud Formation console for the correct AWS region.
5. Click on the stack name to see details of the deployment.
6. Expand the “Outputs” and “Resources” sections.
7. Now let’s get the Master’s external IP address, go back to “Resources” from step 6.
   1. Scroll down to “ElasticLoadBalancer” and click on the link.
      
   2. Click on the “Instances” tab and then click on the instance ID for the master.
      
   3. Copy the public IP of the master into a text editor for later.
   4. You should see two security groups for the master, click on the MasterSecurityGroup.
      
   5. In the Security Group console, add a new inbound TCP rule for the MasterSecurityGroup as shown below (inbound Custom TCP Rule for port 5050 and the IPs defined here).

      

8. Login to the DC/OS console
   1. Take the IP you copied back in 8.3 above and paste this into your web browser.
   2. It will ask you to configure authentication, I used Google since I was already signed in.
   3. You should see at least three nodes (depends on your CF Parameters in step two above).
      
9. Now that the cluster is up and running let’s configure some Spot instances via the Elastigroup integration.
   1. Log into the Spotinst console at http://console.spotinst.com (free trial at https://spot.io/signup).
   2. Click on the “Create” button to create a new Elastigroup.
   3. At the bottom left of the screen click on the red “Import” button, then select “Auto Scaling Group” from the menu.
      
   4. Select the same region you used in step 1, and select the Auto Scaling Group with the name “-SlaveServerGroup-“. 
   5. Pick a descriptive name for your  Elastigroup and click next at the bottom right of the screen.
   6. Select “Linux/Unix” as the Product type.
   7. Configure the strategy and configuration as you see fit, I recommend a target of two instances for testing purposes.
   8. Scroll down to “Spot Types” and select m3.large, m3.xlarge, m4.large, m4.xlarge.
   9. Select the checkbox for Mesosphere DC/OS in the integration section. Paste in the API server from 7.3 above as port: 5050 as you can see below. Click on test connection (you should see a green checkbox if successful). Note: set as HTTP rather than https.
      
   10. Click “Next” at the bottom right of the screen.
   11. Leave scaling policies as they are and click “Next” at the bottom right of the screen.
   12. Review the JSON file and click “Create” to create your new Elastigroup configured with DC/OS.
   13. In about 10 minutes two new Spot instances should be created and automatically added as a node to your DC/OS cluster.

Before:

After:

Final Words

We hope you enjoyed learning more about our Mesosophere DC/OS Infrastructure-aware Scheduler integration. The official documentation can be found here. Give it a try today and let us know what you think!

Best Regards,

The Spotinst Team.