Cost Optimization: How MoneySmart Group Leveraged AWS to “Live and Breathe Efficiency”

Guest post by Eugene Ho, Senior Cloud Operations Specialist at MoneySmart Group

 About MoneySmart

MoneySmart.sg started life as a mortgage comparison company operating under the brand SmartLoans.sg. We have since expanded beyond mortgages to help consumers maximize their financial decisions by putting the power in their hands to compare loans, insurance, and credit cards. Today, MoneySmart Group is the largest financial portal in the Southeast Asia region, helping over 100 million people across Singapore, Hong Kong, Taiwan, and the Philippines to make smarter financial decisions.

Context

Amazon Web Services has been part of our journey for a long time and is central to the foundation of applications we build and host in MoneySmart today. Like all service providers available in the market today, it does not come cheap when your service footprint grows larger. The question that we constantly ask ourselves is whether we are overpaying for what we use, and is that totally necessary?

Our AWS Footprint

We have evolved our infrastructure over time, transiting from running applications on Amazon EC2 instances to containerized applications on Amazon ECS, and now looking to adopt Kubernetes using Amazon EKS.

Our AWS Spending Pattern

Need to Optimize Cost

Key drivers for cost optimization based on our spending pattern are:

• A month on month increase since June 2019, averaging $1,000 per month
• Increased cost driven by:
  • The rapid increase of newly launched applications
  • Abandoned environments where unused resources were not cleaned up
  • Too many tech stacks/debts that still existed
  • New project initiatives
  • No immediate attention to optimizing cost as cost was not one of the problems we were trying to solve in the beginning

Our Approach

We started out by analyzing our bills, identifying the outlier services, and further investigating the cost and usage pattern of these services using tools within AWS like AWS Cost Explorer and Trusted Advisor. At the same time, we engaged the AWS team to help provide insights on focused areas where we could target and what approach we could take to reduce costs. Based on the findings and insights, we took the actions below to start out our journey of cost savings.

EC2 Spot Instances

Utilizing spot instances, which let you take advantage of unused EC2 capacity in the AWS cloud and are available at up to a 90% discount compared to on-demand prices, is a great way to reduce compute cost. With the majority of our workload on ECS and EKS clusters, we started to integrate with Spot.io for cluster management and also to run our fleet of nodes with spot instances.

Spot.io for Spot Optimization

Spot.io is a cloud management solution that is primarily used to automate infrastructure with reduced cost. They enabled us to use a mixed pool of spot instances for our ECS and EKS clusters and at the same time provide container driven auto-scaling and resource right-sizing right of the box without much customization.

Savings

As an example, on our latest readings in Spot.io, we were paying $2,078 for Spot instances usage with savings of $5,656 per month.

We have managed to reduce the ratio of on-demand usage since June 2019 and also achieved significant saving over time of adoption with spot instances.

Cost of Spot.io

Spotinst charges 20% of the monthly savings, which still gives us a good proportion of savings, approximately 58%.

Non-working hours Policy

Non-working hours policy is a strategy to reduce cost by scaling down/shutting down resources when the office is closed, especially during the weekends. To implement this, we made an AWS Lambda function to scale down the EC2 instances of our staging environment from 22:00 to 07:00, Monday to Friday and every weekend.

Background on the Scheduler

•  Slack commands on a private channel to check status, start and stop the staging clusters on-demand using Amazon API Gateway
•  AWS Scheduled Task configured to run on the schedule
• Runs on Lambda and Python

Savings

With this policy in place, we have reduced close to 300 hours of compute cost in a 30-day month, giving us a daily saving of approximate $10 for weekdays and $30 for weekends.The estimated monthly savings were $610.

Reserved Instances for RDS

Now with compute running on spot and having a non-office hour policy in place, we started to work on capacity planning for Amazon RDS in order to plan for our commitment of database resources via Reserved Instances.

Evaluating the different options on the table, we decided on the most flexible option for us: opt-in for a 1-year term with no upfront payment.

It is a conservative approach as we only planned for databases that would still be relevant in a year’s time. And by going with no upfront payment, it did not require us to come out with the bulk of the payment at once.

Savings

The savings were modest at $240 per month, which was reasonable given that we could re-evaluate or accommodate architectural changes faster before committing to more RIs or changing instance family types when the plan expired. It fit our circumstances well.

Right-Sizing

Another focus area for reducing cost was to right sizing applications that are underutilized or over-sized. Starting off with legacy applications, we found quite a few over-sized applications running with auto-scaling groups that could be placed in smaller sized instances based on their usage pattern over a month.

Savings

This was a fairly simple and straightforward exercise that gave us a good amount of savings and also taught us the importance of keeping resource utilization in constant check. The daily savings were approximately $14, with an estimated monthly saving of $434.

Deactivation

Over time, we have accumulated a long list of services that we enabled. Most of these services are still active and running, with a bit of optimization required. And then there are some services contributing nothing to our cause or have not been properly deactivated, in short burning our cash unnecessarily. Based on our review, we deactivated the following services since they had been unused for a while and managed to save $332 for a month.

Snapshots

A part of housekeeping for compute was to archive the snapshots that were no longer relevant. We had snapshots from 2016 that were still active no longer required. Having a lifecycle to manage deletion of these snapshots helped to keep our inventory tidy. By cleaning up the snapshots, the estimated monthly savings were $142.

 Data Infrastructure

Similarly, our data team did the following to further reduce our overall spending as an organization:

●      Right-Sizing RDS, SageMaker, Redshift based on usage

●      Removed redundant service, DMS

●      Reduced EC2 usage from EMR and adoption of Spot

Effective Savings

Based on May 2020 billing, our initiatives as a group have given us an aggregated savings of $5,213 over 3 months.

At MoneySmart, we live and breathe efficiency and believe in smart use of AWS which delivers higher ROI to the company. This is not a one time activity and we will continue to monitor and bring in further efficiencies, optimizing our costs on AWS as we continue to grow at a rapid pace.