LogixPath’s Application Migration from Pivotal Web Services to AWS

Guest post by Linda Guo, CEO of LogixPath, in collaboration with AWS Solutions Architect Frank Wang

Introduction

LogixPath designs and develops operations management software for small to medium sized businesses to help them digitize operations. We focus on SME operations management essentials in the product development, sales order and work order management, product manufacturing, customer services, and collaboration. You can learn more about LogixPath’s products and services here.

As a startup, we leverage cloud providers for the heavy-lifting of our infrastructure and application services’ needs. We started out by using Pivotal Web Services (PWS) and its partners for our application platforms hosting. PWS recently decided to discontinue the services in order to focus on other business priorities, so with further objectives to boost our application performance, optimize our cost for the application hosting and administrate cloud assets more efficiently, we decided to migrate our application hosting services to Amazon Web Services (AWS). The migration experience was smoother than we expected, and more importantly, the outcome was great. In this blog post, we walk through our migration journey and conclude with the key lessons learned and our future plans around exploring and adopting more AWS services.

Prior To the AWS Migration

As a self-funded startup, we really focused on our core application functionality development, so it made a lot of sense to use a cloud provider to host and manage our infrastructure needs. We chose Pivotal Web Services (PWS) as our cloud provider as it was providing the advanced PaaS technology based on the Cloud Foundry platform and offered us a simple one-stop-shop experience for the needs of our infrastructure, web application servers, and databases services. The following were the PWS services we used prior to the AWS migration:

PWS services met our initial requirements of easily managing our application and databases. As we progressed, however, we discovered some challenges around performance and deployment downtimes. Since performance and release management consideration are a key drivers for us, we began to consider other cloud hosting alternatives, especially as PWS was discontinuing its services. With an initial self-conducted evaluation and further introduction and deep dive on the relevant AWS services, we quickly concluded that AWS would be a good fit and embarked on our AWS migration journey.

The Migration Journey

We understood that AWS provides all the functionality our application used in the PWS Cloud. In fact, for each service we need, there were corresponding AWS alternatives. We spent about a week reading through all relevant AWS document, and came up with a list of service candidates. With the help of an AWS Solutions Architect (SA), we learned about the pros and cons of the targeted services and potential challenges we were facing (such as the quite old database version in the current hosting environment). He also provided guidance and best practices around adopting these services and mitigation plans for potential challenges, which gave us a higher confidence for the migration. Essentially, our evaluation was based on the following criteria:

·       Hardware specifications and software requirements
·       Ease of ongoing maintenance
·       Cost
·       Scalability for the current load and future growth

After the evaluation and analysis of the relevant AWS services, we mapped out the application deployment architecture on AWS as depicted in the following diagram.

In the following sub-sections, we will describe in more details our migration process for each different application components.

Database Migration

Data migration was our biggest concern, even though we had 10+ years of enterprise data migration experiences. LogixPath’s production database contains real customers’ live business operations management data, including sales orders, manufacturing data, etc., so, the data migration had to be 100% accurate.

Our initial challenge was the PostgreSQL database version. In the PWS environment, we were using PostgreSQL 9.2.13, while AWS supports the more recent versions from 9.5 through 12.4. To mitigate the initial migration risk, the SA suggested migrating to as close to the currently used version as possible first and then using in-cluster PostgreSQL to upgrade to the newer version in the future. So, we decided to go with 9.6.19 (the last 9.x version) and plan to upgrade to the higher versions in the future.

As suggested by AWS documentation, we used the native database migration tool pg_dump and psql to export and import the data (by following the AWS migration guide here). We first imported ElephantSQL data to the free tier AWS dev environment and resolved issues encountered during the data migration. The actual production data import went very smoothly. See the following diagram for the PostgreSQL database migration flow. The data migration went surprisingly fast. It took 41 minutes to import a 3.3 GB database file which contains a lot of foreign key relationships among records which requires a lot of index building while importing. (NOTE: as of this blog post writing, it has been 30+ days since the migration, so far, we haven’t seen any data issues).

Applications Migration

For application migration, we loved  AWS Elastic Beanstalk. It’s a magical framework! With just a few clicks, we got everything needed for serving a web application, from the “physical machine” (Amazon EC2), to Tomcat web server, and all the way to the load balancer. Additionally, because is a white box instead of black box service, we can see and administrate all the components and services auto-generated when creating an Elastic Beanstalk application. Under PWS, we could only adjust very basic configuration (such as application RAM), but were not able to troubleshoot environment issues at all. The Elastic Beanstalk made our application migration smooth and frictionless.

Other AWS Service Adoption

The other services we loved and adopted as part of the migration were AWS Certificate Manager and Amazon Route 53. Since LogixPath software provides SMEs customers with the company website/portal builder and hosting capability, we needed to map multiple custom domains and certificates to a single web application. So, our requirement was to have a service that helps us manage multiple domains and certificates in an easier and cost effective manner. In PWS, though we could route multiple custom domains to one web application, we could not do so for the SSL certificates management for the following reasons:

1.     Each SSL service costs $20/month (plus developers need to upload their own certificates purchased from somewhere else). This is not practical for our SME customers.

2.     Technically, we were not sure if the PWS SSL service was able to map multiple SSLs to a single web application. Without SSL, our customers’ web sites were “Not Secure.”

In the AWS Cloud, we can create SSL Certificates via Certificate Manager and map multiple certificates to a single Elastic Beanstalk application’s load balancer listener. As a result, websites built and hosted by LogixPath software in AWS are more secure. (See a live example at: https://site.bayartacademy.com.)

In conjunction with Amazon Route 53, it is very easy to route multiple custom domains to an Amazon Elastic Beanstalk web application.

Post Migration Retrospectives

The migration itself did not take much time to complete. It took less than 1 month from learning and evaluating AWS to officially launch LogixPath on AWS. Since the migration completed, we have already observed the following benefits:

·       Performance: LogixPath application in AWS performs very well. It is responsive and fast (we were not able to provide quantitative improvement data at the moment, but hope to have more performance metrics comparison to share in the near future). One noticeable difference was that we no longer see any application “dormant” mode occurrences, which we used to observed frequently in the old PWS environment.

·       Ease of software update deployment with almost no downtime: One Amazon Elastic Beanstalk application can have multiple versions of software. Our old deployment process used to be as follows:

1.     Stop our application in the PWS cloud

2.     Upload new version of software from local computer to PWS cloud

3.     Start our application in the cloud

4.     Total downtime: 6-10 minutes (depending on network speed and potential internet interruptions)

With Amazon Elastic Beanstalk, our new deployment process was simplified as follows:

1.     Upload new version of software from local machine to AWS Cloud

2.     Deploy the new version of software

3.     Total downtime: < 50 seconds (independent of network speed)

Notes:

1.     The Elastic Beanstalk document also indicates that we can even achieve zero downtime by setting Immutable Updates. Although we haven’t tried it yet, we believe we can potentially further minimize our deployment down time in the future.

2.     With the software versioning feature, rollback software deployment also becomes much easier in the Elastic Beanstalk environment.

·       Better monitoring capability: Amazon CloudWatch is a very useful monitoring tool, especially the CloudWatch Alarm feature with text or email notifications. We used to periodically login to hosting environment console or LogixPath application to view the current system health. With Amazon CloudWatch Alarm Notifications, we do so only if we receive any notifications in emails or text messages.

·       Better Price/Performance Ratio: With PWS, our total cost used to be about $85/month (including application and database). With current AWS deployment, our projected total cost will be about $115/month. (NOTE: this is based on the On-Demand rate not Reserved Instances). However, in consideration of the benefits we got:

1.     For the compute spec (memory and storage) in AWS, it is double of what we had in PWS.

2.     For database, we have our own dedicated AWS PostgreSQL instance, while in ElephantSQL, we used shared instance.

3.     SSL service: AWS provides SSL certificate and routing services with minimum cost ($0.50/month per domain)

NOTE: We also learned to apply for AWS Activate, a program that provides business and technical support as well as developer credits to startups. Special thanks for AWS team to grant us those credits to facilitate startups to start using AWS! We sincerely appreciate AWS’s support. This program is extremely helpful for self-funded entrepreneurs.

Of course, not everything is perfect. We did notice the side-effect issue once we migrated our applications to AWS, including that there are many more DDoS (hacking) attack attempts than before. LogixPath application monitoring subsystem filters malicious IPs addresses in the “blacklist.” After deploying to AWS, the frequency of being attacked and severity has been much higher than before. We designed and developed an “auto-detect-and-block” mechanism to cope with the situation. We also read an AWS white paper “AWS Best Practices for DDoS Resiliency,” and learned more about AWS security services such as  Amazon VPC, Amazon Shield, and AWS Web Application Firewall (AWS WAF), we had since then setup the CloudWatch monitor and notification, plus using VPC ACLs inbound rules to deny “brutal spam” IPs. Together, the AWS “infrastructure” layer’s protection plus LogixPath application layer preventive mechanism worked very well to alleviate the attack situations.

Conclusion

In this blog post, we shared our experience of the migrating journey at LogixPath from PWS to AWS. In the past couple of years, several people suggested us to switch to use AWS. Human beings usually do the things in the way we get used to do and hesitate to make changes. Although the main reason that led us to this migration was the discontinuation of VMWare’s Pivotal Web Services, we did have options to continue with another VMWare cloud hosting service or choose other cloud providers. We selected AWS because we were impressed by the breadth and depth of the AWS technologies and cloud services. The free tier offering also helped us freely explore and experiment with various technologies so that we could choose the right hosting architecture for LogixPath’s application.

As for future plans, several AWS services were brought to my attention: AI/Machine Learning services, Blockchain, and AR/VR, to name a few. These technologies can benefit business operations greatly. As our application complexity and functionality increase along with the business growth, we do expect we to explore and adopt more AWS services in the future. We would also highly encourage other small startups like us leverage the free-tier and credit offering and enjoy the ease-of-use of AWS services to start your migration journey.

About the Authors

Linda Guo is the founder and CEO of LogixPath, LLC. She is currently on the journey to apply her computer science and engineering academic knowledge and 20+ years enterprise software design, development, and business management experiences to help small businesses digitize daily business operations management with LogixPath software.

Frank Wang is a Startup Senior Solutions Architect at AWS. He has worked with a wide range of customers with focus on Independent Software Vendors (ISVs) and now startups. He has several years of engineering leadership, software architecture, and IT enterprise architecture experiences, and now focuses on helping customers through their cloud journey on AWS.