AWS Partner Network (APN) Blog
Rapid Application and Database Assessments Using migVisor by EPAM
By Gal Licht, Technology Consulting – EPAM Systems
By Andre Boaventura, Global Principal Solutions Architect – AWS
 |
| EPAM Systems |
 |
Building future-ready applications that support today’s growing volume of data requires moving to a modern data infrastructure that lets you save time and costs while improving performance, availability, and scale.
Running on-premises or self-managed databases on the cloud creates tedious management tasks, such as provisioning, patching, and backups.
Many on-premises databases live with old-guard, commercial-grade database providers that are expensive, proprietary, and have high amounts of lock-in and punitive licensing terms. As a result, some customers are moving as fast as they can to open-source databases.
However, getting the same performance on open-source databases as you get on commercial-grade databases turns out to be challenging most of the time.
The most straightforward and simple solution for customers who are struggling to maintain their own relational databases at scale is to migrate and modernize to managed database services like Amazon Relational Database Service (Amazon RDS) or Amazon Aurora by following the AWS Migration Acceleration Program (MAP).
In this post, you will learn how EPAM’s database assessment tool, migVisor, can play a key role in the overall MAP methodology. Additionally, you’ll learn how using migVisor in combination with other AWS assessment solutions and services can help you gain a holistic view of your environment and make informed decisions to determine the feasibility of migrating your databases and correlated applications to AWS.
EPAM Systems is an AWS Premier Tier Services Partner with the Migration Consulting Competency. EPAM simplifies your infrastructure and applications by moving them to the cloud and eliminating technical debt so you can focus on your business objectives.
Three-Phase Methodology for Migration
Having a comprehensive migration plan with prescriptive guidance about the capabilities of AWS managed database services, refactoring operations, and necessary tooling are paramount for a successful rollout and implementation.
The impact can be significant if mission-critical database migrations are not properly planned, managed, and executed based on well-established business outcomes, capabilities, constraints, and risks that consider existing operations and technology stacks.
The most common issue with creating a complete plan for migration is that many companies devote the majority of their IT resources to supporting existing workloads, and only a small fraction of their IT budget is allocated to innovation.
Aiming to address this gap, AWS MAP helps customers prioritize and accelerate these initiatives by providing:
- Methodology for migrating and modernizing workloads on the cloud.
- Robust set of tools to automate the most common scenarios.
- AWS funding to help offset the overall cost.
To help customers achieve their migration goals, MAP uses a three-phased framework (Assess, Mobilize, and Migrate and Modernize), which is adopted to drive large database migrations and modernization projects.
The “Assess” phase is an instrumental step toward a successful database migration and modernization project implementation, since it helps customers predict how much the new environment will cost.
In this phase, the main goals include identifying customer business drivers to execute migration, formalizing business value metrics with executive sponsors, and reviewing how ready the customer is to start a cloud migration and modernization journey using mechanisms like the Migration Readiness Assessment (MRA).
After the Assess phase, customers should have a clear understanding for the rationale behind moving to AWS, including business, costs, and technical drivers.
How EPAM’s migVisor Accelerates the MAP Assess Phase
To further optimize the MAP Assess phase, EPAM uses migVisor for AWS, a unique application and database assessment product designed to assess heterogeneous, cross-database engine migrations.
migVisor paves the way for large database migration and modernization projects by providing all database business metrics and inventories, helping you identify the best migration targets that may be used during the MAP Assess phase, for example.
migVisor currently supports the following database engines as sources: Oracle, Microsoft SQL Server, PostgreSQL, and MySQL. migVisor acts as a precursor to AWS Database Migration Service (AWS DMS) and AWS Schema Conversion Tool (AWS SCT), helping in the pre-planning phase of database migration projects.
The solution focuses on the user’s entire ecosystem, which enables it to capture the following data points:
- Database features, including Oracle Virtual Private Database, SQL Server Always-ON, PostgreSQL extensions, and more.
- Database and application interdependencies—it also lists the database links and the applications and clients using the database.
- Workload and target sizing characteristics based on details included in the database. migVisor captures and presents data on the CPU, RAM, I/O usage, data change rate, and transaction processing vs. data warehouse behavior indications.
Acquiring these data points and details results in a more complete, accurate, and confident assessment prior to a migration project, allowing the customer to plan where to start and which applications to consider first.
migVisor’s kick-start to the planning process also accelerates timelines to the actual migration phase when a customer utilizes AWS DMS and AWS SCT.
migVisor Overview
EPAM’s migVisor collects over 300 database attributes for assessment and performs an application data access layer assessment to determine the amount of effort required to refactor a database from one database engine to another, and to adjust the associated applications to work with the new database.
Using the migVisor Metadata Collector (mMC) instead of a long-running agent, migVisor queries only metadata information. Since migVisor gathers metadata only and granting access to user/application data is unnecessary, data security is high. The collection is also fast, running in 10 minutes or less. The user creation scripts for the mMC can be found in the migVisor documentation: migVisor required permissions.
For every migration path, migVisor keeps different scores for the diverse features detected. Oracle-to-PostgreSQL is a prime example for distinct scores and features. These scores allow migVisor to use internal formulas to provide the overall migration score (low/medium/high).
In addition to database features, migVisor collects other data points to provide more information about sizing, application interdependencies, coupled databases, and database behavior/usage (online transaction processing vs. data warehouse).
All of these details are combined into dashboards that help managers and database administrators (DBAs) view the database inventory and potential migration paths for each database or schema. These dashboards include information that help support decisions about which applications to move or refactor first.
Quick Start with migVisor
migVisor is divided into two main components: the migVisor Metadata Collector (mMC) and the console. The mMC is a small Java utility that can run from any machine that has network connectivity to relevant databases.
This utility does not have to be installed anywhere and usually is executed from a DBA’s laptop or a jump server (bastion). Complete information on mMC can be found in the mMC-DS GUI documentation.
Following is an example of an active scan, its progress, and the status.
Figure 1 – migVisor mMC list of source databases to be scanned.
migVisor, unlike other assessment tools, is agentless; it scans databases and features relevant information from historical tables without requiring an agent to monitor and fetch information for an extended period.
Unfortunately, for some database engines the workload capture capabilities are limited, so migVisor is now supporting an extended option that runs a few queries to capture workloads over time.
The other main migVisor component is the console, where you can upload databases and review on-the-fly assessments for different target databases. The migVisor console allows users to choose the desired migration path (like target database) to see the outcome and complexity.
It also helps users easily see—especially for a large fleet of databases—which databases are the easiest to migrate and refactor and which are more complex and will require more time to refactor.
Figure 2 shows the migVisor console’s “Portfolio” view, where summary information is presented. In the upper left section are the main features being used across the account. In the upper right section are the scanned application repositories and the complexity to adjust these. Users can then compare database complexity scores in the context of the specific target.
Figure 2 – Portfolio view in the migVisor console.
When you need more in-depth information, the following migVisor components show important database specifics:
Overview and Details Section
The “Overview and Details” section offers a summary of all findings and relevant details about the source database.
Figure 3 – Overview and Details section of the migVisor console.
Features Section
The “Features” section presents information about how the migration complexity was calculated; it’s important to know that some non-schema-level features are presented here and can affect the solution or approach.
The features identified in the source database are divided into three sections: high/medium/low migration impact. Each feature falls into the relevant category based on the migVisor proprietary complexity score for it, the number of times that feature was used, and the specific target that was chosen.
Figure 4 – Features section of the migVisor console.
These features can be extended to understand where they were detected in the database, general info about the feature, and information on viable solutions. For example, the user could exclude schemas and the score will update automatically to find additional information.
Figure 5 – migVisor location of detected feature and feature information.
Target Sizing Section
The “Target Sizing” section presents crucial information like user transaction per second, average CPU time ratio, average instance CPU utilization, and other information about database workload and potential target sizing for the resources that serve the future database.
It helps you understand if the database is over- or under-provisioned and if performance tuning efforts are needed during the database migration project.
Figure 6 – Target Sizing section of the migVisor console.
Application Assessment Section
migVisor also features an “Application Assessment” section. To cover most of the scope, migVisor analyzes application code to assess how complicated it will be to adjust the application to use the new database engine.
The application assessment scope can be viewed in the migVisor console, and it is broken down into three areas:
- Language breakdown: Shows the application languages that were captured.
- Overall complexity: Mentions the complexity distribution of adjusting the applications.
- Modern vs. legacy: Shows if applications can be considered modern and will be easier to adjust, and how many are legacy and more coupled to the current database engine.
Figure 7 – Application Assessment section of the migVisor console.
EPAM’s Refactoring Methods and Process for a Successful Migration
EPAM believes a successful migration is driven by a combination of quick wins, meticulous planning, and business requirements being met.
Although it’s tempting to remove licensed databases and refactor to non-licensed databases, it’s especially important to understand the migration investment required and decide if it makes sense and provides return on investment (ROI).
Using migVisor, you can easily see which portions of your organization’s databases will be rehosted or refactored—these being the two different timelines for potential projects. While rehost can be considered an easier solution, it still requires some effort to ensure all features being used today will be available and operate the same on the target solution.
Figure 8 – Assessment and migration process.
For the databases being refactored, it’s recommended to gather information after the migration and look for additional aspects that can be tuned and/or enhanced for scalability and cost-efficiency.
Conclusion
The desired outcome of migrating applications and databases during a migration can be hindered by several factors, including a lack of planning, time, cost, and accuracy. Using a proven methodology like AWS Migration Acceleration Program (MAP), AWS services like AWS DMS and SCT, and automated tools like EPAM’s migVisor accelerate the vital assessment phase and add confidence for the eventual migration itself.
EPAM Systems leverages agile methodologies, proven customer collaboration frameworks, engineering excellence tools, hybrid teams, and its award-winning proprietary global delivery platform to roll out transformative AWS solutions to clients.
Before starting your next migration or modernization program, learn more about how EPAM combines migVisor with its engineering, consulting, and delivery capabilities to accelerate and build confidence in each phase of the project. Visit the migVisor by EPAM page, review migVisor documentation, or reach out to EPAM for more information.
.
.
EPAM Systems – AWS Partner Spotlight
EPAM Systems is an AWS Premier Tier Services Partner that simplifies your infrastructure and applications by moving them to the cloud and eliminating technical debt so you can focus on your business objectives.