How Ness Digital Engineering Replatformed and Modernized Instinet’s Trading System

By Subir Grewal, Global Head, AWS Practice – Ness Digital Engineering
By Hari Ramesh, Sr. Partner Solutions Architect, Analytics – AWS
By Maysara Hamdan, Partner Solutions Architect – AWS

Ness Digital Engineering

Instinet is a software-as-a-service (SaaS)-based solution that Ness Digital Engineering built on Amazon Web Services (AWS) for the trading and finance industry. It provides technologically advanced, agency-model brokerage services to institutional clients worldwide and is the wholly-owned execution services arm of Nomura Group.

Instinet helps institutional investors create, protect, and capture alpha, reduce complexity, and lower overall trading costs to improve investment performance.

Ness Digital Engineering is an AWS Premier Tier Services Partner with Competencies in DevOps, Financial Services, and Migration. Ness is a global full lifecycle digital services transformation company and has been named a leader in overall Engineering, Research, and Development (ER&D) for two consecutive years by Zinnov Zones.

This post presents a reference architecture for a scalable trade processing and reporting system which preserved Instinet’s intellectual property while modernizing critical portions of the business process. Instinet’s goal was to address performance bottlenecks that impacted the system’s ability to meet service-level agreements (SLAs) on high volume days.

Instinet also sought to break down the rigid sequential batch-driven approach embedded in its data pipeline to allow room for growth while allowing the organization to manage towards business SLAs. The system has a multi-tenant architecture, and Instinet wished to have the option to stand up dedicated tenants for customers that required it.

Customer Challenges

Instinet’s original on-premises infrastructure faced scaling issues amid continued, exponential increases in market trading volume. On high-volume trading days, the primary database became a bottleneck for both transactions and post-trade reporting and reconciliation.

Further, a rigid and tightly-coupled data pipeline along with sequential processing meant delays cascading through the system, both delaying intra-day processes and end of day processes alike. As a result, technology and non-functional limitations began to directly impact revenue growth prospects.

The high fixed cost of deploying on-premises infrastructure limited Instinet’s ability to serve clients who wanted fully segregated environments. This also posed challenges of long-date contracts and commitments from end customers, and Instinet’s leadership recognized that a migration to AWS would help remove these constraints and accommodate future growth.

Instinet’s equities brokerage business regularly ingests and processes 1TB of trade data daily. The on-premises environment was built around a MySQL database that served as both a transaction store and reporting warehouse. The database had accumulated several petabytes of data throughout its 50+ years, and this data was required to provide historical reporting capability. Maintaining it in a MySQL OLTP database resulted in a heavy operational and capital-expenditure load.

By maintaining its core database (“CoreDB”) in a physical data center, Instinet faced challenges and logistical inefficiencies that impacted the development of new products and created potential risks to the customer experience.

The architecture diagram below describes the data pipeline at Instinet prior to modernization and migration to AWS. The database is central to all critical operations and consequently becomes a bottleneck for both recording transactions and reading them.

Figure 1 – Instinet trading system on-premises data pipeline architecture.

Data collected from various sources was naturally generated in the form of events over the course of the day, but the legacy architecture forced it into batch mode. Source data included summary and raw trade/execution data, statistical data on trading, market events, and actual Financial Information eXchnage (FIX) feeds. These were delivered over different protocols.

Analytical operations for the batch were embedded in various execution platforms, including database stored procedures and stand-alone applications.

Design Requirements

Ness was tasked with designing and implementing a cloud solution that would meet Instinet’s current and future growth requirements.

The design had to meet several requirements:

• Deliver near real-time processing for incoming transaction data. Instinet wished to go from hours to minutes or seconds for specific processes. For example, end-of-day reports like trade day plus one reporting required for the next day, and intra-day functions to support real-time decision making like collateral management or block trading.
• Support all of Instinet’s current applications with minimal modification.
• Deliver a highly-available and fault-tolerant design with recovery time objective (RTO) requirements on the order of minutes and recovery point objective (RPO) requirements of seconds.
• Deliver a working solution that solved several critical performance bottlenecks, within months.

Solution Overview

Ness took a data-first migration strategy that re-orchestrated Instinet’s existing business logic in readiness for optimal cloud deployment. Ness teams created a data ingestion pipeline that utilizes AWS managed services for horizontal scaling, high availability, and ease of operations.

The complete data ingestion and analytics pipeline developed by Ness was built using managed services on AWS, as Instinet’s technology organization had limited experience with cloud infrastructure, like Apache Kafka or Apache Flink.

Using AWS managed services allowed the rapid adoption of new technology with AWS’s management and observability tooling baked in. The modernized data pipeline is highly-available thanks to the multi-Availability Zone (AZ) deployment, and scalable with commodity hardware since key data ingestion steps are parallelized in Apache Kafka and Apache Flink.

Figure 2 – Instinet trading system modernized data pipeline architecture.

The data ingestion and analytics pipeline had seven key stages:

1. Data loaders are deployed on Amazon Elastic Compute Cloud (Amazon EC2) to securely pull data from on-premises sources. These include applications for loading market and reference data and for providing summary and statistical analysis on the data.
2. Amazon Managed Streaming for Apache Kafka (Amazon MSK) holds Kafka topics, allowing for replay and re-ingestion of data. Amazon MSK allows for parallelizable processing, and Kafka topics are consumable by different systems at Instinet.
3. Amazon Managed Services for Apache Flink consumes data from Kafka topics and persists data to Amazon Aurora.
4. Amazon Aurora, a MySQL database service, provides an alternative solution to Instinet’s on-premises MySQL database.
5. Amazon Redshift as a multi-year data warehouse of historical trading and reference data provides long-term queryable storage alongside Aurora.
6. AWS Lambda and AWS Step Functions automate all weekly operational processes that were normally handled by database administrators and operational support.
7. Additionally, trade-related data was moved off of on-premises storage into tiered storage in Amazon Simple Storage Service (Amazon S3).

Solution Impact

The new pipeline services model using Amazon MSK and Amazon Managed Services for Apache Flink has re-orchestrated Instinet’s existing data capture and distribution logic by providing near real-time access to data.

Extending Instinet’s legacy on-premises environment to the cloud has enabled the electronic broker to achieve record scale—deftly handling exponentially higher market volumes amid the unexpectedly disruptive global landscape of the last 36 months.

Instinet has achieved several critical business objectives through this migration:

• The new pipeline processed Instinet’s client data in near real-time, removing the lag experienced on-premises.
• It has proved to be more robust than the on-premises infrastructure, improving up-time for the brokerage business.
• Instinet has been able to deploy much larger database instances, without expensive capital investments.
• Amazon Aurora allows Instinet to rapidly spin up read replicas for analytics tasks, while removing load off the primary instance.
• Release process for new features and functionality has gone from days to mere hours.
• Operational maintenance window has shrunk from an overnight process to one hour.
• Amazon S3 was used to migrate data to AWS cost efficiently and quickly.

Conclusion

The rearchitecting and replatforming of Instinet’s core data pipeline on AWS has given Instinet the ability to manage infrastructure via automation, reducing deployment timelines and improving change management.

The new architecture helps solve intractable scaling challenges that were materially impacting Instinet’s ability to reliably meet business SLAs on high-volume days and serve new clients. The new architecture uses Amazon Managed Service for Apache Flink and Amazon MSK, moving the organization towards an event-driven architecture on a highly scalable and reliable platform.

Transitioning to Apache Kafka (Amazon MSK) and Apache Flink allowed Instinet to move to event driven architecture with two key features:

• Removed bottlenecks to collecting large amounts of data from disparate sources by placing them on a highly scalable Apache Kafka cluster.
• Chart a path away from running business logic on different execution platforms (stored procedures, stand-alone applications, ETL processes) to a stream computing standard using Apache Flink.

The modern architecture also allows Instinet to scale and handle much larger workloads, and provides multi-tenancy to the customer to isolate and scale workloads. The re-architected platform meets Instinet’s technical requirements like high availability, observability, and removing tightly-coupled rigid pipeline challenges that impacted Instinet’s prior data pipeline.

For more information on how Ness Digital Engineering architected and designed Instinet system. contact Ness at rocs@ness.com.
.

.

Ness Digital Engineering – AWS Partner Spotlight

Ness Digital Engineering is an AWS Premier Tier Services Partner that works with organizations to ensure AWS infrastructure will support current business projects, as well as future strategic initiatives.

Contact Ness Digital Engineering | Partner Overview | AWS Marketplace