Accelerate your Data Unification and Customer-360 Journey with Cognizant Data Unification Framework on AWS

By Anindya Maiti, Data Management Global Community Lead – Cognizant
By Manjit Guha, Intelligent Data Management Lead – Cognizant
By Yadu Kishore Tatavarthi, Sr Solutions Architect, WW Data & AI – Amazon Web Services

Cognizant

Data unification integrates and consolidates diverse datasets from various sources into a coherent structure. This process involves standardizing data attributes, resolving inconsistencies, and reconciling discrepancies to create a unified view. By eliminating data silos and reducing redundancy, organizations can ensure a comprehensive and consistent dataset for analysis, reporting, and decision-making. This seamless flow of information promotes efficient operations and enables deeper insights, unlocking the true value of the organization’s data assets.

The data unification journey starts with assessing the existing data landscape, identifying all data sources, formats, and repositories. Understanding the quality, consistency, and relevance of the data is crucial. The next step involves devising a strategy for data ingestion, cleansing, standardization, validation, mastering, and publishing. Establishing protocols for ongoing maintenance ensures the unified data remains accurate and relevant as the organization evolves. Ultimately, the goal is to achieve a high-quality data ecosystem that empowers the organization with actionable insights and a competitive edge.

Cognizant is an AWS Premier Tier Services Partner with several AWS Competencies. With its industry-based approach, Cognizant helps clients envision, build, and run more innovative and efficient businesses.

In this post, we describe how Cognizant Data Unification Framework helps organizations start their data unification journey with a set of customizable services adapting to an organization’s unique structure, processes, and objectives. Cognizant Data Unification Framework supports organizations in each of the above stages in the data unification journey by providing a set of prebuilt framework components that help accelerate these processes.

Solution overview

The Cognizant Data Unification Framework is a custom-built, domain-agnostic solution designed to ingest data from multiple sources and create a unified view. Built using open-source frameworks, it suits both small enterprises and large organizations. The framework personalizes customer engagements by leveraging insights from various touchpoints and offers a reusable structure with metadata-driven configurations, data processing, and a stewardship UI.

It offers a starter kit for custom Master Data Management (MDM) implementations, enabling early business value from the Minimum Viable Product (MVP). Additionally, it lays the groundwork for an enterprise Customer Data Platform (CDP) platform with out-of-the-box features like data unification and segmentation.

Cognizant Data Unification Framework internally uses many AWS services, such as Amazon Simple Storage Service (Amazon S3), Amazon Relational Database Service (Amazon RDS) and Amazon Redshift for storage and metadata management; AWS Fargate and AWS Lambda for processing; Amazon Kinesis for event streaming; AWS Step Functions for workflow orchestrations; Amazon CloudWatch, AWS Key Management Service (AWS KMS), and AWS IAM Identity Center (successor to AWS Single Sign-On) for monitoring and security; and AWS CloudFormation for infrastructure management. The following diagram illustrates this architecture.

Implementation

Cognizant Data Unification Framework provides a set of pre-built cloud-native components that help create the basic structure of a data unification platform.

Data Ingestion Service. The data ingestion service supports both batch and real-time ingestion using a common format, utilized for initial and subsequent data loads into MDM. The Cognizant Data Unification Framework uses AWS Glue-based pipelines for batch and stream data from S3 and Kinesis respectively, while a Java-based microservice on AWS Fargate handles real-time data ingestion.

Data Processing Service. The data processing services provide the core engine of Data Unification which includes typical Data Management processes like data quality checks, match, merge, history maintenance etc. The below microservices are involved in data processing:

Match and Merge Service. The match service, a microservice on AWS Fargate, groups records based on match rules defined at the Entity Level, deriving a match score for each group. The merge service, also on AWS Fargate, handles trust, survivorship, and merging based on these match scores and merge types.

• Data Storage. The storage layer maintains source data versions, golden party profiles, history, and metadata. Amazon S3 stores initial and incremental raw data, as well as cleansed and processed datasets. Amazon Redshift serves as the Curated Layer, storing the latest active master and transactional data. Amazon OpenSearch provides a search layer on top of Amazon Redshift for API and UI consumption.
• Data Stewardship User Interface. This user interface, built using frontend UI framework Angular IO and deployed on AWS Amplify, is designed for data stewards. It is primarily used for data remediation tasks such as suspect case resolution, forced merges, and unmerges as needed.

Publish Service. The publish service is a microservice deployed on AWS Fargate used to publish full or delta data extract for any downstream consumption. It is used if any system wants to consume golden party data and cross reference source information.

Security, Auditing and Notification Management

Cognizant Data Unification Framework is meticulously designed with security at its core, adhering to the principle of least privilege through AWS Identity and Access Management (IAM) for users, roles, groups, and policies. This framework employs a multilayered security strategy utilizing AWS services such as AWS KMS for data encryption at rest. Comprehensive monitoring is achieved through Amazon CloudWatch, while AWS CloudTrail ensures all user activity and API usage are meticulously traced. AWS Config provides resource inventory and configuration history, enhancing governance and compliance. Timely notifications are facilitated by Amazon Simple Notification Service (SNS), ensuring proactive management and response. AWS WAF (Web Application Firewall) plays a crucial role in the Cognizant Data Unification Framework by providing robust security measures to protect web applications and APIs. Amazon S3, Amazon RDS and Amazon Redshift uses server-side encryption (SSE) to encrypt data at REST and supports encryption in transit using SSL/ TLS. By leveraging these AWS tools, the Cognizant Data Unification Framework ensures a robust, secure, and compliant infrastructure that is well-monitored and capable of responding to events in real-time, safeguarding applications and data while maintaining industry standards compliance.

Benefits

• Flexibility and Scalability: Customization allows for adapting Cognizant Data Unification Framework as the organization grows and changes. Microservices-based architecture for Data Acquisition, Transformation, Matching, Survivorship, Data Delivery & Platform services.
• Alignment with Organizational Goals: Template driven code aligns directly with an organization’s strategic aims and industry requirements.
• Improved Customer Experience and User Adoption: The framework uses a user-friendly and intuitive user interface, leading to better user engagement and adoption.
• Enhanced Data Quality and Accuracy: The framework supplies a customizable Data Quality Framework which enhances Data Quality and Accuracy. Cognizant Data Unification Framework is also being extended to use AWS Glue Data Quality as well as Amazon DataZone which would further enhance the capabilities of the framework.
• Jump-start to business use cases like customer segmentation & marketing campaigns: By harnessing the power of their data, organizations can improve decision-making, enhance customer experiences, and optimize operations.

Case Study: Transforming Agriculture Through Data

In today’s digital agriculture landscape, understanding and serving farmers effectively requires a comprehensive view of their needs, behaviors, and interactions across multiple touchpoints. When a leading global crop science company approached us with this challenge, we knew we had an opportunity to revolutionize farmer engagement through our Data Unification Framework.

Key Business Drivers

The customer is on a journey to create a real-time, 360-degree view of farmers across all channels. This provided a consistent, personalized, and relevant experience for farmers, with the ability to add new features and partnerships.

Solution Highlights

The solution provided a comprehensive data ecosystem for smallholder farmers. It involved developing a standardized profile for each farmer, centralizing their interactions, transactions, and social media data into an activity hub, and integrating data from multiple sources such as call centers, advisors, and manual uploads. This data is then analyzed and visualized to provide a 360-degree view of each farmer, enabling personalized support and decision-making.

Business Outcome

• Personalized the interaction with farmers by combining multiple point interaction with the brand to harness the upcoming opportunities.
• Met the personalization needs of farmers in very large numbers across multiple countries.
• Scaled up to the challenges to meet the regulatory needs of countries.
• Seamlessly operates with global and regional capabilities.

Conclusion

In the post, we discussed how the Cognizant Data Unification Framework helps organizations accelerate their data unification journey with a customizable framework that delivers faster time-to-value. Discover how this solution enables better customer insights and improved business outcomes through seamless integration with AWS services. organizations to embark on their data unification journey by applying a MVP based approach, thus delivering early business value. In an era where data is the lifeblood of modern enterprises, effective management and utilization of this invaluable asset stand as paramount imperatives for organizational success.

This blog post highlights the importance of customized solutions in Data Management and Data Unification. It emphasizes that a one-size-fits-all approach is no longer adequate. While standard Data Management platforms provide a foundation, tailoring strategies to the unique characteristics of an organization significantly enhances their potential. Customization offers key benefits such as flexibility, scalability, data accuracy, and strategic alignment. Furthermore, the Cognizant Data Unification Framework stands out as a versatile solution that clients can utilize to meet their specific needs, thereby improving the adaptability and effectiveness of their data management practices.

To learn more, please reach out to Cognizant.
.

.

Cognizant – AWS Partner Spotlight

Cognizant is an AWS Premier Tier Service Partner and MSP that transforms a retailer’s business, operating, and technology models for the digital era by helping organizations envision, build, and run more innovative and efficient businesses.

Contact Cognizant | Partner Overview | AWS Marketplace | Case Studies