Category: Amazon EMR

Data is growing in all aspects of our world; every vertical and technical domain is being pushed to the limit by growing data—geospatial is no exception. Learn about Esri’s ArcGIS GeoAnalytics Engine on Amazon EMR and how its geospatial capabilities can complement your current analytics workflows. GeoAnalytics seamlessly integrates with Amazon EMR and can be deployed with an Esri-provided bootstrap script at EMR cluster creation.

While the global Hadoop-as-a-service market size is growing at a CAGR of ~39%, it leaves out many small, mid, and large-scale organizational players due to the inherent pains of migration. This post explores how Virtusa complements Amazon EMR Migration by providing approaches and utilities to streamline and manage large-scale Hadoop-to-Amazon EMR migrations by creating a migration framework of automated modules.

Privacera is an AWS Partner that provides security and privacy tools for enterprises to secure and govern user access to databases and datastores in the cloud. PrivaceraCloud reduces the burden of self-managing Apache Ranger by providing Ranger as a hosted service. It provides centralized management of data access, authorization policies, and auditing. Learn how Amazon EMR can integrate with PrivaceraCloud to provide a fully-managed data governance solution.

Global business leaders recognize the value of advanced and augmented big data analytics over various internal and external data sources. However, technical leaders also face challenges capturing insights from data silos without unified master data. Learn how migrating Tamr’s data mastering solutions from on-premises to AWS allowed a customer to process billions of records five times faster with fully managed Amazon EMR clusters.

Data warehouse modernization has been a key aspect of many customers’ broader cloud transformation stories. Legacy data warehouse systems, however, present many challenges when dealing with today’s enterprise data needs. Learn how AWS and Infosys collaborated to transform a legacy Netezza platform on AWS for a large retail customer. With Infosys tools, processes, and industry knowledge, the collaboration between AWS and Infosys enables customers to transform their analytics platforms.

Apache Spark and its parallel processing framework, along with the ease of scaling up in public clouds, have pushed out the limits for data analytics. Learn how Bigstream addresses growing Spark needs, with software that optimizes existing CPU infrastructure and can also seamlessly incorporate advanced programmable hardware. With the same number of servers, Bigstream can accelerate Spark clusters 3x with software alone and 10x when introducing FPGAs.

Developing and maintaining an on-premises data lake is a complex undertaking. To maximize the value of data and use it as the basis for critical decisions, the data platform must be flexible and cost-effective. Learn how to build a hybrid data lake with Alluxio to leverage analytics and AI on AWS alongside a multi-petabyte on-premises data lake. Alluxio’s solution is called “zero-copy” hybrid cloud, indicating a cloud migration approach without first copying data to Amazon S3.

Apache Hudi on Amazon EMR is an ideal solution for large-scale and near real-time applications that require incremental data pipelines and processing. This post provides a step-by-step method to perform a proof of concept for Apache Hudi on Amazon EMR. Learn how a non-customer-facing PoC solution from nClouds set up a new data and analytics platform using Apache Hudi on Amazon EMR and other managed services, including Amazon QuickSight for data visualization.

Fully managed cloud services enable global enterprises to focus on strategic differentiators versus maintaining infrastructure. They do this by creating data lakes and performing big data processing in the cloud. SnapLogic eXtreme allows citizen integrators, those who can’t code, and data integrators to efficiently support and augment data-integration use cases by performing complex transformations on large volumes of data. Learn how to set up SnapLogic eXtreme and use Amazon EMR to do Amazon Redshift ETL.

As organizations continue to build data lakes on AWS and adopt Amazon EMR, especially when consuming data at enterprise scale, it’s critical to govern your data lakes by establishing federated access and having fine-grained controls to access your data. Learn how to implement SAML-based authentication (AuthN) using Okta for Amazon EMR, querying data using Zeppelin notebooks, and applying column-level authorization (AuthZ) using AWS Lake Formation.