How AWS and Redis Labs partner to make a better Redis

One of the best things about open source is its community. It’s the community that helps to popularize a project. It’s the community that helps to build the project through code, documentation, and other contributions. It’s the community that sometimes turns into paying customers, too.

But it’s also the community that may compete with you. Consider Redis, for example. Redis is an incredibly popular open source in-memory data store, generally used as a database, cache, or message broker. It’s so popular, in fact, that a host of companies offer Redis as a managed database service, including AWS (Amazon ElastiCache) and Redis Labs (Redis Enterprise Cloud). Redis Labs is the primary sponsor of Redis development and employed, up until his retirement from the Redis community in 2020, Redis founder and maintainer Salvatore Sanfilippo.

Although AWS and Redis Labs continue to compete for Redis workloads, we also partner closely to serve our mutual customers. This partnership involves a range of activities, including collaborating on the Redis code. In this post we will outline how our partnership has matured, and we will highlight some of the customers we serve together.

Making a better Redis

When Sanfilippo announced his departure from Redis, Redis Labs had a choice. The company could lock down and control all Redis development, or they could open up. There was plenty of precedent for keeping the community somewhat confined to a small group of contributors, as Sanfilippo managed things that way, as he told me, because he sometimes struggled to delegate.

But Redis Labs decided to move in the opposite direction. As announced in 2020, “[W]e wish to use this opportunity and create a more open, scalable, and community-driven ‘core team’ structure to run the project.” Although the company named three employees with a long history of contributions to Redis as maintainers, it also did what might have seemed unthinkable from a competitive perspective: It named two engineers from cloud providers that offered competitive Redis services.

One of those engineers was Madelyn Olson, part of the ElastiCache team. Madelyn contributed to Redis for a few years, and had become a valued member of the Redis community. Yet she worked for a direct competitor. Her inclusion reflected real sincerity in Redis Labs’ stated goal to open up the Redis community.

This move also reflects a reality of how open source works, when done well. Madelyn, in addition to other AWS employees, plus many other contributors from a growing array of companies, combine to make Redis the database that, for the fourth consecutive year, developers have crowned as their most-loved database. Beyond collaborating on code, AWS and Redis Labs also work closely together to ensure that customers have the best possible experience running Redis Enterprise on AWS.

Customers in common

Of the 16 Amazon leadership principles, customer obsession ranks highest, and partnering with Redis Labs helps both companies provide developers services to meet customers’ unique needs. With most Redis Labs customers choosing to run Redis Enterprise Cloud workloads on AWS, collaborating to ensure that these customers get the best possible performance and scale is imperative. Among other benefits, by running Redis Enterprise on AWS, customers get:

• High availability: Uninterrupted high availability (99.99% uptime as a baseline and 99.999% uptime in Active-Active deployments), transparent to users, with diskless replication, instant failure detection, and single-digit-seconds failover across racks, Availability Zones, and geographies.
• Active-Active: This feature enables simultaneous read and write operations on the same dataset across multiple geographic locations. To achieve this feature, Redis Labs engineers employed conflict-free replicated data types (CRDTs). This helps by automatically resolving conflicting writes, without changing the way a developer’s application uses Redis.
• AWS Outposts: Running Redis Enterprise on AWS Outposts is great for customers who may have data residency requirements in a given specific geography for regulatory compliance requirements, or for information security reasons. AWS Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any data center, colocation space, or on-premises facility for a consistent hybrid experience. This means customers get Redis Enterprise on fully managed infrastructure with native AWS APIs, wherever you need to run it.

For example, Ekata, which Mastercard acquired in April 2021, delivers identity verification solutions that leverage machine learning (ML) technology for use cases such as payment authentication and financial transaction fraud review. Ekata traditionally had done business mostly with customers in the US and Canada, but as their business expanded globally, they added more than 1,700 customers across 238 countries.

Over time, Ekata evaluated a range of disparate relational and non-relational databases, but ultimately settled on Redis as the primary database for its Identity Graph application. This application housed more than 5 billion entities, with more than 8 billion relationship links between each entity, resulting in 10 TB of graph data stored in Redis. To serve these customers, Ekata needed a system that could scale linearly with consistent latency and stability of at least 99.99% uptime so that when one of more than 2 billion API queries came in, the application was ready to respond. This also demanded their systems to be highly available across multiple Availability Zones with failover and replica capabilities.

Ekata knew they needed AWS infrastructure to scale, with experience depending on 81 Availability Zones across 25 geographic regions. They also liked Redis Enterprise Cloud, wanting a fully managed Redis service to remove operational headaches they’d experienced with self-hosting. Running Redis Enterprise Cloud on AWS helped them hit their performance and availability goals, achieving 0.1-millisecond database latency during peak traffic that span three to four levels of depth on their graph identities.

Customers like Ekata benefit from this cloud collaboration between AWS and open source partners like Redis Labs. At AWS, we want to provide the best place to run open source workloads like Redis, giving customers choice in how they choose to run it. For customers that choose to run Redis Enterprise Cloud, we work closely with Redis Labs to ensure it runs best on AWS.

Interested in trying Redis Enterprise Cloud on AWS? Click here to get started today.