The Instaclustr sign of open source success

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. Visit the website to learn more.

In a 2001 interview, Brian Behlendorf, then president of the Apache Software Foundation, was asked to identify the most foundational right in open source—the thing that, if removed, makes open source not open source. His response? “The right to fork.” Of course Behlendorf wasn’t riffing on flatware, but rather “the right for a user or somebody outside the developer pool or even a subset of the main development pool to be able to take the code and start a new project.”

The ability to fork is what makes open source truly open.

Given our commitment to open source, it is fantastic to see Instaclustr, a company that offers a variety of open source software as cloud services, offer a managed Elasticsearch service that is based on Open Distro for Elasticsearch. When AWS launched Open Distro for Elasticsearch a year ago, we said a key reason was “to keep open source truly open and enable anyone to benefit from our contributions.” Well, that “anyone” includes those who, like Instaclustr, compete head-to-head with the Amazon Elasticsearch Service. Why? Because that’s what makes truly open source communities thrive.

To discuss this in more detail, I spoke with Ben Bromhead, Instaclustr’s co-founder and CTO.

Open source “IRL”

Founded in 2013, Instaclustr offers open source technologies such as Apache Cassandra, Apache Spark, Apache Kafka, and Elasticsearch as cloud services. With more than 60 million node hours under management, enterprises large and small alike trust Instaclustr to deliver “reliability at scale.” Central to the company’s customer commitment is open source because of its “cost, flexibility, transparency, security, and technology freedom.”

Given Instaclustr’s commitment to open source, I asked Bromhead about Instaclustr’s decision to embrace and collaborate on Open Distro for Elasticsearch. “It mainly came down to a few reasons,” Bromhead said, including:

1. Licensing. With a growing community of plugins being licensed as Apache 2.0, Instaclustr could go forward with confidence that they were treating IP correctly.
2. Ownership. Open Distro was also started by two companies (Netflix and Amazon) that had diverse reasons to bring forward a new distro that was aligned with Instaclustr goals.
3. Community. With a view toward building a community with an explicit goal of pure open licensing, the Open Distro is well positioned for future growth.

One of the best parts about open source is community. Community is the essential ingredient that makes Kubernetes and PostgreSQL so interesting, for example, and it’s been a decisive factor in the success (or failure) of a number of open source projects over the years—for example, why Linux is big and OpenSolaris never was, despite excellent technology. Bromhead says that currently Instaclustr is focused internally on improving and building out its support and managed service around Open Distro for Elasticsearch. “As we mature our offering, we will look to contribute code back to the community as we have done with all our other offerings,” he says.

Bromhead says that Instaclustr generally increases its involvement in open source communities as the company matures its offering and gains the experience to make meaningful contributions. “The best example of this is our work in the Apache Cassandra community where we have made many code contributions and open sourced many tools and capabilities to make running Cassandra much easier,” he says. Some of Instaclustr’s better-known contributions to Cassandra include the Kubernetes operator for Cassandra, LDAP Authenticator, and tools for working with SSTables, among others.

This isn’t to say the company is sitting on the sidelines of Open Distro for Elasticsearch. Even now, Bromhead tells me, Instaclustr contributes back to the community by running a number of (currently virtual) events, writing blog posts, and assisting customers with implementations.

Open source benefits

Similar to AWS, Instaclustr contributes to open source projects, such as Open Distro for Elasticsearch, for tangible, customer-centric reasons. Bromhead says:

All our contributions start from a very practical purpose. We fix a bug we directly see in production, or are implementing a feature that is a direct requirement from a customer. We generally don’t speculatively build features in terms of making an open source technology more competitive: that comes from working with our customers. Combined with the shared ownership and validation we get from our customers and the broader community it means whatever code we write is generally useful for a broader group of people from day one.

Such customer focus also plays out in how Instaclustr sees the value of the code itself. Echoing the sentiment expressed by AWS vice president and long-time open source developer Tim Bray (“In AWS engineering, we develop stuff and we operate stuff. I think the second is more important.”), Bromhead says, “We also don’t generally view code as a competitive advantage, so the more we can build capability that doesn’t exist or is locked behind proprietary licensing, it makes it easier to compete on our own core competencies—building, running, and supporting highly scalable, highly available stateful systems.”

One key reason for launching Open Distro for Elasticsearch, among others, was to ensure the availability of security features critical to running Elasticsearch safely, an area in which AWS continues to invest. By working together on Open Distro for Elasticsearch, AWS, Instaclustr, and others can continuously improve Elasticsearch security for customers.

In turn, Bromhead says, the focus on 100% open source contributions gives customers the confidence to adopt technology with minimal commercial risk, as they can continue running their production code without needing to worry about licensing and commercial contracts. He says the next goal for Instaclustr is to take care of the operational and implementation risk, which enables customers to focus on getting on with their core business rather than needing to architect their applications around licensing costs.

Competing in the open

As in the AWS world, in Instaclustr’s world the customer comes first. That customer increasingly runs an ever-growing percentage of their business in the cloud, running open source software, including Open Distro for Elasticsearch. It’s excellent that AWS, Instaclustr, and others can collaborate on this and other open source software, while also competing to best solve customer needs. Collaborative competition is what open source is all about.

Open source is also about being able to run that software wherever the customer needs it. When I asked Bromhead whether having AWS as the primary sponsor for Open Distro for Elasticsearch development crippled, in any way, Instaclustr’s ability to run that software on other clouds, the answer was a clear “no.”

“If anything,” he added, “It gives companies looking to adopt Open Distro for Elasticsearch confidence that it truly is portable and open. Plus if there was anything that happened from a code or licensing perspective that jeopardized [our ability to run the software on other clouds], we could simply fork and continue on!”

Indeed you could, Instaclustr. Because it’s open source.

Feature image via Pixabay.