FlyData: Pack Your Bags, We’re Moving to the Cloud

FlyData provides real-time continuous big data integration to data warehouses such as Amazon Redshift. FlyData is considered an emerging choice for data warehousing, business intelligence, reporting, and analytic deployments needs. You can read more about FlyData (formerly Hapyrus) here.

FlyData was founded in April 2011 and is headquartered in Mountain View, CA.

Prior to FlyData, founder Koichi Fujikawa was at Yahoo! Japan, Cirius Technologies Inc., and numerous startups.

FlyData founder Koichi Fujikawa

What’s your background, and what led you to start FlyData?

I worked as a software engineer at Yahoo! JAPAN and other Japanese startups for more than 12 years. During that time, I also got funded by the “Mitoh” project, a government sponsored technology development initiative, through which I contributed to open source Hadoop Middleware projects. That’s when I realized the problems surrounding the handling of big data in the cloud, and so I went off to found FlyData to tackle these issues.

Tell us more about FlyData, what does it do?

FlyData helps integrate big data in the cloud, and to the cloud. We are especially focused on providing ways to transfer and integrate data to cloud data storages such as Amazon Redshift. Our customers can maximize the benefits by taking advantage of FlyData, which allows you to transfer and load data continuously, cheaply, and securely.

What are the biggest technical challenges that FlyData solves?

The fresher your data, the more value you can extract out of it. That’s why at FlyData we focus on continuously integrating your data from on-premise/cloud servers to destinations such as Amazon Redshift, making it possible to query that data on a near real-time basis. There are many steps involved in making this possible, such as seamlessly scaling and handling errors appropriately. These are steps that users don’t want to, and shouldn’t have to, worry about. We try to take away these needless pain points as much as possible, so that it feels as if the data already existed in Amazon Redshift in the first place.

To provide such a user experience, we had to solve many technical challenges, but our team continues to overcome these hurdles, and we strive to make big data integration easier.

You’ve built a business on top of an AWS service. What did you see in the Redshift product and market opportunity that gave you the confidence to do that?

As you can see from past benchmarks we have taken, Amazon Redshift is truly a game-changing product, making it possible to process big data that was previously only processable through Hadoop. What’s really is noteworthy is the combination of high-performance with low cost. On top of that, the fact that you can run queries in SQL (unlike Hadoop) is, among other things, helping the rapid adoption of Amazon Redshift at various companies. We also have had the privilege of working directly with the Amazon Redshift team at AWS, and our collaboration has helped us provide a superior service to our customers.

How should startups be using Amazon Redshift?

Amazon Redshift is actually perfect for startups because it offers so much scalability. The newly released SSD version (dw2) has less storage compared to the HDD version (dw1), but at the initial stage of a startup where data is not huge yet, you can take advantage of its superior query and loading performance. And then if your data size increases, you can even resize to an HDD version. That’s the beauty of Amazon Redshift, and this scalability is exactly what allows startups to compete against larger players.

How has AWS helped you innovate faster for your customers?

The biggest thing about AWS is that it is an innovator. In order to be able to offer their services at very low costs, they implement a wide array of innovations behind the scenes. They have pushed into the relatively stale data warehousing market by offering Amazon Redshift in the cloud at a very aggressive price point. They have, in a way, democratized the data warehousing market following suit of Hadoop. Now, even smaller companies can reap the benefits of big data analysis. Further, at a time when Hadoop seemed to become the de facto solution, AWS introduced a solution that is better suited for many of the customers who want to perform data analysis. We have great respect for AWS in this regard.

What are the big business challenges facing FlyData? What have you learned about growing a startup that you could share with other startups?

We’ve build a product valuable to enterprise customers, and have succeeded in attaining some level of product-market fit. In order for us to exponentially grow our business, we need to find good ways of reaching more customers effectively, and providing more value to each of our customers. Although we provide enterprise products, we want to create an open ecosystem for sending big data across multiple destinations, which will be convenient, efficient, and secure.

You went through the 500 startups program. What did you learn from your experience there that you can share with other startups?

Because we came to Silicon Valley from Japan, we did not know what were the norms or common sense for entrepreneurs in the area. 500 Startups allowed us to absorb this just by being there and communicating with various founders who face the same kind of challenges. If you are just launching your startup and have the chance of joining 500 Startups or YC, then you definitely should. After graduating you will realize how many things you previously did not know. You will also acquire good habits around running startups.

If you were mentoring a new startup on making the decision to build on AWS, what would you tell them?

From day one, develop with scalability in mind. By using AWS, you can do this easily. For example, you can start with a simple setup of one ELB with two EC2 instances, S3, and RDS, and you have a scalable setup. Startups are all about growth and being able to scale, so your back-end needs to be designed to be able to do so as well.

What are you building now? What’s next for FlyData?

Right now, we are building FlyData Sync, which allows you to continuously sync your data from relational databases (RDBs) such as MySQL and Microsoft SQL Server to Amazon Redshift. In addition to the big data analysis on log files, which we traditionally supported, with FlyData Sync, you will be able to add the data assets you currently store in RDBs into your data analysis. And you will be able to do all of this in near real-time, in Amazon Redshift. We think this will be valuable to many users, especially in the coming years.

We are also developing API/SDK support. This will make it easier to integrate data across different platforms, especially to Amazon Redshift. We are aiming to become a “hub” for data in the cloud, and will continue to strive to serve our customers better for their cloud data integration needs.