How AWS Helps Skylight Build Weather Reports for Aviation Companies

Who likes to talk about the weather? Pilots. In fact they need to do a lot more than talk about the weather. They need to know the data from the inside out. But of course predicting weather patterns remains not only one of the oldest challenges for humanity but one of the most complicated challenges for computers of the future. And so companies like SkySight have grown into the niche of building custom, detailed—far more exact than anything we’d find on the Internet—reports on the weather for aviation companies.

Cloud computing’s been a huge shift for the industry. “Previously nations would spend hundreds of millions of dollars on a supercomputer that may only spend a few hours a day forecasting and then sit largely underutilized the rest of the time,” says SkySight founder Matthew Scutter. “We are able to spin up competitive clusters and turn them off when we don’t need them, offering forecast options previously only available to national scale projects.”

It wasn’t always this way. SkySight built its initial infrastructure on local servers they had to manage and maintain. They powered the company’s website, the database and both front and back ends of the application. In a sense they were in the same bind as those aforementioned nations. SkySight works on a project by project basis. So there’s plenty of time where the company doesn’t need all that computing power and so they’d forecast server usage—an idea that sounds increasingly strange in today’s world of ubiquitous cloud computing.

“Serving forecasts off physical instances was also fraught,” says Scutter. “If a machine died we lost all our current data.”

There’s also the question of calculation speed. Every analytics company wants to drive down the cost of compute. But for young companies, the difference in those costs can mean whether a deal’s profitable or not. Scutter says the compute power required to increase the quality of their reports goes up exponentially. That makes competing with established competitors especially difficult, “If you can’t get your costs much lower,” he says.

So SkySight decided to jump to AWS. They evaluated the competition and realized AWS’s EC2 exceeded the capabilities the company needed for scalability, while reducing their cost per compute substantially. The entire migration took less than two weeks. Scutter says the APIs were simple enough that the replacements were essentially (re)plug and play; and the customer support and documentation made migrating their container infrastructure seamless.

Now?

“We’re able to spin up and spin down enormous quantities of resources at zero notice,” Scutter says and that the company’s saving “a fortune.” Now when a client asks for a special report, or needs something yesterday, as it were, they’re not scrambling. “We’re able to scale up and down our forecasts for custom projects in minutes rather than the days or weeks.” And when it comes to securing database integrity, he adds, “AWS S3 is a no brainer. Even if all our infrastructure goes offline, the forecasts and website stay up.”

The power and agility of EC2 lets SkySight deliver forecasts using far more recent data. “If it takes 24 hours to process the days weather, it’s no use by the time it comes out,” Scutter explains. “We’re also really enjoying the flexibility of S3 and Cloudfront in terms of controlling how and where we serve and cache our forecasts.” The change has helped SkySight secure new contracts and last-minute jobs where they may not have been able to deliver previously. “The reduction in operations load is also very significant,” Scutter says of moving over to AWS. “I think we are spending 90% less time managing physical machines.

The next step will be seeing how SkySight can utilize AWS Lambda; Scutter says they’re examining how to move the web front end over. SkySight does a lot of heavy on-demand calculations that need sub-second response times. But then the infrastructure sits idle for the next twenty-three hours. “The fast spin up and no idle charge of Lambda could suit us well,” he says. And “We can run Lambda at the edge: close to customers. Whereas with other providers they’re all centralized and it’s very hard to run across multiple regions.”

Global coverage is critical for any web startup and SkySight, as one might expect from a company developing the backbone of aviation informatics, is no exception. “We have customers on every continent,” Scutter says.