Fugro Roames was founded as a business unit within Ergon Energy—a Queensland-based electricity distribution network operator and retailer—to provide asset management services that allow clients to remotely investigate the condition and performance of overhead powerline networks. Ergon Energy established Fugro Roames due to the unique challenge of managing Australia’s most geographically distributed network, with an AU$11.5 billion (US$8.8 billion) asset base distributed over 150,000km of overhead powerlines extending from north of Brisbane to islands near Papua New Guinea. The business case was to ensure clearance of vegetation away from the overhead lines to minimize the risk of service disruption, electrocution, and fires.
Ergon Energy divested the Roames unit to the Netherlands-based Fugro in March 2014 to offer the same benefits to power companies worldwide. Fugro Roames now operates aircraft equipped with cameras and laser sensors to fly over customers’ networks and capture data used to build detailed 3D models of the lines and the environment in which they are located. The business then runs analytics and customer rules against the network assets and sends reports that teams use to prioritize and complete maintenance and preservation activities such as clearing vegetation.
Fugro Roames started operations with servers, storage, networking, and associated systems running in an on-premises data center. However, as the business started building what Josh Passenger, Technical Architect at Fugro Roames, describes as a “super-computer friendly environment using fairly traditional supercomputing tools,” infrastructure restrictions such as periodic air-conditioning shutdowns and inadequate storage input-output operations per second soon became apparent.
“We were at the mercy of the building’s maintenance schedule, so there were times we would have to implement work during after-hours just to keep our systems cool,” says Passenger. “In addition, the more capacity we added to our on-premises storage cluster, the more cooling we needed. We outgrew the capacity of our on-premises data center within a few months.”
Fugro Roames experienced severe challenges in scaling its storage and backup systems to capture the rising influx of data, undertake processing, and retain the outcomes for analysis.
“We particularly struggled during demand peaks when we really needed to double or triple our infrastructure capacity,” says Passenger. “We had downstream processors that required data to be ready first thing Monday morning, but our infrastructure held us back from delivering. The cost to add enough headroom in our physical infrastructure to support those demand peaks periods was quite prohibitive and left us at an impasse.”
Following a year in the on-premises data center, Fugro Roames began looking for alternative infrastructure options for its data capture, analysis, and reporting systems. A detailed review of cloud services led to a recommendation that the company use Amazon Web Services (AWS).
“AWS would give us the flexibility to put different workloads with different operating systems into a single architecture,” says Passenger. Fugro Roames was also impressed by the maturity of the AWS APIs (application programming interfaces), the detailed documentation provided by AWS, and the ability to spin up prototypes in a short time to obtain some early wins—helpful for influencing conservative team members who had reservations about moving to a public cloud environment.
Balancing normal business activities during the transition, Fugro Roames initiated a migration to the cloud in 2012. The business now runs all workloads in an AWS infrastructure in the Amazon Asia-Pacific (Sydney) region.
To protect data from unauthorized access or disruption, raw data from numerous sources is loaded into a server and sent over a 10GB per second AWS Direct Connect link to an AWS architecture running in an Amazon Virtual Private Cloud (Amazon VPC). Both raw and interpreted data is stored in Amazon Simple Storage Service (Amazon S3), while resizable compute power is provided by a combination of on-demand and reserved Amazon Elastic Compute Cloud (Amazon EC2) instances running in Windows or Linux operating systems.
Adept use of Amazon EC2 Spot Instances— unused EC2 capacity available in the AWS cloud offered at up-to 90% off of On-Demand prices— has helped Fugro Roames reduce the cost of what Passenger describes as its “heavy lifting computing” to one-eighth of the cost of using on-demand Amazon EC2 instances. A database running in Amazon DynamoDB supports concurrent access from a cluster of Amazon EC2 Spot Instances, while the interpreted data is sent to Amazon Glacier for archiving. Messages are distributed between various components of the AWS architecture using Amazon Simple Queue Service (Amazon SQS). An Amazon Redshift data warehouse batches and models information for clients. Amazon CloudFront is used as the content delivery network between the VPC and end users, while Elastic Load Balancing is used to balance traffic.
”The move to the AWS architecture lifted a huge weight from my shoulders,” says Passenger. “We were able to free ourselves from the constraints of local infrastructure, while reaping the rewards of unlimited storage.”
Fugro Roames is taking advantage of the expertise provided through AWS Support, Enterprise level, to overcome any technical issues. “AWS Support has proven to be extremely responsive and very knowledgeable,” Passenger says. “We have raised several technical questions and if the support desk can’t help us, they will escalate very quickly to a level where we receive answers that are very detailed and helpful.”
Fugro Roames team members are also extremely keen to build their skills in AWS and have taken architecture courses, as well as online training courses and associated resources. “We receive visits once a month from an AWS technical architect who briefs us on future developments, and answers any queries we have,” Passenger says.
The figure below illustrates Fugro Roames’ environment in AWS:
By using AWS services and tools, Fugro Roames can capture large volumes of granular data, such as the details of 100 million trees close to the Ergon Energy network, and can generate timely, accurate reports for the field teams charged with preserving the lines.
“By using AWS and the Amazon EC2 Spot Instances, Fugro Roames has enabled Ergon Energy to reduce the annual cost of vegetation management from AU$100 million to AU$60 million.” says Passenger. Internally at Fugro Roames, technology capital costs have fallen by 95 percent, and the business no longer requires a team of five full-time employees dedicated to infrastructure management. AWS is also delivering infrastructure availability levels exceeding 99.999 percent.
Fugro Roames’ responsiveness to customer needs have improved considerably. In February 2015, Fugro Roames aircraft were mobilized to fly to northern Queensland following the destruction wrought by Cyclone Marcia—an event ranked in the same category as Hurricane Katrina in the United States.
“Our aircraft captured the data that we then processed, interpreted, and sent to the disaster recovery center,” says Passenger. “Thanks to AWS, we had detailed Google Earth imagery available for Ergon Energy and the emergency services by the following morning. This enabled them to prioritize network repair, including providing generators where they were needed in affected areas and restoring power as quickly as possible. Up to 70,000 people were without power, and Ergon Energy was able to restore it to all within a record-breaking time of 10 days.”
Historic data is also easily accessible. “We can present our customers with data going back three years in a matter of minutes,” says Passenger.
Fugro Roames has also become more agile and dynamic in meeting customer demand for customized services. These can be launched in hours rather than weeks. “Developers can now run their algorithms over large data sets across multiple compute nodes. They have moved from being able to test a scenario three or four times a week to 100 times a week,” Passenger says. “The result is much faster feedback and a considerably quicker turnaround for service delivery. Some of these algorithms that would take a year to run on a desktop can take two hours in the AWS infrastructure.”
The business has now successfully expanded into the United Kingdom and is reviewing opportunities in other markets after establishing a presence on AWS in Ireland.
Passenger notes that Fugro Roames is extremely pleased with AWS. “The rate of innovation and improvement at AWS aligns perfectly with the model that Fugro Roames follows,” says Passenger. “We know the rate of change with AWS is absolutely through the roof. This means that we can take advantage of new ways of performing our analytics and services.”
Learn more about big data and analytics on AWS, and how to use Amazon EC2 Spot Instances to reduce IT costs.