BARC India Delivers Insights that Powers Content Decisions 80% Faster with Data Pipeline on AWS

In 2020, the average time spent daily in front of the TV in India stood at 4 hours, 2 minutes. This reaffirmed the strength of TV as a medium in India among both viewers and advertisers, proving that it is indeed the screen of choice for the household. In the same year, TV ad volumes grew by 34% in the second half of 2020 compared to the first half. For many companies that rely on TV advertising to attract customers, understanding viewer profiles is key to a successful ad campaign.

 

The Broadcast Audience Research Council India (BARC) was formed in 2010 to measure viewing data and generate insights relating to TV audiences. BARC is a joint industry body founded by stakeholders representing broadcasters and advertisers alongside advertising and media agencies. Built upon a robust and future-ready technology backbone, BARC India owns and manages a transparent, accurate, and inclusive TV audience measurement system.

 

BARC owns and operates a proprietary data center, which it was using for data processing and analytics until 2019. However, as its client base and data ingestion grew, the business required a cloud-based analytics solution that could scale on demand. BARC was also planning to launch a new workflow that would drastically increase the volume of data for analysis.

 

To collect data each day, BARC installed IoT devices in select homes in 456 districts across India. When the number of devices reached 44,000, the business decided to migrate core data processing to the Amazon Web Services (AWS) Cloud. BARC selected AWS for its reliability, scalability, and processing power. Vaibhav Bhargava, vice president at BARC, says, “Time and scale are the two KPIs for our analytics projects, and the native AWS data processing services together with the core AWS platform provided an all-in-one solution. Plus, it was quick to setup.”

 

BARC had migrated its data processing workloads within 2.5 months of commencing a proof of concept on AWS. Engineers had multiple in-depth conversations and on-the-fly training sessions with AWS solutions architects to review and optimize the code used in their data pipeline. “The experience with AWS has always been like that of a teacher and student. Our developers learned fast and were able to adopt new services quickly. Every time we came back to AWS with a question or follow-up point, we discovered something new,” says Bhargava.

 

BARC then built a data warehouse on AWS to support the new analytics workflow for landing pages. The landing page is the first channel viewers see when they switch on their cable boxes. BARC has designed algorithms on AWS to detect and analyze the landing page from multiple cable box vendors. This ensures the landing page channel isn’t factored into the rating systems alongside standard channels users elect to watch.

 

BARC is now using Amazon EMR as its big data platform, which ingests IoT device data from Amazon Simple Storage Service (Amazon S3). “Amazon EMR gives us the agility and dynamic availability of big data infrastructure that our business demands, especially when we’re deploying a new algorithm or code,” Bhargava says. BARC also plans to use AWS Lambda for serverless code deployment and Amazon Relational Database Service (Amazon RDS) for database administration in the near future.

Processing speed, one of the key criteria for migration success, has greatly accelerated on AWS. In the data center, it was taking BARC 7 hours to process seven days of data. On AWS, BARC can process 14 days of data in just 1.5 hours. These time savings, amounting to an 80 percent reduction, are critical when deploying landing page algorithms, which consider data from 14 weeks’ prior and thus consume data on a huge scale. BARC’s clients now receive more detailed weekly reports with insights dating further back than seven days, which helps them make more informed advertising and programming decisions.,

 

Since migration, BARC has increased the number of IoT devices supplying data to Amazon S3 to 55,000 from 44,000. The company anticipates it will continue increasing the number of devices by about 10,000 each year in line with business growth. “Processing time has not increased by much, even with the higher volume of data under analysis,” Bhargava says.

 

Mahendra Upadhyay, chief information/technology officer at BARC, adds, “Working with AWS and the services they provide allows us to perform data processing and deliver client reports at a much faster pace, with greater velocity than we could’ve ever achieved before."

BARC also appreciates the flexibility gained in the AWS Cloud. Most data processing jobs are completed in a few hours, after which BARC can shut down Amazon EMR instances to control costs. “We believe AWS can yield good business value over the long term. There are also many services for which you can bring your own licenses and use them on AWS, adding flexibility to our future plans,” Bhargava says.

 

Next, BARC plans to augment its existing pipeline to build a real-time analytics platform that consolidates more data from disparate sources. The business is considering integrating artificial intelligence and machine learning into its platform and plans to increase the amount and frequency of reports generated for clients. Bhargava concludes, “We’re able to consistently meet and even exceed the expectations of our growing client base thanks to AWS.”