Nintendo Turns to Amazon Aurora to Support The Mario Kart Tour

Nintendo is well known for its gaming console, but the global game maker is also committed to delivering its services via mobile devices, including on smartphones. The company, together with DeNA, began developing mobile versions of its games in 2015 and, today, games available on mobile devices include 'Super Mario Run, 'Fire Emblem Heroes', and 'Animal Crossing Pocket Camp'. Taken together, Nintendo mobile games have reached over 650 million downloads world-wide in September 2020. To launch the "Mario Kart Tour" in 2019, Nintendo and DeNA adopted Amazon Aurora as their database, instead of using a self-managed MySQL (MySQL running on Amazon EC2). This ensured the performance and scalability necessary for users to play the game comfortably, and at the same time, greatly reduced the person-power required for operation.

With the mission of "bringing smiles to the faces of everyone," Nintendo continues to take on the challenge of providing new entertainment experiences. Currently, Nintendo's basic strategy is to "expand the number of people who come into contact with Nintendo's intellectual property (IP)" and to continue to plan and develop original products and services. The development of games for smart devices is also an effort to expand the company's business by increasing the number of opportunities to come into contact with Nintendo's IP outside of dedicated game consoles, and by increasing the number of touch points with customers. In line with this objective, Nintendo developed titles such as "Super Mario Run," "Fire Emblem Heroes," and "Animal Crossing Pocket Camp" in collaboration with DeNA, starting with "Miitomo," which was released in the spring of 2016.

“Mario Kart Tour is our fifth joint development with DeNA, and we began development with the idea of delivering the best experience to users around the world," says Kotaro Fukawa, Deputy Director of the Smart Device Division and Group Manager of the Business System Development Group at Nintendo.

The challenges they faced with its database (DB) were scalability to allow comfortable play even with a large number of users, performance without compromising the user experience, and optimization of server and operation costs.

Spikes in game downloads strained Nintendo’s MySQL deployment and its engineers scrambled to add capacity to meet demand. It led to a poor user experience for players and the overprovisioning of resources, leading to wasted money, time, and effort.

"Until then, we had undertaken operations by building MySQL on Amazon EC2, but that added a high workload to our engineers as they needed to manually respond to scaling-out and scaling-in," said Fukawa.

To reduce operational overhead and to improve the scale and performance of its services, Nintendo migrated from its self-managed MySQL database on EC2 to Amazon Aurora with MySQL Compatibility. As a fully managed service under the Amazon Relational Database Service (RDS), Aurora automates many administrative tasks, like patching and backups, and makes it simple to allocate database capacity with just a few clicks.

"Even for highly skilled engineers, operating MySQL on Amazon EC2 produced a heavy workload and resulted in an environment in which there was a great amount of pressure to always be focused and it took a high mental toll on our engineers,” said Shunichi Kaneko, Division Director from the IT Management Division, System Management Unit at DeNA. “With future expansion in mind, we chose Amazon Aurora in view of both lightening the operational load and compatibility with MySQL. With Amazon Aurora, we were able to perform instance operations with a single command, and we could automatically utilize fail-over and respond to malfunctions.”

Because Aurora is compatible with MySQL, Nintendo was able to migrate to Aurora using standard MySQL import/export tools and snapshots. It also meant the code, applications, drivers, and tools Nintendo engineers were already using also worked with Amazon Aurora with little or no change. And the team at AWS was there every step of the way.

The development of the DB area started in the summer of 2017, and the teams spent a year verifying, operationally testing, and load testing. The key points of the development were scalability, performance, and reduction of operation person-power. For DB scalability, the teams took separate measures for “Write” and “Read”, and for Write, they prepared a cluster that could accept three times the number of users as predicted.  

The development was completed in September 2019, culminating in the launch of 600 Aurora clusters with 1,200 instances. As the day kicked off and global traffic intensified, the number of queries recorded across all of Amazon Aurora hit 300,000 per second at its peak.

"We were able to ensure capacity, which we were worried about, and Amazon Aurora remained stable and continued to respond even when the number of requests exploded,” says Fukawa.

As part of the development, the team also optimized its write-heavy applications running on Aurora for performance and undertook fine tuning to reduce the I/O load at the same time. Nintendo was also happy with Aurora’s continuous data backup to Amazon S3, as Aurora had been processing approximately 30 TB of data per month.

"Changes to the way the player’s race records were saved was the most effective thing,” said Kei Tsuchiya, Deputy Manager, DeNA’s IT Platform Department. “It produced a large amount of data and the update load was high, so we reduced the I/O load by moving data storage to Amazon S3, and offloaded it from Amazon Aurora."

Nintendo achieved faster operations with an improved fail-over mechanism thanks to Aurora. It also achieved downtime of less than 10 seconds, and ensured a level of quality that its millions of players have come to expect from the company and its games.

"The Amazon Aurora Service Team, the AWS account manager, and solution architect shared the context, and provided us with a service that had a high level of additional value,” said Fukawa. “The project went smoothly thanks to proactive discussions.”

Meanwhile, the downloads of games have steadily increased and the development team at Nintendo continues to add new functionality to games, such as 'multi-play' function for 'Mario Kart Tour' that enables up to eight players to race each other.

Aurora’s cloud-native architecture and automation of administrative tasks together support Nintendo’s modern software development practices, focused on continuous innovation. And Aurora also easily scales capacity to meet growing demand, so the gaming performance doesn’t take a hit.

"By freeing up our engineers from operational work, we can increase the amount of time they can use to be creative and dream up games that are even more fun,” said Kaneko. "And even now, around a year after the initial release, the game servers continue to run stably. There have been no issues, despite sudden spikes in download numbers and requests due to newly added features and the like.” said Tsuchiya .

Of course, Amazon Aurora doesn’t run in a bubble. Nintendo is utilizing other AWS services in conjunction with Aurora, resulting in improved overall efficiency.

"By combining Amazon Aurora, Amazon CloudWatch, and Amazon Athena, we have increasingly automated monitoring and log analysis, which has reduced loads,” said Tsuchiya. “Implementing managed services for certain components has released us from server load balancing and managing storage capacity, allowing us to go about our operations with confidence.”

With AWS and Amazon Aurora, Nintendo is well positioned for a future of high growth and happy customers.

“As a result of running on AWS, Nintendo, and DeNA working together as a team on this project, we were able to give customers around the world the kind of high quality gaming experience we had hoped for,” said Fukawa. “We hope to further improve the operational efficiency of our three companies in the future.”

To learn more, visit thinkwithwp.com/rds/aurora/.