AWS Open Source Blog

Securing PyPI for the Future

We are excited to announce that Amazon Web Services is now the Python Package Index (PyPI) Security Sponsor at the Python Software Foundation, the non-profit devoted to advancing open source technology related to the Python programming language. Through this sponsorship, AWS is providing funding to the PSF to hire a full-time Safety and Security Engineer dedicated to improving the security posture of PyPI. This effort is part of our broader initiative at Amazon Web Services (AWS) to support open source software supply chain security.

Python snake defending its eggs with armor and a shieldPython is an extremely popular open source programming and scripting language among our customers, partners, and Amazon engineers. It is number one on both the TIOBE Index (April 2023) and the PopularitY of Programming Language (PYPL) Index. PyPI is the primary repository of software for the Python programming language. Since Python is modular in nature, most Python applications rely heavily on PyPI to provide the necessary dependencies for core functions rather than reinventing them each time. PyPI is also the primary distribution point for Python applications and libraries.

At AWS, we know that scale and success bring broad responsibility. Amazon and its customers build solutions with Python and we recognize the need to give back to the open source communities that we depend on and help ensure their long term sustainability. AWS is a maintaining sponsor of the PSF and has supported PyPI since 2018, when the index was rewritten to run on AWS in order to address performance and scalability concerns. Today, PyPI scales beautifully due to the significant work from PSF Director of Infrastructure Ee Durbin and the PyPI infrastructure team. AWS is pleased to be able to continue to support PyPI via AWS credits, which offset their infrastructure costs.

PyPI is now facing a new challenge at scale: keeping Python software packages secure. PyPI is regularly threatened by malicious actors, with attacks including typosquatting, dependency injection, and dependency confusion. Companies (including AWS) publish business-critical software on PyPI, and packages are being maliciously published to appear to be from users who represent a large target. These attacks on PyPI have lead to a lengthy support ticket backlog, which are currently addressed by a single part-time volunteer. Their efforts to date to stay on top of this have been nothing short of incredible, but they can be more sustainable.

As the first PyPI Security Sponsor, we are providing additional funding which will allow the PSF to hire a full-time Safety and Security Engineer for PyPI. This will provide PyPI with additional resources to take down malware from the site and respond more quickly to support tickets related to security issues. Additionally, it will allow PyPI to shift from a reactive approach to security to a proactive one in which they can develop a security plan with improvement milestones and enable proper security audits of new PyPI features before launch.

Supply chain security is an industry wide concern, and Python is not alone in these challenges. The Python Package Index is critical to countless users around the world. A new safety and security engineer will help alleviate the current bottleneck of support issues, remove malware faster, and keep PyPI secure for the benefit of all its users. We look forward to continuing our work with the Python Software Foundation as we work towards improving open source supply chain security.

Tom Callaway

Tom Callaway

Tom is a Principal Open Source Evangelist for AWS. He has been a part of the open source community since 1997, when he skipped his last day of junior high to go to Linux Expo. During college, he worked for a high-availability startup to cover tuition, and when they crashed along with the majority of the IT sector, he dropped out of college and went to work for Red Hat full-time. He worked for Red Hat for almost twenty years, in Support, Sales Engineering, Release Engineering, Engineering Management, University Outreach (CTO’s office), and Employment Brand. He’s an active contributor to Fedora and helped to write the Fedora Packaging and Legal Guidelines which are still in use today. He’s spoken at a number of conferences and events including SxSW, OSCON, Open Source Summit, and Red Hat Summit. He has one patent on a crazy idea that never got implemented in the real world, and is co-author of Raspberry Pi Hacks (2013, O’Reilly). When he’s not working, he finds enjoyment in 3D printing, pinball, games (board & video), geocaching, craft beer, B-movies, science fiction, trivia, traveling, and his wife and two boys. He lives in Cary, NC. Tom is also known as “spot” by many people in the open source universe, he’s gone by that nickname since the 1st grade, and he happily answers to it. Follow him on Twitter @spotfoss.