Running Vintage Software: PyPI's Aging Codebase.

2:40pm - 3:05pm on Saturday, October 7 in PennTop South

Ernest W. Durbin III

Audience Level:

Intermediate

Watch:

https://youtu.be/AUpTQzjEths

Overview

The Python Package Index, has grown enormously since it’s introduction in 2002. As a critical piece of the Python Community’s infrastructure, it has suffered many growing pains over the years. Let’s review the recent history of PyPI, lessons learned, and techniques applied to keep it running.

Description

From it’s origins in the early days as a metadata server to the current trove of hosted packages, PyPI has grown into a critical part of the Python Communities Infrastructure.

Originally conceived and implemented in 2002 as a simple set of metadata, the codebase that backs PyPI has been pushed well beyond it’s limits. While some may call it “legacy” software, I choose to view it like one may view a classic car that still sees it’s time on the road.

This talk will present some of the challenges common to keeping a Vintage Car on the road as well as their counter parts when dealing with Vintage Software and take a look at the approaches that have worked well to keep PyPI up and running as a full rewrite is underway.

Also covered will be an overview of the history of PyPI as a whole, as well as what the future holds and how you (yes you!) can contribute.

Technical Topics covered include:

• External caching, Content Delivery Networks
• Internal caching, to speed up common or repetitive requests which just can’t be cached in the CDN
• Metrics, Metrics, Metrics! Not just for monitoring! Keep track of what features just aren’t worth supporting any longer or what methods are prime for optimization
• Dealing with issues related to scaling even when major refactors or changes aren’t feasible.
• Increasing reliability

The desired outcome for attendees of this talk is a sense of how to approach a Vintage codebase that must remain up and running even as it’s predecessor may be in progress. Big rewrites can take time! Don’t feel defeated sustaining an old project, it is what your users need!

Novice attendees may find the history and high level concepts interesting, but Intermediate attendees should get the most out of this talk. Specifically some of the technical bits will make the most sense and be most applicable if you’ve ever worked on a medium to large production web service.