This week, I'm stepping aside to let Jonathan Freeman, a developer at my company, take center stage. The topic is Django, the Python Web framework, and how to scale it. To deliver the goods, Jonathan interviewed two experts at the ticketing service Eventbrite with extensive experience putting Django to task: Simon Willison, Director of Architecture, and John Shuping, Principle Systems Engineer. Shuping has been at Eventbrite for years, helping it grow and transform, while Simon joined the team last year when Lanyrd, a company he co-founded, was acquired. If you're interested in learning more, be sure to check out The Lanyrd Blog and Eventbrite's Engineering Blog. -- Andrew Oliver
Jonathan Freeman: How are you using Django?
Simon Willison: We have two code bases we can talk about. We have the Lanyrd code base and the Eventbrite code base. As you know, Lanyrd was acquired by Eventbrite just over seven months ago. It's very interesting to contrast the two. Lanyrd is a greenfield development: We started out with Django three and a half years ago from a blank slate. Eventbrite's code base is much older, seven years old, and Eventbrite adopted Django three years ago, but there's still some code at Eventbrite that is in a sort of custom Python framework that predates Django.
[ Work smarter, not harder -- download the Developers' Survival Guide from InfoWorld for all the tips and trends programmers need to know. | Keep up with the latest developer news with InfoWorld's Developer World newsletter. ]
They operate at very different scales. Eventbrite sold over $1 billion worth of tickets last year, so there's a huge amount of traffic and cash transactions going through that stack. Lanyrd is much smaller and doesn't have to deal with payment operations, so there's more flexibility. We can take risks with the code base because we're not dealing with people's cash transactions.
Freeman: "Scale" can mean a lot of things. What do you mean by "scale"?
Willison: There's more than one thing you need to scale. There's scaling up the amount of traffic that your site's handling, which is a very straightforward definition. The more hits per second you can handle, the better. There's also scaling the complexity of your code base. As code bases get larger and more complex, there are tricks you use to manage that complexity. Finally, there's scaling your engineering team. The Lanyrd team was six people, the Eventbrite engineering team is well over 80 now. There are things that you have to do differently when you have that many engineers working on the same code base.
Freeman: What do you see as being most important when scaling for traffic?
Willison: When scaling for traffic, the big thing -- this is something you get from Django and PHP and most Web frameworks these days -- is the concept of the shared nothing architecture. The application servers are dumb boxes that have databases and caches in the background that they're talking to, but fundamentally you can handle more traffic at the application layer by deploying more application servers. That moves the scaling challenges to the database and to the caching layer.
Scaling databases is always difficult. At Eventbrite, we make very heavy use of MySQL and MySQL replications. I'm not sure how many slaves we're running now?
John Shuping: For our core primary database, it's two masters and 10 slaves. There's no ability built into Django to route requests to a particular database for writes versus reads. One thing we've done at Eventbrite is instrument Django in a way that we can route inserts and updates to the masters and selects to the slaves.