Where Django Caching Bust at the Seams

Editor's Notes

1. try to focus on middle size applications. strategies, tools, implementation details, warnings. and ways of thinking about the problem space\n\n- you've written a mid-sized django app\n- maybe you already have some caching.\n\n
2. \n
3. \n
4. \n
5. \n
6. \n
7. \n
8. \n
9. \n
10. \n
11. \n
12. \n
13. apologies, promised\nstale cache from previous talks\n
14. \n
15. \n
16. \n
17. \n
18. \n
19. \n
20. \n
21. \n
22. planning and implementing is what we focus on\n
23. \n
24. \n
25. \n
26. \n
27. \n
28. \n
29. \n
30. \n
31. * what -- more on this\n\n* where rendered, could be multiple spots\n\n* Updates outside: Asynchronous: e.g. cron, sphinx. \nAsynchronous may be triggered by request-response cycle\n\n** can mix and match. beware dependencies. even of same type. two-phased template rendering\n- two phase template renderings\n\n
32. * what -- more on this\n\n* where rendered, could be multiple spots\n\n* Updates outside: Asynchronous: e.g. cron, sphinx. \nAsynchronous may be triggered by request-response cycle\n\n** can mix and match. beware dependencies. even of same type. two-phased template rendering\n- two phase template renderings\n\n
33. * what -- more on this\n\n* where rendered, could be multiple spots\n\n* Updates outside: Asynchronous: e.g. cron, sphinx. \nAsynchronous may be triggered by request-response cycle\n\n** can mix and match. beware dependencies. even of same type. two-phased template rendering\n- two phase template renderings\n\n
34. * what -- more on this\n\n* where rendered, could be multiple spots\n\n* Updates outside: Asynchronous: e.g. cron, sphinx. \nAsynchronous may be triggered by request-response cycle\n\n** can mix and match. beware dependencies. even of same type. two-phased template rendering\n- two phase template renderings\n\n
35. next up strategies for this by wiggins/kyle\n
36. \n
37. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
38. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
39. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
40. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
41. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
42. twitter sidebar asynch update every 5 minutes. fine.\nuser does x. x replaces y. user asks for x and gets y. problem.\n\nIf changes constantly, caching no value\n
43. \n
44. !!! “string theory” - helpful to imagine reaching into a db (or persistent layer) with strings attached\n\n!!! reach into database with a join. cache in query. \n\nhigher up you cache, likely the more \n\nstrategy next\n
45. - Who in the crowd is currently using memcache in their Django applications?\n- What about Django's basic per-site caching?\n- Has anyone seen Noah Silas talk about Cache Rules Everything Around Me?\n- in that talk, he briefly mentions the source of his title, but doesn’t want to talk about it\n\n>> We ARE going to talk about Wu Tang Clan\n\n
46. - but only for this slide\n- they had the hit Cash Rules Everything Around Me\n\n
47. - If you want to discuss more, see me after the talk\n\n
48. - Back to Django\n\n- So does caching in Django just mean using memcache?\n\n- Well, it is commonly used and easy to set up\n- A lot of cache frameworks have documentation along the lines of "... developed specifically with the use of memcached in mind. "\n\nNext:\n- We'd like to talk about caching in the context of an imaginary website. \n- We will walk through the growth of this site and what issues we'd hit along the way\n\n\n
49. - As many of you may have experienced, we have a great idea, but we don't have the time or money to build a huge, scaling site from the very start\n- We shouldn't completely ignore scalability, but with limited resources, a site that launches is better than one that is prematurely optimized.\n- Also, marketing or management might have constraints on the project outside of development's control\n- We aren't here to wag our finger at you and tell you how you should have done things all along, we're here to walk through some real issues and present solutions\n
50. - So we have our site live\n- Although there isn't much data on it\n- And not many users\n- Perhaps a significant amount of the site traffic is from the development team testing the site.\n- it happens\n- Perhaps all of the actual business is from family and friends of the team.\n- But now that the site is live, the dev team has time to focus on the next phase of features.\n- Let's talk about ways that we can improve the site, hopefully to help prepare it for growth\n
51. \n- Of course, some of the primary concerns of the Django developers on the team are the responsiveness of our application, and the load it creates on the web server.\n\n- Lucky for us developers that like to fly by the seat of our pants, Django provides some extremely quick ways of adding basic per-site caching to every page. This basically means that if a page has been requested within the cache timeout, it will be served to any other users from the cache instead of the database\n\n- This is close to upstream caching like Varnish, but it’s still aware of things like the request user’s authentication status.\n\n- Hopefully everyone here is already past this level, but we will talk more about deploying memcache later.\n
52. \n- Of course, some of the primary concerns of the Django developers on the team are the responsiveness of our application, and the load it creates on the web server.\n\n- Lucky for us developers that like to fly by the seat of our pants, Django provides some extremely quick ways of adding basic per-site caching to every page. This basically means that if a page has been requested within the cache timeout, it will be served to any other users from the cache instead of the database\n\n- This is close to upstream caching like Varnish, but it’s still aware of things like the request user’s authentication status.\n\n- Hopefully everyone here is already past this level, but we will talk more about deploying memcache later.\n
53. \n- So then it happens. Your management lands a deal with a rival company and suddenly you have way more data than you anticipated.\n\n- The per-site caching helps with a lot of the site, but there are still some expensive queries happening, and probably on the index page\n
54. So what is a solution?\n- perhaps there are only a few expensive queries to make.\n- Just wrap a cache read/write around them, and they will be cached for subsequent\n\n- don’t name your cache key “my_cached_items”\n\n- this will result in keys generated as-needed, sprinkled througout the code\n\n
55. So what is a solution?\n- perhaps there are only a few expensive queries to make.\n- Just wrap a cache read/write around them, and they will be cached for subsequent\n\n- don’t name your cache key “my_cached_items”\n\n- this will result in keys generated as-needed, sprinkled througout the code\n\n
56. So what is a solution?\n- perhaps there are only a few expensive queries to make.\n- Just wrap a cache read/write around them, and they will be cached for subsequent\n\n- don’t name your cache key “my_cached_items”\n\n- this will result in keys generated as-needed, sprinkled througout the code\n\n
57. - Pros: quick\n- Cons: dirty\n
58. - One of the first issues this site is going to run into is what happens when there is a cache miss?\n\n- Pretending that our new expanded dataset is actually enticing customers, In the case of heavy traffic, a cache miss means many requests will trigger database reads until the cache can be filled. This stampede of requests is commonly referred to as a ‘Thundering Herd’.\n\n- Even though a small percentage of requests trigger a read, they are occurring at the same time. \n
59. \n- A solution to Thundering Herd is to not drop values completely from the cache, but to refresh the value when the first miss occurs, or when an object is deemed to be ‘stale’, and continue to provide the cached value to the rest of the herd until the value can be updated. We refer to this as the Publish Model, and we will talk more about it later.\n\n- One issue is that the rest of the herd will be getting the stale value, but it probably is fine for the short time.\n\n- The new value will probably take seconds, if that, to calculate and refresh the cache, but only one read will be sent to the database.\n\n
60. So even with \n\n- Often, requests are spread out evenly across a large dataset\n\n- Sometimes, users are not requesting anything seen in a long time. The data might be unique to that user\n\n- This doesn’t apply to our croupon site, but it would with Address lookups on a map, mail clients, etc\n\n- If you are running into these sorts of issues, then your site is moving along that spectrum of what is an app and what is a content site. Remember the slide from Adrian’s keynote yesterday on the sweet spot that Django aims to serve. It doesn’t reach all the way to the ‘app’ end of the spectrum\n\n- Luckily Croupon is solidly toward the content end, and is pretty well suited for a lot of the solutions we are talking about\n\nPASS TO WIGGINS to talk about deploying caching\n
61. Versioning your caches.\nSeparating your caches based on use.\nUsing consistent hashing algorithms.\nDeploying memcached with Elasticache.\n
62. - What happens when you need to incrementally push a code change that alters the way a cache value is generated?\n- You need a way to have the new code write to a different key so that the old values are preserved\n - staging and production\n- Versioning according to git commit hash is nice, but that means settings needs to know the hash before it is made, fabric is a good solution for this.\n
63. - What happens when you need to incrementally push a code change that alters the way a cache value is generated?\n- You need a way to have the new code write to a different key so that the old values are preserved\n - staging and production\n- Versioning according to git commit hash is nice, but that means settings needs to know the hash before it is made, fabric is a good solution for this.\n
64. - What happens when you need to incrementally push a code change that alters the way a cache value is generated?\n- You need a way to have the new code write to a different key so that the old values are preserved\n - staging and production\n- Versioning according to git commit hash is nice, but that means settings needs to know the hash before it is made, fabric is a good solution for this.\n
65. \nuse a separate cache for sessions\n
66. \nuse a separate cache for sessions\n
67. \nuse a separate cache for sessions\n
68. \n
69. \n
70. \n
71. \n
72. \n
73. \n
74. \n
75. \n
76. \n
77. \n
78. MOVING ON TO: elasticache\n
79. PASS TO KYLE to talk about optimizing cache\n
80. PASS TO KYLE to talk about optimizing cache\n
81. PASS TO KYLE to talk about optimizing cache\n
82. PASS TO KYLE to talk about optimizing cache\n
83. \n
84. \n
85. NOW ONTO PROFILING\n
86. NOW ONTO PROFILING\n
87. NOW ONTO PROFILING\n
88. NOW ONTO PROFILING\n
89. NOW ONTO PROFILING\n
90. \n
91. - I’m going to talk about three topics related to optimizing your cache\n- This is going to be very high-level\n
92. - I’m going to talk about three topics related to optimizing your cache\n- This is going to be very high-level\n
93. - I’m going to talk about three topics related to optimizing your cache\n- This is going to be very high-level\n
94. - Everyone should be using debug toolbar\n
95. - Everyone should be using debug toolbar\n
96. - Everyone should be using debug toolbar\n
97. \n- Croupon marketing decides that we need to have croupon counts peppered around the site to let customers know just how many croupons there are available in their area for each category. \n- Unfortunately, calculating these counts is very expensive. \n- The site may be lightly used, but each index page would generate a lot of joins and filters.\n-very expensive database lookups\n\n- > This is what I see in my head\n\n- The default behavior of cache is to let the process of the first read populate the cache\n-That’s for a static page that takes two seconds to load, but a cached version gets it down to 100 milliseconds\n- But that won’t work in this instance, because you don’t want any users to have to wait for these expensive lookups\n- Especially because these reads are from data that doesn’t change often\n
98. - so what is cache warming? It is filling the cache in the background, asynchronously with web requests\n\n- The solution is to warm the cache manually, not passively. Give the user what they requested, THEN do the work. Don’t make a user suffer to make your job easier.\n\n- Before the cache expires, refresh the data and reset the expiration date.\n\n- If the data in the cache is a list of items, it is easy to iterate over each item in a queryset.\n\n- For more complicated queries, write a custom manage.py command\n\n- All of this allows the synchronous web requests to remain fast for users\n
99. \n- manage.py command is simplest\n\n- Run the command just slightly more often than your cache timeout is set for (16 -> 15)\n\n- some caching frameworks want you to register with a warming task\n\n- That can be a quick solution, but it is better to be explicit than implicit. \n- If a different developer down the road wants to see what is being warmed, it is hard to trace back if you aren’t calling a function that explicitly lists what is being done\n
100. Here are some of the more popular frameworks\n\nHere’s a shameless plug for our own CacheModel\n\nYou can find the entire grid on django packages\n
101. - You use Cache Machine by inheriting from its CachingMixin and setting its CachingManager as your model’s default manager\n\n- It uses “flush lists” to keep track of the cached queries an object belongs to. Then it will iterate over that list and flush each one when an object is saved or deleted.\n- It also follows foreign keys and flushes them as well, so that’s nice\n\n- Cache Machine also includes a Jinja2 extension to cache template fragments based on the querysets used inside that fragment. That tag will get added to the flush lists, so it will get invalidated just like normal querysets.\n\n\n
102. - This is a popular framework and is updated fairly often. \n- So, of course, it works with Django 1.4\n\n- "Johnny provides a number of backends, all of which are subclassed versions of django builtins that cache “forever” when passed a 0 timeout."\n\n- Wiggins will talk about how this relates to the publish model\n- The important thing is that the framework gives you the flexibility to use various backends with the same code, much like the Django ORM does with various databases.\n\n\n
103. - Drop-in caching\n- Your model can just inherit from CacheModel and you get caching on your default manager\n\n
104. - Written by Noah Silas, who we’ve mentioned\n\n- It provides a controller that you can set on a model, then call that instead of calling your default manager\n\n- myModel.cache.get() instead of myModel.objects.get()\n\n\nPASS TO WIGGINS to talk about the Publish Model\n
105. The Publish Model - 10min (wiggins)\n - what is the publish model\n - how do you implement it?\n - cachemodel\n\n
106. \n
107. \n
108. - well defined keys in one DRY place\n- define functions that celery can use to publish to the cache\n- you need to pre-warm since the cache will be empty initially.\n
109. - well defined keys in one DRY place\n- define functions that celery can use to publish to the cache\n- you need to pre-warm since the cache will be empty initially.\n
110. - well defined keys in one DRY place\n- define functions that celery can use to publish to the cache\n- you need to pre-warm since the cache will be empty initially.\n
111. - well defined keys in one DRY place\n- define functions that celery can use to publish to the cache\n- you need to pre-warm since the cache will be empty initially.\n
112. - this is an example of how we would implement the publish model on croupon\n\n- cachekeys is a dictionary of tuples, (key_format_string, publish_function)\n\n- publish functions are decorated as celery tasks so they can be run async or inline.\n
113. - this is an example of how we would implement the publish model on croupon\n\n- cachekeys is a dictionary of tuples, (key_format_string, publish_function)\n\n- publish functions are decorated as celery tasks so they can be run async or inline.\n
114. - this is an example of how we would implement the publish model on croupon\n\n- cachekeys is a dictionary of tuples, (key_format_string, publish_function)\n\n- publish functions are decorated as celery tasks so they can be run async or inline.\n
115. - before we get into the weeds of the functions, here is how they are used.\n
116. - before we get into the weeds of the functions, here is how they are used.\n
117. - before we get into the weeds of the functions, here is how they are used.\n
118. - before we get into the weeds of the functions, here is how they are used.\n
119. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
120. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
121. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
122. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
123. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
124. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
125. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
126. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
127. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
128. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
129. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
130. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
131. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
132. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
133. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
134. - key_fmt % kwargs is naive and you will need to do better cleaning of hash keys based on arguments\n\n- FOREVER_TIMEOUT can be 0 if you are using pylibmc, otherwise set it to one year...\n\n- we fall back to thundering herd if there is a cache miss using .apply()\n
135. \n