Add max_requests_jitter configuration option. #862
Conversation
alienth
force-pushed
the
master
branch
2 times, most recently
from
3570fb8
to
aa6e82c
Compare
|
I'm 👍 for this change. A really good idea. Glad to hear it works well for you. What values do you find work well? Perhaps we should just make this the default and just make the range some fraction of the max requests setting. Say hi to @chromakode for me. |
| The maximum jitter to add to the max_requests setting. | ||
|
|
||
| The jitter causes the restart per worker to be randomized by | ||
| random(range(0, this_value)). This is intended to stagger worker |
There was a problem hiding this comment.
Nitpick: random(range(0, this_value)) -> random(range(0, this_value))
|
Thanks for the feedback! Right now we're using a There may be some use cases where someone wants restarts to be closely clustered together, although I'm having trouble thinking of any. I can alter this to be default-behaviour, or a toggleable behaviour with a static fraction of I cleaned up the doc item. Let me know if you'd like me to add the 'version' indicator. |
|
LGTM @tilgovi @berkerpeksag ? |
|
Yeah, seems fine. Let's get the version doc in there. Thanks, @alienth! |
An issue we've had when using max_requests is almost all of our workers would restart at the same time due to even balancing of requests to the workers. This would result in a resource spike during the wave of worker restarts and a bunch of queuing due to fresh workers getting slammed. This jitter splits up the restart of the workers, which will help prevent the above issue.
This is something we've been using at reddit for a while now. Not sure it is suitable for the main project, but I figured I'd pass it along. I'd be happy to alter the name of the option or any of the documentation to meet any standards or requirements of the project.