stabilizing the registry C J Silverio director of engineering, npm @ceejbot This is the story of a plucky package registry named npm scaling problem manifesting itself as a stability problem "scaling"capacity to meet growing demands "At scale"huge demand & lots of data "stability"not falling over under normal demand What's normal demand? 129K packages 239 GB package tarballs 40 million pkg dls/day 1500 req/sec, peak 3200 "Legacy" Anything you've put into production this is the story of a legacy system becoming more flexible January 2013 20K packages .5 million dls/day Oct 2013 44K packages 108 million dls/month 3.6 million dls/day our plucky little registry had to change step 1: CDNPut in front of the registry cache ruleseverythingaround me step 2: tarballs get them out of couchdb tarballs are huge! couch runs better without them base64 decoding is work. January 2014 60K packages 6+ million dls/day step 3: visibility are things going wrong? what's going wrong? reactive monitoring monitor deeply fix things quickly proactive monitoring self-healing (also things don't break) monitoring is unit testing Add monitoring after every outage visibility is a prerequisite but not a solution act on what monitoring and metrics reveal step 4: redundancy several CouchDBs! reads, writes, & replication fewer responsibilities for each piece isolates errors step 5: automation ansible no server is special June 2014 Superficially similar. June 2014 80K packages 10 million dls/day step 6: simplification now that it's not on fire we can modify at leisure Nov 2014 105K packages 28 million dls/day peak 50/50 AWS region split no AWS-specific magic Ubuntu 14.04 Trusty Fastly: geoloc + varnish haproxy + CouchDB nginx + a filesystem where's the node? registry 2electric boogaloo with 500% more node haproxy + node services couchdb ➜ postgres redis for caching nginx + filesystem more complicated more flexible & redundant more scaling dials to turn excited about postgres ad-hoc queries are fun scaling node is exactly like scaling everything else Understand system get visibility cool down hot spots add redundancy npm client <3 npm install -g npm@latest npm loves you




需要 5 金币 [ 分享pdf获得金币 ] 0 人已下载





下载需要 5 金币 [金币充值 ]
亲,您也可以通过 分享原创pdf 来获得金币奖励!