Improving_the_Performance_and_Scalability_of_Twitter.pdf

《Improving_the_Performance_and_Scalability_of_Twitter.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、FixingTwitter...andFindingyourownFailWhaleJohnAdamsTwitterOperationsOperations•Smallteam,growingrapidly.•Whatdowedo?•SoftwarePerformance(back-end)•Availability•CapacityPlanning(metrics-driven)•ConfigurationManagement•Wedon’tdealwiththephysical

2、plant.ManagedServices•Dedicatedteam(NTTA)•24/7Handsonremotesupport•Noclouds.Wetriedthat!•Needrawprocessingpower,latencytoohighinexistingcloudofferings•Freesustodealwithreal,intellectual,computerscienceproblems.752%2008Growth53.752.51.250Dec07Feb08Apr08Jun08Au

3、g08Oct08Dec08UniqueVisitors(inMillions)Thatwasonlythebeginning...previousgraph!UniquesNotslowingdown,despitewhatoutsiderssay.HardforoutsiderstomeasureAPIusage!Growth=Pain+anappreciationforInstitutionalizedFearMantra!FindWeakestPointMetrics+Logs+Science=Analys

4、isMantra!FindWeakestTakeCorrectivePointActionMetrics+ProcessLogs+Science=AnalysisMantra!FindWeakestTakeCorrectiveMovetoNextPointActionWeakestPointMetrics+ProcessRepeatabilityLogs+Science=AnalysisFindtheWeakestPoint•Metrics+Graphs•Individualmetricsareirrelevan

5、t•Logs•SCIENCE!•Findoutwhattheactionableitemsare.InstrumentEverything(cc)seenoevil@flickrMonitoring•Graphandreportcriticalmetricsinasnearrealtimeaspossible•Youalreadyhavethetools.•RRD•Ganglia+customgMetricscripts•MRTGDashboards•“Criticals”view•Smokeping/MRTG•G

6、oogleAnalytics•NotjustforHTTP200s/SEO•XMLFeedsfrommanagedservices•DataPorn!Analyze•Turndataintoinformation•Whereisthecodebasegoing?•Arethingsworsethantheywere?•Understandtheimpactofthelastsoftwaredeploy•Runcheckscriptsduringandafterdeploys•CapacityPlanning,no

7、tFireFighting!ForecastingCurve-fittingforcapacityplanning(R,fityk,Mathematica,CurveFit)unsignedint(32bit)Twitpocolypsestatus_idsignedint(32bit)Twitpocolypser2=0.99Deploys•Graphtime-of-deployalongsideserverCPUandLatency•Displaytime-of-last-deployondashboardlastd

8、eploytimesWhale-Watcher•Simpleshellscript,•MASSIVEWIN.•Whale=HTTP503(timeout)•Robot=HTTP500(error)•Examineslast100,000linesofaggregateddaemon/wwwlogs•“WhalesperSecond”>Wthreshold•Tharbewh