performance tuning profiling and concurrency

performance tuning profiling and concurrency

ID:7323454

大小:1.06 MB

页数:55页

时间:2018-02-11

performance tuning  profiling  and concurrency_第1页
performance tuning  profiling  and concurrency_第2页
performance tuning  profiling  and concurrency_第3页
performance tuning  profiling  and concurrency_第4页
performance tuning  profiling  and concurrency_第5页
资源描述:

《performance tuning profiling and concurrency》由会员上传分享,免费在线阅读,更多相关内容在工程资料-天天文库

1、PerformanceTuning,Profiling,andConcurrency"Prematureoptimizationistherootofallevil"–DonaldKnuth,arenownedcomputerscientistandmathematicianIntherealworld,therearemoreimportantthingsthanperformance,suchasfeatures,robustness,maintainability,testability,andusability.That'soneofthereasonsthatwedelayed

2、discussingthetopicofperformanceuntilthelastchapterofthebook.Wewillgivehintsonimprovingperformancewithprofilingasthekeytechnique.Formulticore,distributedsystems,wewilldiscusstherelevantframeworkstoo.Wewilldiscussthefollowingtopicsinthischapter:•Profilingthecode•InstallingCython•CallingtheCcode•Cre

3、atingapoolprocesswithmultiprocessing•SpeedingupembarrassinglyparallelforloopswithJoblib•ComparingBottlenecktoNumPyfunctions•PerformingMapReducewithJug•InstallingMPIforPython•IPythonParallelPerformanceTuning,Profiling,andConcurrencyProfilingthecodeProfilingisaboutidentifyingpartsofthecodethataresl

4、oworusealotofmemory.Wewillprofileamodifiedversionofthesentiment.pycodefromChapter9,AnalyzingTextualDataandSocialMedia.Thecodeisrefactoredtocomplywithmultiprocessingprogrammingguidelines.Youwilllearnaboutmultiprocessinglaterinthischapter.Also,wesimplifiedthestopwordsfiltering.Thethirdchangeistohav

5、efewerwordfeaturesasthereductiondoesn'timpactaccuracy.Thislastchangehasthemostimpact.Theoriginalcoderanforabout20seconds.Thenewcoderunsfasterthanthatandwillserveasthebaselineinthischapter.Somechangeshavetodowithprofilingandwillbeexplainedlaterinthissection.Pleaserefertotheprof_demo.pyfileinthisbo

6、ok'scodebundle:importrandomfromnltk.corpusimportmovie_reviewsfromnltk.corpusimportstopwordsfromnltkimportFreqDistfromnltkimportNaiveBayesClassifierfromnltk.classifyimportaccuracyfromlprof_hackimportprofile@profiledeflabel_docs():docs=[(list(movie_reviews.words(fid)),cat)forcatinmovie_reviews.cate

7、gories()forfidinmovie_reviews.fileids(cat)]random.seed(42)random.shuffle(docs)returndocs@profiledefisStopWord(word):returnwordinsworlen(word)==1@profiledeffilter_corpus():review_words=movie_reviews.words()print"#Review

当前文档最多预览五页,下载文档查看全文

此文档下载收益归作者所有

当前文档最多预览五页,下载文档查看全文
温馨提示:
1. 部分包含数学公式或PPT动画的文件,查看预览时可能会显示错乱或异常,文件下载后无此问题,请放心下载。
2. 本文档由用户上传,版权归属用户,天天文库负责整理代发布。如果您对本文档版权有争议请及时联系客服。
3. 下载前请仔细阅读文档内容,确认文档内容符合您的需求后进行下载,若出现内容与标题不符可向本站投诉处理。
4. 下载文档时可能由于网络波动等原因无法下载或下载错误,付费完成后未能成功下载的用户请联系客服处理。