资源描述：
《校园bbs可定制爬虫的设计与实现》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、华中科技大学硕士学位论文AbstractAstheInternetnetworkisgrowingpopularityinourcountry,forthoseyoungcollegestudentswhoarewillingtoacceptnewthings,theInternethasalmostbecamethebasicnecessitiesoflife,aswellasfoodandclothing.Atthesametime,thecampusforumsbecomethemainplatformwh
2、erestudentsexpressandexchangetheirviews.Inordertounderstandthehottopicsincampus,itisverymeaningfultobuildacampusnetworkinformationmanagementsystem.Theforumspiderdesignedandimplementedbythisthesisisasub-systemoftheinformationmanagementsystem,it’smaintaskiscoll
3、ectingforumdatawhichispreparedforfutureanalyzing.Whilecrawlingforums,traditionalgeneralcrawlerswouldencounteralargenumberofduplicatelinks.Thiswouldbeawasteofresourcesandinefficient.Ontheotherhand,mostexistingforumcrawlersaretailoredforspecificusers,thereforet
4、heyonlyactonasingleforum.Thisthesishasanalyzedthedifferencesinthestructureofmanyforums,andstudiedthefeaturesandsystemarchitecturesofseveralmainstreamcrawlers,andfinallyproposedanimplementationofincrementalwebcrawlersystemwhichcouldbeappliedtomanyforums.Aftera
5、nalyzingsystemrequirements,thisthesisdesignedeachsub-module,andthenelaborateddetailsoftheimplementationofeachmodule.Themainworkofthisthesisincludesthefollowingaspects.First,analyzedthefeaturesofmanycampusforums,extractedtheircommonalitiesanddifferences,determ
6、inedcrawlingmodeforeachstyleofforums.Then,accordingtotheheatandfeaturesofforumsections,determinedincrementalcrawlingstrategybasedontheweightofforumsections.Atlast,onthepurposeofimprovingtheversatilityandflexibilityofthecrawler,thisthesisusedXQuerytemplatestop
7、arsethewebpagesandextractthecontent.Afterdeployingandrunningthecrawler,thisthesisanalyzedthetestresults,itshowsthatthespidersystemwasrunningstably,sothesystemhasmettheneedsofdesign,andisuseful.Keywords:IncrementalWebCrawler,InformationExtraction,XQueryTemplat
8、es,SaxonII华中科技大学硕士学位论文目录摘要...........................................................................................................IAbstract.............................................
