微博热点话题发现的方法实现

《微博热点话题发现的方法实现》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、万方数据AbstractMicro—blogplatfornahasbeenpopularinthevastnumberofnetworkuserswithitshightransmissionefficiencyandapproximateinstantinformationacquisitionspeed．Forthemicro-blogmessagecanbeissuedbyavarietyofintelligentmobileterminal，somehottopicsinmicro-blogplatformCanberef

2、lectednearlyreal-time．Micro-blogplatformgeneratesalargeamountofdataeveryday，it’Sverydifficultforuserstoobtainthecurrenthottopicsbybrowsingallthemicro—blogdata，Discoveryhottopicsfromalargeamountofdataisveryhelpfultouserstounderstandthecurrentsocialhotspotquickly．Therefo

3、re，thepapercarriedhottopicdiscoveryinmicroblogiswithgoodresearchandpracticalsignificance．ThispaperdesignedandimplementedawebcrawlerbasedonWebMagicreptilesframeworktocrawlingthemicro-blog’Sdata．Accordingtothecharacteristicsofmicro-blogtextdata,preprocessedthedataincludi

4、ngdatacleansing，segmentationandlocallexiconfiltering．Builtahashtablestructurebasedonthepreprocesseddata,inordertocalculatethecorrespondingTFIDFweightmatrix，anddesignedaparallelprogramundertheMapReducecomputingframeworktosolvetimecomplexityandspacecomplexitybottleneckpr

5、oblemofthecalculationofTFIDFweight；Inaddition，clustertheTFIDFweightmatrixbytheBRICHhierarchicalclusteringalgorithmandK-Meansclusteringalgorithmrespectively,andMapreducedtheK·Meansalgorithm．Combinedwiththechisquaretestcalculatedtheunexpectedvalueofeachmicro-blogdocument

6、cluster,tofindthehotone．Finallyextractsomecom_monwordsfromthehotclusterSOastogetahottopic．Finally,inordertoverifythevalidityofthehottopicdiscoverymethod，thispapercalculatedeachmiceoblog’SheatwiththenumberofcommentsandforwardingnumberSOastofindoutthehotmicro-blogs，toven

7、ifythehottopicsbyjudgingifthehotwordsareinthehotmicro-blogs．Theresultsshowthatthemethodthispaperproposedhasagoodeffectonhottopicdiscoveryinsomefield．万方数据AbstractKeyWords：Microblog；topicsdetection；BRICH；K-MeaIls；MapReduce万方数据目录摘要⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯．IAbstract⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯

8、⋯．．II目录⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯．IVII第一章绪论⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯1第一节研究背景和意义⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯．1第二节国内外研究现状⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯．2第三节论文主要内容⋯⋯⋯⋯⋯⋯⋯