python data analysis analyzing textual data

《python data analysis analyzing textual data》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。

1、AnalyzingTextualDataandSocialMediaInthepreviouschapters,wefocusedontheanalysisofstructureddata,mostlyintabularformat.Inreality,plaintextisthemostpredominantformofdataavailabletoday.Textanalysisappliesanalysisofwordfrequencydistributions,patternrecognition,tagging,linkandassociationanalysis,sentimen

2、tanalysis,andvisualization.WewillanalyzetextwiththePythonNaturalLanguageToolkit(NLTK)library.NLTKcomeswithacollectionofsampletextscalledcorpora.Asmallexampleofnetworkanalysiswillalsobecovered.Thefollowingtopicswillbediscussedinthischapter:•InstallingNLTK•Filteringoutstopwords,names,andnumbers•Theba

3、g-of-wordsmodel•Analyzingwordfrequencies•NaiveBayesclassification•Sentimentanalysis•Creatingwordclouds•SocialnetworkanalysisAnalyzingTextualDataandSocialMediaInstallingNLTKNLTKisaPythonAPIfortheanalysisoftextswritteninnaturallanguages,suchasEnglish.NLTKwascreatedin2001andwasoriginallyintendedasatea

4、chingtool.InstallNLTKwiththefollowingcommand:$sudopipinstallnltk$pipfreeze

5、grepnltknltk==2.0.4Asusual,wewillchecktheinstallationwithanewversionofthepkg_check.pyfile.Thefollowingimportstatementisrequired:importnltkIfeverythingworks,weshouldgetaresultsimilartothefollowing:nltkversion2.0.4nltk.appDESC

6、RIPTIONchartparser:ChartParserchunkparser:Regular-ExpressionChunkParsercollocations:Findcollocationsintextconcordance:Partnltk.ccgDESCRIPTIONFormoreinformationseenltk/doc/contrib/ccg/ccg.pdfPACKAGECONTENTSapichartcombinatorlexiconDATABackwardApplication

7、Theseperformsimplepatternmatchingonsentencestypedbyusers,andrespondwithautomaticallygnltk.chunkDESCRIPTIONClassesandinterfacesforidentifyingnon-overlappinglinguisticgroups(suchasbasenounphrases)inunrestrictedtext.Thisnltk.classifyDESCRIPTIONClassesandinterfacesforlabelingtokenswithcategorylabels(or

8、"classlabels").Typically,labelsarerepresentedwithstrinltk.clusterDESCRIPTIONThismodulecontainsanumberofbasicclusteringalgorithms.Clusteringdescribesthetaskofdiscoveringgroupsofsimilaritenltk.corpusnltk.draw