Analyzing Data with Python Presentation.pdf

《Analyzing Data with Python Presentation.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、SarahGuidoANALYZINGDATAWITH@sarah_guidoReonomyPYTHONOSCON2014ABOUTMEDatascientistatReonomyUniversityofMichigangraduateNYCPythonorganizerPyGothamorganizerABOUTTHISTALKBird’s-eyeoverview:notcomprehensiveexplanationofthesetools!Takedatafromstart-to-fi

2、nishPreprocessing:PandasAnalysis:scikit-learnAnalysis:nltkDatapipeline:MRjobVisualization:matplotlibWhatnext?WHYPYTHON?SomanytoolsPreprocessing,analysis,statistics,machinelearning,naturallanguageprocessing,networkanalysis,visualization,scalabilit

3、yCommunitysupport“Easy”languagetolearnBothascriptingandproduction-readylanguageFROMPOINTATOPOINT…X?Howtofindthebesttool(s)?The90/10ruleSimpleisbetterthancomplexWHYICHOSETHESETOOLSAvailableresourcesDocumentation,tutorials,books,videosEaseofuse(wi

4、thagrainofsalt)CommunitysupportandcontinuousdevelopmentWidelyusedPREPROCESSINGTheimportanceofdatapreprocessingAKAwrangling,munging,manipulating,andsoonPreprocessingisalsogettingtoknowyourdataMissingvalues?Categorical/continuous?Distribution?PANDAS

5、DataanalysisandmodelingSimilartoRandExcelEasy-to-usedatastructuresDataFrameDatawranglingtoolsMerging,pivoting,etcPANDASKeepeverythinginPythonCommunitysupport/resourcesUseforpreprocessingFileI/0,cleaning,manipulation,etcCombinablewithothermodule

6、sNumPy,SciPy,statsmodel,matplotlibPANDASFileI/OPANDASFindingmissingvaluesPANDASRemovingmissingvaluesPANDASPivotingPANDASOtherthingsStatisticalmethodsMerge/joinlikeSQLTimeseriesHassomevisualizationfunctionalityMACHINELEARNINGApplicationofalgori

7、thmsthatlearnfromexamplesRepresentationandgeneralizationUsefulineverydaylifeEspeciallyusefulindataanalysisMACHINELEARNINGSupervisedlearningClassificationandregressionUnsupervisedlearningClusteringanddimensionalityreductionSCIKIT-LEARNMachinelearn

8、ingmoduleOpen-sourceBuilt-indatasetsGoodresourcesforlearningSCIKIT-LEARNScikit-learn:yourdatahastobecontinuousHere’swhatoneobservation/labellookslike:SCIKIT-LEARNTransformcategoricalvalues/labelsSCIKIT-LEARNClassifi