Predicting Global Unrest with GDELT and SQL on Hadoop Presentation.pdf

《Predicting Global Unrest with GDELT and SQL on Hadoop Presentation.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、PredictingGlobalUnrestwithGDELTandInfiniDBAboutMeCTO&FounderatInfiniDBDBA,datageekPerformancetuningfanatic2AgendaGDELTBackgroundGlobalDatabaseofEvents,LanguageandToneIsaacAsimovandtheFoundationSeriesInfiniDBBackgroundPuttingittogetheranddivingintothedata3GDELTBac

2、kgroundGlobalDatabaseofEvents,LanguageandTonehttp://gdeltproject.org/FoundedbyKalevH.LeetaruistheYahoo!FellowinResidence,GeorgetownUniversityinWashington,DCAGlobalDatabaseofSocietySupportedbyGoogleIdeas,theGDELTProjectmonitorstheworld'sbroadcast,print,andwebnewsfromnearl

3、yeverycornerofeverycountryinover100languagesandidentifiesthepeople,locations,organizations,counts,themes,sources,andeventsdrivingourglobalsocietyeverysecondofeveryday,creatingafreeopenplatformforcomputingontheentireworld.5GDELTBackgroundResourcesProjectSource:http://gdel

4、tproject.org/ipython:http://nbviewer.ipython.org/github/dmasad/GDELT_Intro/tree/master/R:http://cran.r-project.org/web/packages/GDELTtools/index.htmlBlogging/Analysis/Commentary:http://johnbeieler.org/http://andrewhalterman.com/2013/08/28/gdelt_dplyr_sqlite/https://dartt

5、hrowingchimp.wordpress.com/tag/gdelt/6GDELTBackgroundDataAvailability:http://data.gdeltproject.org/events/index.htmlEventscovering1979throughpresent1979through20051fileperyear2006throughMarch20131filepermonthApril1,2013throughpresent1f

6、ileperdayDimensiondata:http://gdeltproject.org/data.html#documentation7GDELTBackground–DataDefinitionmysql>selectmin(SQLDATE),Events->max(SQLDATE),->count()fromgdelt;36yearsunderanalysis+--------------+--------------+-----------+min(SQLDATE)max(SQLDATE)count()+---------

7、-----+--------------+-----------+¼billionrowstodate1979010120140713263219134+--------------+--------------+-----------+Updateddaily1rowinset(1.65sec)mysql>descgdelt;+-----------------------+-----------++-----------------------+-----------+FieldTypeFieldType+-----------

8、------------+-----------++-----------------------+-----------+...GLOBALEVENTIDbigint(2)Actor1Codechar(1