基于Hadoop平台的亿贝用户邮件数据分析(苏立).pdf

《基于Hadoop平台的亿贝用户邮件数据分析(苏立).pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、eBay M2M Email Analysis on Hadoop Pla4orm ForestSu,DanielZhangOctober 2011 eBay Inc. Confiden.al TheBirthofeBay...StartedwithaBrokenLaserPointer...AuctionWebwasbornontheLaborDayweekendinSeptember1995$30eBayFounder...soldfor$14.83USDPierreOmidyareBay Inc.

2、 Confiden.al 2 eBay Fact – 15 Years A=er … $2,000+USDTradingValuePerSecond220+MillionActiveItem300+FeaturesListingforsaleperquarter450+Million100,000linesRegisteredUsersofcoderolledoutevery2weeks10+MillionNewItemsAdded50,000CategoriesOver2BillionPhotosPe

3、rDay48BillionSQLCallsGlobalPresentsPerday2PetabytesIn33International5.5BillionAPICallsStoredMarketsPermonth25PetabytesProcesseddaily>4.4GBSourceCodeeBay Inc. Confiden.al 3 VelocityoftradingOnanaveragedayoneBay…ADiamondRingissoldeverytwominutes4Velocityof

4、tradingOnanaveragedayoneBay…Over3600MP3playersaresold5VelocityoftradingOnanaveragedayoneBay…Over300stampsaresoldeveryhour6VelocityoftradingOnanaveragedayoneBay…Anautomobileissoldeveryminute7HadoopontheLandscapeDualLoadDualLoadBusinessUsersSearchAnalysts

5、QueryDirectorDataFeedHadoopDataIntensiveSocPArcStrucDtuuraeldL/oadPlatformStrDucutuarleLdoDaadtaUnstructuredDataNode:500+DeepSystemDiskSpace:Approx.4.25PBResearchPlatformComponent:HDFS/HiveHardware:CommodityHardwarePrimary/SecondaryKnowledgeFeedbackDual

6、LoadETLLoadStruDcutuarleLdoDaadtaUnstDruucatulrLeodaDdataDataFeedDualLoadUnstructuredDataKnowledgeTransactionalDatabasesFeedbackeBay Inc. Confiden.al 8 eBay Email Universe •Members write to each other (“M2M”) •Members write to eBay •eBay writes to Member

7、s ‐Millions of times per day! •Where are these messages? oIn My Messages: Messages through eBay oIn a user’s private system: Messages outside of eBay eBay Inc. Confiden.al 9 Single User Study •What does a typical user send / receive? •Follow a single use

8、r for 50 days during Oct/Nov 2010 •User ProNile: ―User subscribed to daily deal ―User was member of: eBay Bucks eBay VIP PayPal Advantage ―User is mainly a buyer but also a casual seller eBay Inc. Confiden.al 10 Single User Stu