基于Spark的面向10亿级别特征的大规模机器学习

《基于Spark的面向10亿级别特征的大规模机器学习》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、基于Spark的面向十亿级别特征的大规模机器学习YanboLiang(ApacheSparkcommitter@Hortonworks)Outline•Background•Vector-freeL-BFGSonSpark•Logisticregressiononvector-freeL-BFGS•Performance•IntegratewithexistingMLlib•FutureworkOutline•Background•Vector-freeL-BFGSonSpark•Logisticregressiononvector-f

2、reeL-BFGS•Performance•IntegratewithexistingMLlib•FutureworkBigdata+bigmodels=betteraccuraciesCTRPipelineFeatureModelExperimentSelectionTrainingExperiUsermentFeatureModelCTRasTransformEvaluationServicemodelAdFeaturebaseFeatureModelReal-timeEncodingValidationFeatureonlineC

3、ontextfeatureFeatureModelCalibrationEvaluationStagingCTRModel•Logisticregression(LR)•LRonSGD/LBFGS–batch•Followtheregularizedleader(FTRL)-online•Factorizationmachine(FM/FMM)•Gradientboostingtree(GBT)•Deepneuralnetworks(DNN)•EnsembleState-of-the-artsolution•GBDT+LR•DNN+LR

4、Large-scaleoptimization•MPI-Sync•Spark-Sync•ParameterServer-AsyncSpark:aunifiedplatform“HiddenTechnicalDebtinMachineLearningSystems”,GoogleOptimization•L-BFGS•SGD•WLSMLliblogisticregressionComputlossandgradientforeachinstance,andsumthemuplocallyComputlossandReducefromHan

5、dleregularizationBroadcastcoefficientsgradientforeachexecutorstogetandtoExecutorsinstance,andsumlossSumanduseL-BFGS/OWL-QNthemuplocallygradientSumtofindnextstepFinalmodelcoefficientsComputlossandgradientforeachDriver/Controllerinstance,andsumInitializecoefficientsthemupl

6、ocallyExecutors/WorkersloopuntialconvergeBottleneck1Computlossandgradientforeachinstance,andsumthemuplocallyComputlossandReducefromHandleregularizationBroadcastcoefficientsgradientforeachexecutorstogetandtoExecutorsinstance,andsumlossSumanduseL-BFGS/OWL-QNthemuplocallygr

7、adientSumtofindnextstepFinalmodelcoefficientsComputlossandgradientforeachDriver/Controllerinstance,andsumInitializecoefficientsthemuplocallyExecutors/WorkersloopuntialconvergeSolution1Computlossandgradientforeachinstance,andsumReducefromthemuplocallyexecutorstogetComputl

8、ossandpartiallossSumgradientforeachandgradientSumReducefrominstance,andsumexecutorstogetthemuplocallyfi