欢迎来到天天文库
浏览记录
ID:33942390
大小:1.13 MB
页数:51页
时间:2019-02-28
《12[Nov 12]Time series&sequential hypothesis testing&anomaly detection[Alex Shyr].pdf》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、AnomalydetectionandsequentialstatisticssequentialstatisticsintimeseriesAlexShyrCS294PracticalMachineLearning11/12/2009(manyslidesfromXuanLongNguyenandCharlesSutton)TwotopicsTwotopicsAnomaldetectionAnomalydetectionSequentialstatisticsReview•DimensionalityReduction•e.g.PCA•HMM•ROCcurve
2、sROCcurvesOutline•ItdtiIntroduction•AnomalyDetection–StaticExample–TimeSeries•SequentialTests–StaticHypothesisTestingStaticHypothesisTesting–SequentialHypothesisTesting–Change-pointDetectionpointDetectionAnomaliesintimeseriesdata•Timeseriesisasequenceofdatapoints,,ymeasuredtyppically
3、yatsuccessivetimes,spacedat(oftenuniform)timeintervals•AnomaliesintimeseriesdataaredataAnomaliesintimeseriesdataaredatapointsthatsignificantlydeviatefromthenormalpatternnormalpatternofthedatasequenceofthedatasequenceExamplesoftimeseriesdataExamplesoftimeseriesdataNetworktrafficdataFi
4、nancedataFinancedataHumanActivitydataApplications•Failuredetection•Frauddetection(creditcardtelephone)Frauddetection(creditcard,telephone)•Spamdetection•Biosurveillance–detectinggeographichotspots•ComputerintrusiondetectionOutline•ItdtiIntroduction•AnomalyDetection–StaticExample–Time
5、Series•SequentialTests–StaticHypothesisTestingStaticHypothesisTesting–SequentialHypothesisTesting–Change-pointDetectionpointDetectionExample:Networktraffic[Lakhinaetal,2004]Goal:Findsource-destinationpairswithhightraffic(e.g.,byratevolume)rate,volume)Backbonenetwork….Y=10030422121729
6、13….Example:NetworktrafficDatamatrixPerformPCAonmatrixY….Y=1003042212172913….Low-dimensionaldataEigenvectors….Yv=yTvyTvt1t2vv…12….Example:NetworktrafficAbilenebackbonenetworktrafficvolumeover41linkscollectedover4weeksPerformPCAon41-dimdataSelecttop5componentsSelecttop5componentsanoma
7、liesthresholdProjectiontoresidualsubspaceConceptualframework•Learnamodelofnormalbehavior•FindoutliersundersomestatisticFindoutliersundersomestatisticalarmCriteriainanomalydetection•Falsealarmrate(typeIerror)•Misdetectionrate(typeIIerror)•Neyman-Pearsoncriteria–minimizemisdetectionrat
8、ewhilefalsea
此文档下载收益归作者所有