introduction 和kernel function 在functional data analysis上的应用

《introduction 和kernel function 在functional data analysis上的应用》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。

1、(chapters1,2,3,4)IntroductiontoKernelsMaxWellingOctober120041Introduction•Whatisthegoalof(pickyourfavoritename):-MachineLearning-DataMining-PatternRecognition-DataAnalysis-StatisticsAutomaticdetectionofnon-coincidentalstructureindata.•Desiderata:-Robustalgorithmsinsensitivetooutliersandwrongm

2、odelassumptions.-Stablealgorithms:generalizewelltounseendata.-Computationallyefficientalgorithms:largedatasets.2Let’sLearnSomethingFindthecommoncharacteristic(structure)amongthefollowingstatisticalmethods?1.PrincipalComponentsAnalysis2.Ridgeregression3.Fisherdiscriminantanalysis4.Canonicalcor

3、relationanalysisAnswer:TWeconsiderlinearcombinationsofinputvector:fxwx()=Linearalgorithmareverywellunderstoodandenjoystrongguarantees.(convexity,generalizationbounds).3Canwecarrytheseguaranteesovertonon-linearalgorithms?FeatureSpacesdΦ:x→Φ(),xRF→non-linearmappingtoFΦ1.high-DspaceL22.infinite-

4、Dcountablespace:3.functionspace(Hilbertspace)22example:(,)(,,2)xy→xyxy4RidgeRegression(duality)T22problem:minwii∑(ywx−+)λ

5、

6、w

7、

8、i=1regularizationinputtargetTT−1solution:wXX=()+λIXydxdinversedTT−1=X()XX+λIy×inverseT−1=+XGIy()λG=<>xx,ijij=∑xiiαGram-matrixi=1DualRepresentation5linearcomb.dat

9、aKernelTrickNote:InthedualrepresentationweusedtheGrammatrixtoexpressthesolution.KernelTrick:kernelReplace:xx→Φ(),ΦG=→G=<Φ(),()xΦx>=Kxx(,)ijijijijijIfweusealgorithmsthatonlydependontheGram-matrix,G,thenweneverhavetoknow(compute)theactualfeaturesΦThisisthecrucialpointofkernelmethods6Modula

10、rityKernelmethodsconsistoftwomodules:1)Thechoiceofkernel(thisisnon-trivial)2)ThealgorithmwhichtakeskernelsasinputModularity:Anykernelcanbeusedwithanykernel-algorithm.somekernels:somekernelalgorithms:-supportvectormachine2(

11、

12、−−xyc

13、

14、/)kxye(,)=-Fisherdiscriminantanalysisdkxy(,)(,=+θ)-kernelr

15、egressionkxy(,)=tanh(αθ+)-kernelPCA1kxy(,)=-kernelCCA722

16、

17、xyc−+

18、

19、WhatisaproperkernelDefinition:Afinitelypositivesemi-definitefunctionkxyR:×→isasymmetricfunctionofitsargumentsforwhichmatricesformedbyrestrictiononanyfinitesubsetofpointsisp