[dataguru.cn]Understanding Convolutional Neural Networks

《[dataguru.cn]Understanding Convolutional Neural Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、UnderstandingConvolutionalNeuralNetworksJayanthKoushikLanguageTechnologiesInstituteCarnegieMellonUniversityPittsburgh,PA15213jkoushik@cs.cmu.eduAbstractConvoulutionalNeuralNetworks(CNNs)exhibitextraordinaryperformanceonavarietyofmachinelearningtasks.However,theirmathematicalpropertiesandbehaviora

2、requitepoorlyunderstood.Thereissomework,intheformofaframework,foranalyzingtheoperationsthattheyperform.Thegoalofthisprojectistopresentkeyresultsfromthistheory,andprovideintuitionforwhyCNNswork.1Introduction1.1ThesupervisedlearningproblemWebeginbyformalizingthesupervisedlearningproblemwhichCNNsare

3、designedtosolve.Wewillconsiderbothregressionandclassification,butrestrictthelabel(dependentvariable)tobeunivariate.LetX2XRdandY2YRbetworandomvariables.WetypicallyhaveY=f(X)forsomeunknownf.Givenasamplef(xi;yi)gi=1;:::;ndrawnfromthejointdistributionofXandY,thegoalofsupervisedlearningistolearnamapp

4、ingf^:X!Ywhichminimizestheexpectedloss,asdefinedbyasuitablelossfunctionL:YY!R.However,minimizingoverthesetofallfunctionsfromXtoYisill-posed,sowerestrictthespaceofhypothesestosomesetF,anddefinef^=argminE[L(Y;f(X))](1)f2F1.2LinearizationAcommonstrategyforlearningclassifiers,andtheoneemployedbykernelm

5、ethods,istolinearizearXiv:1605.09081v1[stat.ML]30May2016thevariationsinfwithafeaturerepresentation.AfeaturerepresentationisanytransformationoftheinputvariableX;achangeofvariable.Letthistransformationbegivenby(X).NotethatthetransformedvariableneednothavealowerdimensionthanX.Wewouldliketoconstruct

6、afeaturerepresentationsuchthatfislinearlyseparableinthetransformedspacei.e.f(X)=h(X);wi(2)forregression,orf(X)=sign(h(X);wi)(3)forbinaryclassification1.ClassificationalgorithmslikeSupportVectorMachines(SVM)[3]useafixedfeaturerepresentationthatmay,forinstance,bedefinedbyakernel.1Multi-classclassifica

7、tionproblemscanbeconsideredasmultiplebinaryclassificationproblems.29thConferenceonNeuralInformationProcessingSystems(NIPS2016),Barcelona,Spain.1.3SymmetriesThetransformationinducedbykernelmethodsdonotalwayslinearizefesp