Understanding intermediate layers using linear英文文献资料

《Understanding intermediate layers using linear英文文献资料》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、UnderstandingintermediatelayersusinglinearclassifierprobesGuillaumeAlain&YoshuaBengioDepartmentofComputerScienceandOperationsResearchUniversitédeMontréalMontreal,QC.H3C3J7guillaume.alain.umontreal@gmail.comAbstractNeuralnetworkmodelshaveareputationforbeingblackboxes

2、.Weproposeanewmethodtounderstandbettertherolesanddynamicsoftheintermediatelayers.Thishasdirectconsequencesonthedesignofsuchmodelsanditenablestheexperttobeabletojustifycertainheuristics(suchastheauxiliaryheadsintheInceptionmodel).Ourmethoduseslinearclassifiers,referre

3、dtoas“probes”,whereaprobecanonlyusethehiddenunitsofagivenintermediatelayerasdiscriminatingfeatures.Moreover,theseprobescannotaffectthetrainingphaseofamodel,andtheyaregenerallyaddedaftertraining.Theyallowtheusertovisualizethestateofthemodelatmultiplestepsoftraining.W

4、edemonstratehowthiscanbeusedtodevelopabetterintuitionaboutaknownmodelandtodiagnosepotentialproblems.1IntroductionTherecenthistoryofdeepneuralnetworksfeaturesanimpressivenumberofnewmethodsandtechnologicalimprovementstoallowthetrainingofdeeperandmorepowerfulnetworks.T

5、hemodelthemselveshadareputationforbeingblackboxes,andtheystillhavethatreputation.Neuralnetworksarecriticizedfortheirlackofinterpretability,whichisatradeoffthatweacceptbecauseoftheiramazingperformanceonmanytasks.Effortshavebeenmadetoidentifytheroleplayedbyeachlayer,b

6、utitcanbehardtofindameaningtoindividuallayers.Therearegoodargumentstosupporttheclaimthatthefirstlayersofaconvolutionnetworkforimagerecognitioncontainfiltersthatarerelatively“general”,inthesensethattheywouldworkgreatevenifweswitchedtoanentirelydifferentdatasetofimages.T

7、helastlayersarespecifictothedatasetbeingarXiv:1610.01644v1[stat.ML]5Oct2016used,andhastoberetrainedwhenusingadifferentdataset.InYosinskietal.(2014)theauthorstrytopinpointtheatwhichthistransitionoccurs,buttheyshowthattheexacttransitionisspreadacrossmultiplelayers.Inth

8、ispaper,weintroducetheconceptofthelinearclassifierprobe,referredtoasa“probe”forshortwhenthecontextisclear.WestartfromtheconceptofShanonentr