5849-semi-supervised-convolutional-neural-networks-for-text-categorization-via-region-embedding

《5849-semi-supervised-convolutional-neural-networks-for-text-categorization-via-region-embedding》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、Semi-supervisedConvolutionalNeuralNetworksforTextCategorizationviaRegionEmbeddingRieJohnsonTongZhangRJResearchConsultingBaiduInc.,Beijing,ChinaTarrytown,NY,USARutgersUniversity,Piscataway,NJ,USAriejohnson@gmail.comtzhang@stat.rutgers.eduAbstractThispaperpresentsan

2、ewsemi-supervisedframeworkwithconvolutionalneuralnetworks(CNNs)fortextcategorization.Unlikethepreviousapproachesthatrelyonwordembeddings,ourmethodlearnsembeddingsofsmalltextregionsfromunlabeleddataforintegrationintoasupervisedCNN.Theproposedschemeforembeddinglearni

3、ngisbasedontheideaoftwo-viewsemi-supervisedlearning,whichisintendedtobeusefulforthetaskofinteresteventhoughthetrainingisdoneonunlabeleddata.Ourmodelsachievebetterresultsthanpreviousap-proachesonsentimentclassificationandtopicclassificationtasks.1IntroductionConvoluti

4、onalneuralnetworks(CNNs)[15]areneuralnetworksthatcanmakeuseoftheinternalstructureofdatasuchasthe2Dstructureofimagedatathroughconvolutionlayers,whereeachcomputationunitrespondstoasmallregionofinputdata(e.g.,asmallsquareofalargeimage).Ontext,CNNhasbeengainingattentio

5、n,usedinsystemsfortagging,entitysearch,sentencemodeling,andsoon[4,5,26,7,21,12,25,22,24,13],tomakeuseofthe1Dstructure(wordorder)oftextdata.SinceCNNwasoriginallydevelopedforimagedata,whichisfixed-sized,low-dimensionalanddense,withoutmodificationitcannotbeappliedtotext

6、documents,whicharevariable-sized,high-dimensionalandsparseifrepresentedbysequencesofone-hotvectors.InmanyoftheCNNstudiesontext,therefore,wordsinsentencesarefirstconvertedtolow-dimensionalwordvectors.Thewordvectorsareoftenobtainedbysomeothermethodfromanadditionallarg

7、ecorpus,whichistypicallydoneinafashionsimilartolanguagemodelingthoughtherearemanyvariations[3,4,20,23,6,19].Useofwordvectorsobtainedthiswayisaformofsemi-supervisedlearningandleavesuswiththefollowingquestions.Q1.HoweffectiveisCNNontextinapurelysupervisedsettingwitho

8、uttheaidofunlabeleddata?Q2.CanweuseunlabeleddatawithCNNmoreeffectivelythanusinggeneralwordvectorlearningmethods?Ourrecentstudy[11]addressedQ1onte