Dependency-based Convolutional Neural Networks for Sentence Embedding

《Dependency-based Convolutional Neural Networks for Sentence Embedding》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、Dependency-basedConvolutionalNeuralNetworksforSentenceEmbeddingMingboMayLiangHuangyzBingXiangzBowenZhouzyGraduateCenter&QueensCollegezIBMWatsonGroupCityUniversityofNewYorkT.J.WatsonResearchCenter,IBMfmma2,lhuangggc.cuny.eduflhuang,bingxia,zhoug@us.ibm.comAbstractIndeed,inthesentimentanalysisliter

2、ature,re-searchershaveincorporatedlong-distanceinfor-Insentencemodelingandclassification,mationfromsyntacticparsetrees,buttheresultsconvolutionalneuralnetworkapproachesaresomewhatinconsistent:somereportedsmallhaverecentlyachievedstate-of-the-artre-improvements(Gamon,2004;Matsumotoetal.,sults,butall

3、sucheffortsprocesswordvec-2005),whilesomeotherwise(Daveetal.,2003;torssequentiallyandneglectlong-distanceKudoandMatsumoto,2004).Asaresult,syn-dependencies.Tocombinedeeplearn-tacticfeatureshaveyettobecomepopularintheingwithlinguisticstructures,wepro-sentimentanalysiscommunity.Wesuspectoneposeadepen

4、dency-basedconvolutionap-ofthereasonsforthisisdatasparsity(accordingproach,makinguseoftree-basedn-gramstoourexperiments,treen-gramsaresignificantlyratherthansurfaceones,thusutlizingnon-sparserthansurfacen-grams),butthisproblemlocalinteractionsbetweenwords.Ourhaslargelybeenalleviatedbytherecentadvan

5、cesmodelimprovessequentialbaselinesonallinwordembedding.Canwecombinetheadvan-foursentimentandquestionclassificationtagesofbothworlds?tasks,andachievesthehighestpublishedSoweproposeaverysimpledependency-basedaccuracyonTREC.convolutionalneuralnetworks(DCNNs).OurmodelissimilartoKim(2014),butwhilehisse

6、-1IntroductionquentialCNNsputawordinitssequentialcon-Convolutionalneuralnetworks(CNNs),originallytext,oursconsidersawordanditsparent,grand-inventedincomputervision(LeCunetal.,1995),parent,great-grand-parent,andsiblingsonthede-hasrecentlyattractedmuchattentioninnaturalpendencytree.Thiswayweincorpor

7、atelong-languageprocessing(NLP)onproblemssuchasdistanceinformationthatareotherwiseunavail-sequencelabeling(Collobertetal.,2011),seman-ableonthesurfacestring.ticparsing(Yihetal.,2014),andsearchqueryExperimentsonth