深度学习在NLP结构化模型中的应用ppt课件.pptx

《深度学习在NLP结构化模型中的应用ppt课件.pptx》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、DeepLearningforStructurePredictioninNaturalLanguageProcessingWenzhePei(裴文哲)PekingUniversityOutlineIntroduction&&MotivationDeepLearningforChineseWordSegmentationDeepLearningforGraph-basedDependencyParsingConclusion&FutureWorkIntroduction&&MotivationIntroduction&&MotivationStructureinNLPInt

2、roduction&&MotivationStructurePredictionKeyComponentsModelforfDecodingIntroduction&&Motivation–…ConventionalModelsforfCRFStructurePerceptronStructureSVM–…DecodingAlgorithmViterbiCKYEisnerIntroduction&&MotivationConventionalModelsforfCRFStructurePerceptronStructureSVM–…DecodingAlgorithmVit

3、erbiCKYEisnerThisishowmostresearcherspublishpapers–…Introduction&&MotivationConventionalModelsLinear(Log-Linear)Feature-basedProsIncorporateknowledgeintostatisticalmodelsEasytoexplainConsToomanyfeaturesmakethemodeltendtooverfitFeaturedesignsometimesrequiresdomainknowledgeIntroduction&&Mo

4、tivationHowaboutDeepLearning?NaïveBayes—>HMMMaximumEntropy—>CRFPerceptron—>StructurePerceptronSVM—>StructureSVMNeuralNetwork—>StructureNeuralNetwork?Inourwork,westudiedtwotasksChineseWordSegmentation(SequenceStructure)DependencyParsing(TreeStructure)DeepLearningforChineseWordSegmentationD

5、eepLearningforCWSChineseWordSegmentation–AtypicalsequencelabelingtaskSSBME我爱天安门DeepLearningforCWSConventionalNeuralNetworkModelsDeepLearningforCWSModelTraining–Mairgupetal.(2013):MLE-styletraining–Zhengetal.(2013):Perceptron-styletrainingDeepLearningforCWSPros:Minimizetheeffortinfeatureen

6、gineeringCons:Hardtocapturecomplex“interactions”betweentagsandcontextsrelyingonlyonhiddenlayers–“Interactions”infeature-basedmodel:DeepLearningforCWSOurwork:Max-MarginTensorNeuralNetworkAmodelthatcan:MinimizetheeffortoffeatureengineeringCapturemoreinteractionsbetweentagsandcontextBeeasily

7、generalizedtoothersequencemodelingtasksMax-MarginTensorNeuralNetworkArchitectureofourmodelTagEmbeddingTensor-basedtransformationTensorFactorizationMax-MarginTensorNeuralNetworkTagEmbeddingMax-MarginTensorNeuralNetworkTensor-basedTransformation–Weusea3-waytensor?,1:?