CS224d-Lecture13_Convolutional neural networks -- for sentence classification

《CS224d-Lecture13_Convolutional neural networks -- for sentence classification》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、CS224d: Deep NLP Lecture 13: Convolu8onal Neural Networks (for NLP) Richard Socher richard@metamind.ioT@1TT1TTTr(A+XCX)Tr(X(ABX+)XTCX=)12(CXXT(BXA+)X=CX)2CXX(BXA+(AXT+CXX)CX1@X@XT1Midterm +2BX(A+XCX+2)BX(A+XTCX(127)See[7].See[7].•Generally good performance 

2、@Tr(sin(X))@Tr(sin(TX))=cos(X)=cos(X)T(128)@X@X2.6Derivativesofvectornorms2.6Derivativesofvectornorms2.6.1Two-norm2.6.1Two-norm@x@axa@x

3、

4、xa

5、

6、2=

7、

8、xa

9、

10、

11、

12、xa

13、

14、2=(129)@x2

15、

16、xa

17、

18、2T@xaI@x(axa)(xIa)(xa)(xa)T==3(130)@xkxak2kx@xakk2xakkxkxak2akkxak32222T•

19、But: @

20、

21、x

22、

23、2@

24、

25、xx

26、

27、2@

28、

29、x

30、

31、2@

32、

33、xTx

34、

35、==2x2=2=2x(131)@x@x@x@xLecture 1, Slide 2 Richard Socher 5/12/15 2.7Derivativesofmatrixnorms2.7DerivativesofmatrixnormsFormoreonmatrixnorms,seeSec.10.4.Formoreonmatrixnorms,seeSec.10.4.2.7.1Frobeniusnorm2.7.1Frobeniusnorm@@2@H@

36、

37、

38、X

39、

40、F=Tr(XX

41、

42、X)=2

43、

44、2=XTr(XXH)=2X(132)@X@X@XF@XSee(248).Notethatthisisalsoaspecialcaseoftheresultinequation119.See(248).Notethatthisisalsoaspecialcaseoftheresultinequation119.2.8DerivativesofStructuredMatrices2.8DerivativesofStructuredMatricesAssumethatthematrixAssu

45、methatthematrixAhassomestructure,i.e.symmetric,toeplitz,etc.Ahassomestructure,i.e.symmetric,toeplitz,etc.Inthatcasethederivativesoftheprevioussectiondoesnotapplyingeneral.Inthatcasethederivativesoftheprevioussectiondoesnotapplyingeneral.Instead,considerthefollowin

46、ggeneralrulefordiInstead,considerthefollowinggeneralrulefordi↵erentiatingascalarfunction↵erentiatingascalarfunctionf(A)f(A)"#"#dfX@f@AX@fT@AT=kldf=Tr@f@Akl@f(133)@A==TrdAijkl@Akl@AdAijij@@AAkl@@AAijij@A@AijklPetersen&Pedersen,TheMatrixCookbook,Version:November

47、15,2012,Petersen&Pedersen,TheMatrixCookbook,Version:November15,2012,Page14Overview of today •From RNNs to CNNs •CNN Variant 1: Simple single layer •ApplicaDon: Sentence classificaDon •More details and tricks •EvaluaDon •Comparison between sentence models: BoV, RNNs

48、2, CNNs •CNN Variant 2: Complex mulD layer Lecture 1, Slide 3 Richard Socher 5/12/15 From RNNs to CNNs  1 5 5.5 6.1 1 2.5 3.5 3.8 0.4 2.1 7 4 2.3 0.3 3.