【上海交通大学】【神经网络原理与应用】作业2.pdf

《【上海交通大学】【神经网络原理与应用】作业2.pdf》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。

1、NeuralNetworkTheoryandApplicationsHomeworkAssignment2oxstar@SJTUJanuary19,2012Supposetheoutputofeachneuroninamultilayerquadraticperceptron(MLQP)networkis01NXk1x=f@(ux2+vx)+bAkjkjik1;ikjik1;ikji=1fork=2;3;:::;Mandj=1;2;:::;Nkwherebothukjiandvkjiaretheweightsconnectingtheithunitinthe

2、layerk1tothejthunitinthelayerk,bkjisthebiasofthejthunitinthelayerk,Nkisthenumberofunitsinthek(1kM),andf(:)isthesigmoidalactivationfunction.1.Pleasedesignthecorrespondingbackpropagationalgorithm.Ans.Fromthenetwork,wehaveinput=x1(1)x=f(ux2+vx+b);k=2;3;:::;M(2)kkk1kk1koutput=xM(3)Th

3、esteepestdescentalgorithmfortheapproximatemeansquareerroris@F^ukji(m+1)=ukji(m)@ukji@F^vkji(m+1)=vkji(m)@vkji@F^bkj(m+1)=bkj(m)@bkjThenetinputtolayerkisanexplicitfunctionoftheweightsandbiasinthatlayerNXk1n=(ux2+vx)+bkjkjik1;ikjik1;ikji=1Therefore@nkj2@nkj@nkj=xk1;i;=xk1;i;=1@u

4、kji@vkji@bkjLet@F^@F^skj;sk@nkj@nk1Usingthechainruleofcalculus,wehaveu(m+1)=u(m)s(x2)T(4)kkkk1v(m+1)=v(m)s(x)T(5)kkkk1bk(m+1)=bk(m)sk(6)Inordertoobtaintherecurrencerelation,weshouldcalculateP@Nk(ux2+vx)+b@nk+1;ji=1k+1;jikik+1;jikik+1;j=@nki@nki@x2@x=uki+vkik+1;jik+1;ji@nki@nk

5、i@f2(n)@f(n)kiki=uk+1;ji+vk+1;ji@nki@nki=(2uk+1;jixki+vk+1;ji)f_(nki)=(2uk+1;jixki+vk+1;ji)(1xki)xki@nk+1=(2uk+1xk+vk+1)F_(nk)@nkWecannowwriteouttherecurrencerelationforthesensitivitybyusingthechainruleinmatrixform:T@F^@nk+1@F^sk==(7)@nk@nk@nk+1=F_(n)(2ux+v)Ts(8)kk+1kk+1k+1Thestart

6、ingpoint,sM,fortherecurrencerelationhasbeenderived:sM=2F_(nM)(txM)(9)2.Writeaprogramtorealizeit(3layers).Ans.cf.`src/mlqp.m'3.Runyourprogramforpatternclassicationonthetwo-spiraldataset,whichisthesameashomeworkone.Youcanchoose10hiddenunitsinthisproblem.Ans.Wecanrstdiscussaboutthepr

7、ocessofconvergence.Whenlearningrateissetto0.1andinitialvaluesarerandomlychosen.WechooseMSE(MeanSquaredError)topresentthelearningqualityofourNN.However,thenalresultsareclassications,sowealsopresentErrorRate(Figure1).InFigure2,wepresenttheprocessofconvergence.FromFigure1bwecannoticeth

8、atwhe