Temporal Autoencoding Improves Generative Models of.pdf

《Temporal Autoencoding Improves Generative Models of.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、JMLR:WorkshopandConferenceProceedings0:1–14,00TemporalAutoencodingImprovesGenerativeModelsofTimeSeriesChrisH¨ausler∗,MartinP.Nawrotchausler@gmail.com,martin.nawrot@fu-berlin.deNeuroinformatics,FreieUniversit¨atBerlinAlexSusemihl∗,ManfredOpperalexsusemihl@gmail.com

2、,opperm@cs.tu-berlin.deArtificialIntelligence,TechnischeUniversit¨atBerlin∗Theseauthorshavecontributedequallytothiswork.AbstractRestrictedBoltzmannMachines(RBMs)aregenerativemodelswhichcanlearnusefulrep-resentationsfromsamplesofadatasetinanunsupervisedfashion.Theyh

3、avebeenwidelyemployedasanunsupervisedpre-trainingmethodinmachinelearning.RBMshavebeenmodifiedtomodeltimeseriesintwomainways:TheTemporalRBMstacksanumberofRBMslaterallyandintroducestemporaldependenciesbetweenthehiddenlayerunits;TheConditionalRBM,ontheotherhand,consid

4、erspastsamplesofthedatasetasacon-ditionalbiasandlearnsarepresentationwhichtakestheseintoaccount.HereweproposeanewtrainingmethodforboththeTRBMandtheCRBM,whichenforcesthedynamicstructureoftemporaldatasets.Wedosobytreatingthetemporalmodelsasdenoisingautoencoders,cons

5、ideringpastframesofthedatasetascorruptedversionsofthepresentframeandminimizingthereconstructionerrorofthepresentdatabythemodel.WecallthisapproachTemporalAutoencoding.Thisleadstoasignificantimprovementintheper-formanceofbothmodelsinafilling-in-framestaskacrossanumber

6、ofdatasets.Theerrorreductionformotioncapturedatais56%fortheCRBMand80%fortheTRBM.Takingtheposteriormeanpredictioninsteadofsinglesamplesfurtherimprovesthemodel’ses-timates,decreasingtheerrorbyasmuchas91%fortheCRBMonmotioncapturedata.Wealsotrainedthemodeltoperformfor

7、ecastingonalargenumberofdatasetsandhavearXiv:1309.3103v1[stat.ML]12Sep2013foundTApretrainingtoconsistentlyimprovetheperformanceoftheforecasts.Further-more,bylookingatthepredictionerroracrosstime,wecanseethatthisimprovementreflectsabetterrepresentationofthedynamicso

8、fthedataasopposedtoabiastowardsreconstructingtheobserveddataonashorttimescale.Webelievethisnovelapproachofmixingcontrastivedivergenceandautoencodertrain