lecture5缺失值处理策略

《lecture5缺失值处理策略》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、1Lecture5IncompletedataZiadTaibBiostatistics,AZMay3,2011OutlineoftheproblemMissingvaluesinlongitudinaltrialsisabigissueFirstaimshouldbetoreduceproportionEthicsdictatethatitcan’tbeavoidedThereisnomagicmethodtofixitMagnitudeofproblemvariesacrossareas8-weekdepressiontrial:25%−50%maydropoutbyfinalv

2、isit12-weekasthmatrial:maybeonly5%−10%2DateName,department3OutlineofthelecturePartI:MissingdataPartII:MultipleimputationExample:Theanalgesictrial45DateName,department6PartI:MissingdataInrealdatasets,like,e.g.,surveysandclinicaltrials,itisquitecommontohaveobservationswithmissingvaluesforoneormore

3、inputfeatures.Thefirstissueindealingwiththeproblemisdeterminingwhetherthemissingdatamechanismhasdistortedtheobserveddata.LittleandRubin(1987)andRubin(1987)distinguishbetweenbasicallythreemissingdatamechanisms.Dataaresaidtobemissingatrandom(MAR)ifthemechanismresultinginitsomissionisindependentofi

4、ts(unobserved)value.Ifitsomissionisalsoindependentoftheobservedvalues,thenthemissingnessprocessissaidtobemissingcompletelyatrandom(MCAR).Inanyothercasetheprocessismissingnotatrandom(MNAR),i.e.,themissingnessprocessdependsontheunobservedvalues.http://www.emea.europa.eu/pdfs/human/ewp/177699EN.pdf

5、1.Introductiontomissingdata??????VariablesCases??=missing7Whatismissingdata?Themissingnesshidesarealvaluethatisusefulforanalysispurposes.Surveyquestions:WhatisyourtotalannualincomeforFY2008?Whoareyouvotingforinthe2009electionfortheEuropeanparlament?8Whatismissingdata?Clinicaltrials:StartFinishce

6、nsoredatthispointintimetime9MissingnessItmatterswhydataaremissing.Supposeyouaremodellingweight(Y)asafunctionofsex(X).Somerespondentswouldn'tdisclosetheirweight,soyouaremissingsomevaluesforY.Therearethreepossiblemechanismsforthenondisclosure:Theremaybenoparticularreasonwhysomerespondentstoldyouth

7、eirweightsandothersdidn't.Thatis,theprobabilitythatYismissingmayhasnorelationshiptoXorY.InthiscaseourdataismissingcompletelyatrandomOnesexmaybelesslikelytodiscloseitsweight.Thatis,theprobabilitythatYismissingdependsonlyonthe