资源描述：
《深度学习—训练集、验证集和测试集概念.doc》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。
1、ØTraining,ValidationandTestDataExample:(A)Wehavedataon16dataitems,theirattributesandclasslabels.RANDOMLYdividetheminto8fortraining,4forvalidationand4fortesting.TrainingItemNo.d–AttributesClass1.02.03.KNOWNFORALL14.15.DATAITEMS16.17.08.0Validation9.010.011.112.0Test13
2、.014.015.116.1(B).Next,supposewedevelop,threeclassificationmodelsA,B,Cfromthetrainingdata.Letthetrainingerrorsonthesemodelsbeasshownbelow(recallthatthemodelsdonotnecessarilyprovideperfectresultsontrainingdata—neithertheyarerequiredto).ClassificationresultsfromItemNo.
3、d-AttributesTrueClassModelAModelBModelC1.00112.ALLKNOWN00003.10104.11015.10006.11117.00008.0000ClassificationError2/83/83/8(C).Next,usethethreemodelsA,B,Ctoclassifyeachiteminthevalidationsetbasedonitsattributevales.Recallthatwedoknowtheirtruelabelsaswell.Supposeweget
4、thefollowingresults:ClassificationresultsfromItemNo.d-AttributesTrueClassModelAModelBModelC9.010010.001011.101012.0010ClassificationError2/42/41/4Ifweuseminimumvalidationerrorasmodelselectioncriterion,wewouldselectmodelC.(D).NowusemodelCtodetermineclassvaluesforeachd
5、atapointinthetestset.Wedosobysubstitutingthe(known)attributevalueintotheclassificationmodelC.Again,recallthatweknowthetruelabelofeachofthesedataitemssothatwecancomparethevaluesobtainedfromtheclassificationmodelwiththetruelabelstodetermineclassificationerroronthetests
6、et.Supposewegetthefollowingresults.ClassificationresultsfromItemNo.d-AttributesTrueClassModelC13.0014.ALLKNOWN0015.1016.11ClassificationError1/4(E).Basedontheabove,anestimateofgeneralizationerroris25%.WhatthismeansisthatifweuseModelCtoclassifyfutureitemsforwhichonlyt
7、heattributeswillbeknown,nottheclasslabels,wearelikelytomakeincorrectclassificationsabout25%ofthetime.(F).Asummaryoftheaboveisasfollows:ModelTrainingValidationTestA2550----B37.550----C37.52525ØCrossValidationIfavailabledataarelimited,weemployCrossValidation(CV).Inthis
8、approach,dataarerandomlydividedintoalmostkequalsets.Trainingisdonebasedon(k-1)setsandthek-thsetisusedfortest.Thisprocessisrepeatedk
