资源描述：
《Semi-Supervised Consensus Labeling for Crowdsourcing众包的半监督一致性标记》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、Semi-SupervisedConsensusLabelingforCrowdsourcingWeiTangMatthewLeaseDepartmentofComputerScienceSchoolofInformationTheUniversityofTexasatAustinTheUniversityofTexasatAustinwtang@cs.utexas.eduml@ischool.utexas.eduABSTRACTtasksthataredifficulttoeffectivelyautomatebutcanbeperformedbyremot
2、eworkers.Becauseindividualcrowdworkersoftenexhibithighvari-OnMTurk,“requesters”typicallysubmitmanyannota-anceinannotationaccuracy,weoftenaskmultiplecrowdtionmicro-tasks,andworkerschoosewhichtaskstoperform.workerstolabeleachexampletoinferasingleconsensusRequestersobtainlabelsmoreq
3、uicklyandaffordably,andlabel.Whilesimplemajorityvotecomputesconsensusbyworkersearnafewextrabucks.Unfortunately,accuracyofequallyweightingeachworker’svote,weightedvotingas-individualcrowdworkershasoftenexhibitedhighvariancesignsgreaterweighttomoreaccurateworkers,whereaccu-inpaststu
4、diesduetofactorslikepoordesignorincentivesofracyisestimatedbyinner-annotatoragreement(unsuper-tasks,ineffectiveorunengagedworkers,orannotationtaskvised)and/oragreementwithknownexpertlabels(super-complexity.Twocommonmethodsforqualitycontrolare:vised).Inthispaper,weinvestigatetheann
5、otationcostvs.(a)workerfiltering[6](i.e.identifyingpoorqualityworkersconsensusaccuracybenefitfromincreasingtheamountofandexcludingthem)and(b)aggregatinglabelsfrommulti-expertsupervision.Tomaximizebenefitfromsupervision,pleworkersforagivenexampleinordertoarriveatasingleweproposeasemi
6、-supervisedapproachwhichinfersconsen-“consensus”label.Inthispaper,wefocusontheconsensussuslabelsusingbothlabeledandunlabeledexamples.Weproblem;ourfutureworkwillstudyacombinedapproach.compareoursemi-supervisedapproachwithseveralexistingAccuratelyestimatingconsensuslabelsfromindivi
7、dualunsupervisedandsupervisedbaselines,evaluatingonbothworkerlabelsischallenging.AcommonapproachtothissyntheticdataandAmazonMechanicalTurkdata.ResultsproblemissimpleMajorityVoting(MV)[14,13,16],whichshow(a)averymodestamountofsupervisioncanprovideiseasytouseandcanoftenachieverelat
8、ivelygoodempir-significantbenefit,and(b)co
