0726第16篇论文.pptxVIP免费

贪心科技|让每个人享受个性化教育服务Review:PaperReadingDropoutasaBayesianApproximation:RepresentingModelUncertaintyinDeepLearning范老师2020/07/26贪心科技|让每个人享受个性化教育服务Content•Introduction•RelatedResearch•DropoutasaBayesianApproximation•ObtainingModelUncertainty•Experiments•ConclusionandFutureResearch贪心科技|让每个人享受个性化教育服务IntroductionInthispaperwegiveacompletetheoreticaltreatmentofthelinkbetweenGaussianprocessesanddropout,anddevelopthetoolsnecessarytorepresentuncertaintyindeeplearning.ExploratoryAssessmentonregression,classificationandreinforcementlearning.贪心科技|让每个人享受个性化教育服务RelatedResearch1.FiniteNNswithdistributionsplacedovertheweightshavebeenstudiedextensivelyasBayesianneuralnetworks.Theseofferrobustnesstoover-fittingaswell,butwithchallenginginferenceandadditionalcomputationalcosts.2.Recentadvancesinvariationalinferenceintroducednewtechniquesintothefieldsuchassampling-basedvariationalinferenceandstochasticvariationalinference.ThesehavebeenusedtoobtainnewapproximationsforBayesianneuralnetworksthatperformaswellasdropout.Howeverthesemodelscomewithaprohibitivecomputationalcost.3.AnalternativeapproachtovariationalinferencemakesuseofexpectationpropagationandhasimprovedconsiderablyinRMSEanduncertaintyestimationonVIapproachessuchas(Graves,2011).贪心科技|让每个人享受个性化教育服务DropoutasaBayesianApproximationBayesianinterpretationofdropout:贪心科技|让每个人享受个性化教育服务ObtainingModelUncertainty贪心科技|让每个人享受个性化教育服务Experiments–ModelUncertaintyinRegressionTasksTheuncertaintyisincreasingfarfromthedatafortheReLUmodel,whereasfortheTanHmodelitstaysbounded.贪心科技|让每个人享受个性化教育服务Experiments-ModelUncertaintyinClassificationTasksIftheuncertaintyenvelopeofaclassisfarfromthatofotherclasses’(forexampletheleftmostimage)thentheinputisclassifiedwithhighconfidence.Ontheotherhand,iftheuncertaintyenvelopeintersectsthatofotherclasses(suchasinthecaseofthemiddleinputimage),theneventhoughthesoftmaxoutputcan...