12Outlier.pptVIP免费

1DataMining:ConceptsandTechniques(3rded.)—Chapter12—JiaweiHan,MichelineKamber,andJianPeiUniversityofIllinoisatUrbana-Champaign&SimonFraserUniversity©2012Han,Kamber&Pei.Allrightsreserved.3Chapter12.OutlierAnalysisOutlierandOutlierAnalysisOutlierDetectionMethodsStatisticalApproachesProximity-BaseApproachesClustering-BaseApproachesClassificationApproachesMiningContextualandCollectiveOutliersOutlierDetectioninHighDimensionalDataSummary4WhatAreOutliers?Outlier:AdataobjectthatdeviatessignificantlyfromthenormalobjectsasifitweregeneratedbyadifferentmechanismEx.:Unusualcreditcardpurchase,sports:MichaelJordon,WayneGretzky,...OutliersaredifferentfromthenoisedataNoiseisrandomerrororvarianceinameasuredvariableNoiseshouldberemovedbeforeoutlierdetectionOutliersareinteresting:ItviolatesthemechanismthatgeneratesthenormaldataOutlierdetectionvs.noveltydetection:earlystage,outlier;butlatermergedintothemodelApplications:CreditcardfrauddetectionTelecomfrauddetectionCustomersegmentationMedicalanalysis5TypesofOutliers(I)Threekinds:global,contextualandcollectiveoutliersGlobaloutlier(orpointanomaly)ObjectisOgifitsignificantlydeviatesfromtherestofthedatasetEx.IntrusiondetectionincomputernetworksIssue:FindanappropriatemeasurementofdeviationContextualoutlier(orconditionaloutlier)ObjectisOcifitdeviatessignificantlybasedonaselectedcontextEx.80oFinUrbana:outlier?(dependingonsummerorwinter?)AttributesofdataobjectsshouldbedividedintotwogroupsContextualattributes:definesthecontext,e.g.,time&locationBehavioralattributes:characteristicsoftheobject,usedinoutlierevaluation,e.g.,temperatureCanbeviewedasageneralizationoflocaloutliers—whosedensitysignificantlydeviatesfromitslocalareaIssue:Howtodefineorformulatemeaningfulcontext?GlobalOutlier6TypesofOutliers(II)CollectiveOutliersAsubsetofdataobjectscollectivelydeviatesignificantlyfromthewholedataset,eveniftheindividualdataobjectsmaynotbeoutliersApplications:E.g.,intrusiondetecti...