Introduction to MathematicalStatistics
EighthEdition
GlobalEdition
RobertV.Hogg UniversityofIowa
JosephW.McKean WesternMichiganUniversity
AllenT.Craig LateProfessorofStatistics UniversityofIowa
Director,PortfolioManagement:DeirdreLynch
CoursewarePortfolioManager:PatrickBarbera
PortfolioManagementAssistant:Morgan
Danna
AssociateEditor,GlobalEdition:Shaoni
Mukherjee
ContentProducer:LaurenMorse
ManagingProducer:ScottDisanno
ProductMarketingManager:YvonneVannatta
FieldMarketingManager:EvanSt.Cyr
MarketingAssistant:JonBryant
SeniorAuthorSupport/TechnologySpecialist: JoeVetere
PearsonEducationLimited
KAOTwo
KAOPark
HockhamWay
Harlow
Essex
CM179SR
UnitedKingdom
andAssociatedCompaniesthroughouttheworld
ManagingEditor,DigitalMedia,Global Edition:GargiBanerjee Manager,RightsandPermissions:Gina Cheselka ManufacturingBuyer:CarolMelville, LSCCommunications ManufacturingBuyer,GlobalEdition: KayHolman
ArtDirector:BarbaraAtkinson ProductionCoordinationandIllustrations: Integra CoverDesign:LuminaDatamatics CoverImage:AnnaMente/Shutterstock
VisitusontheWorldWideWebat: www.pearsonglobaleditions.com
© PearsonEducationLimited2020
TherightsofRobertV.Hogg,JosephW.McKean,andAllenT.Craigtobeidentifiedasthe authorsofthisworkhavebeenassertedbytheminaccordancewiththeCopyright,Designsand PatentsAct1988.
AuthorizedadaptationfromtheUnitedStatesedition,entitled IntroductiontoMathematical Statistics, 8thEdition,ISBN978-0-13-468699-8,byRobertV.Hogg,JosephW.McKean,and AllenT.Craig,publishedbyPearsonEducation © 2019.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem, ortransmittedinanyformorbyanymeans,electronic,mechanical,photocopying,recordingor otherwise,withouteitherthepriorwrittenpermissionofthepublisheroralicensepermitting restrictedcopyingintheUnitedKingdomissuedbytheCopyrightLicensingAgencyLtd,SaffronHouse,6–10KirbyStreet,LondonEC1N8TS.Forinformationregardingpermissions,request formsandtheappropriatecontactswithinthePearsonEducationGlobalRights&Permissions department,pleasevisitwww.pearsoned.com/permissions/.
Alltrademarksusedhereinarethepropertyoftheirrespectiveowners.Theuseofanytrademark inthistextdoesnotvestintheauthororpublisheranytrademarkownershiprightsinsuchtrademarks,nordoestheuseofsuchtrademarksimplyanyaffiliationwithorendorsementofthisbook bysuchowners.
PEARSONandALWAYSLEARNINGareexclusivetrademarksownedbyPearsonEducation, Inc.oritsaffiliatesintheU.S.and/orothercountries.Unlessotherwiseindicatedherein,any third-partytrademarksthatmayappearinthisworkarethepropertyoftheirrespectiveowners andanyreferencestothird-partytrademarks,logosorothertradedressarefordemonstrativeor descriptivepurposesonly.Suchreferencesarenotintendedtoimplyanysponsorship,endorsement,authorization,orpromotionofPearson’sproductsbytheownersofsuchmarks,orany relationshipbetweentheownerandPearsonEducation,Inc.oritsaffiliates,authors,licenseesor distributors.
ThiseBookisastandaloneproductandmayormaynotincludeallassetsthatwerepartofthe printversion.ItalsodoesnotprovideaccesstootherPearsondigitalproductslikeMyLaband Mastering.ThepublisherreservestherighttoremoveanymaterialinthiseBookatanytime.
ISBN10:1-292-26476-4
ISBN13:978-1-292-26476-9 eBookISBN13:978-1-292-26478-3
BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary TypesetbyIntegraSoftwareServicesPvt.Ltd.
DedicatedtomywifeMarge andtothememoryofBobHogg
This page intentionally left blank
1ProbabilityandDistributions 15 1.1Introduction ................................15 1.2Sets....................................17
1.2.1ReviewofSetTheory ......................18
1.2.2SetFunctions ...........................21
1.3TheProbabilitySetFunction ......................26
1.3.1CountingRules ..........................30
1.3.2AdditionalPropertiesofProbability ..............32
1.4ConditionalProbabilityandIndependence ...............37
1.4.1Independence ...........................42
1.4.2Simulations ............................45
1.5RandomVariables............................51
1.6DiscreteRandomVariables.......................59
1.6.1Transformations... ......................61
1.7ContinuousRandomVariables.....................63
1.7.1Quantiles .............................65
1.7.2Transformations... ......................67
1.7.3MixturesofDiscreteandContinuousTypeDistributions...70
1.8ExpectationofaRandomVariable ...................74
1.8.1RComputationforanEstimationoftheExpectedGain...79
1.9SomeSpecialExpectations. ......................82
1.10ImportantInequalities ..........................92
2MultivariateDistributions 99
2.1DistributionsofTwoRandomVariables ................99
2.1.1MarginalDistributions ......................103
2.1.2Expectation ............................107
2.2Transformations:BivariateRandomVariables... ..........114
2.3ConditionalDistributionsandExpectations ..............123
2.4IndependentRandomVariables .....................131
2.5TheCorrelationCoefficient. ......................139
2.6ExtensiontoSeveralRandomVariables................148
2.6.1 ∗ MultivariateVariance-CovarianceMatrix. ..........154
2.7TransformationsforSeveralRandomVariables.. ..........157
2.8LinearCombinationsofRandomVariables ...............165
3SomeSpecialDistributions 169
3.1TheBinomialandRelatedDistributions ................169
3.1.1NegativeBinomialandGeometricDistributions ........173
3.1.2MultinomialDistribution ....................174
3.1.3HypergeometricDistribution ..................176
3.2ThePoissonDistribution.. ......................181
3.3TheΓ, χ2 ,and β Distributions .....................187
3.3.1The χ2 -Distribution. ......................192
3.3.2The β -Distribution.. ......................194
3.4TheNormalDistribution... ......................200
3.4.1 ∗ ContaminatedNormals .....................207
3.5TheMultivariateNormalDistribution .................212
3.5.1BivariateNormalDistribution ..................212
3.5.2 ∗ MultivariateNormalDistribution,GeneralCase .......213
3.5.3 ∗ Applications ...........................220
3.6 t-and F -Distributions ..........................224
3.6.1The t-distribution.. ......................224
3.6.2The F -distribution.. ......................226
3.6.3Student’sTheorem.. ......................228
3.7 ∗ MixtureDistributions ..........................232
4SomeElementaryStatisticalInferences 239
4.1SamplingandStatistics... ......................239
4.1.1PointEstimators... ......................240
4.1.2HistogramEstimatesofpmfsandpdfs... ..........244
4.2ConfidenceIntervals ...........................252
4.2.1ConfidenceIntervalsforDifferenceinMeans ..........255
4.2.2ConfidenceIntervalforDifferenceinProportions .......257
4.3 ∗ ConfidenceIntervalsforParametersofDiscreteDistributions ....262
4.4OrderStatistics ..............................267
4.4.1Quantiles .............................271
4.4.2ConfidenceIntervalsforQuantiles ...............275
4.5IntroductiontoHypothesisTesting ...................281
4.6AdditionalCommentsAboutStatisticalTests... ..........289
4.6.1ObservedSignificanceLevel, p-value..............293
4.7Chi-SquareTests.............................297
4.8TheMethodofMonteCarlo.......................306
4.8.1Accept–RejectGenerationAlgorithm ..............312
4.9BootstrapProcedures ..........................317
4.9.1PercentileBootstrapConfidenceIntervals. ..........317
4.9.2BootstrapTestingProcedures ..................322
4.10 ∗ ToleranceLimitsforDistributions ...................329
5ConsistencyandLimitingDistributions 335
5.1ConvergenceinProbability. ......................335
5.1.1SamplingandStatistics .....................338
5.2ConvergenceinDistribution. ......................341
5.2.1BoundedinProbability .....................347
5.2.2Δ-Method .............................348
5.2.3MomentGeneratingFunctionTechnique.. ..........350
5.3CentralLimitTheorem... ......................355
5.4 ∗ ExtensionstoMultivariateDistributions ...............362
6MaximumLikelihoodMethods 369
6.1MaximumLikelihoodEstimation ....................369
6.2Rao–Cram´erLowerBoundandEfficiency ...............376
6.3MaximumLikelihoodTests.......................390
6.4MultiparameterCase:Estimation ....................400
6.5MultiparameterCase:Testing ......................409
6.6TheEMAlgorithm ............................418
7Sufficiency 427
7.1MeasuresofQualityofEstimators ...................427
7.2ASufficientStatisticforaParameter ..................433
7.3PropertiesofaSufficientStatistic ....................440
7.4CompletenessandUniqueness ......................444
7.5TheExponentialClassofDistributions .................449
7.6FunctionsofaParameter.. ......................454
7.6.1BootstrapStandardErrors ...................458
7.7TheCaseofSeveralParameters.....................461
7.8MinimalSufficiencyandAncillaryStatistics ..............468
7.9Sufficiency,Completeness,andIndependence... ..........475
8OptimalTestsofHypotheses 483
8.1MostPowerfulTests...........................483
8.2UniformlyMostPowerfulTests.....................493
8.3LikelihoodRatioTests ..........................501
8.3.1LikelihoodRatioTestsforTestingMeansofNormalDistributions ..............................502
8.3.2LikelihoodRatioTestsforTestingVariancesofNormalDistributions .............................509
8.4 ∗ TheSequentialProbabilityRatioTest .................514
8.5 ∗ MinimaxandClassificationProcedures ................521
8.5.1MinimaxProcedures. ......................521
8.5.2Classification ...........................524
9InferencesAboutNormalLinearModels 529
9.1Introduction ................................529
9.2One-WayANOVA ............................530
9.3Noncentral χ2 and F -Distributions ...................536
9.4MultipleComparisons ..........................539
9.5Two-WayANOVA............................545
9.5.1InteractionbetweenFactors ...................548
9.6ARegressionProblem..........................553
9.6.1MaximumLikelihoodEstimates .................554
9.6.2 ∗ GeometryoftheLeastSquaresFit..............560
9.7ATestofIndependence... ......................565
9.8TheDistributionsofCertainQuadraticForms... ..........569
9.9TheIndependenceofCertainQuadraticForms.. ..........576
10NonparametricandRobustStatistics 583
10.1LocationModels .............................583
10.2SampleMedianandtheSignTest ....................586
10.2.1AsymptoticRelativeEfficiency .................591
10.2.2EstimatingEquationsBasedontheSignTest .........596
10.2.3ConfidenceIntervalfortheMedian ...............598
10.3Signed-RankWilcoxon ..........................600
10.3.1AsymptoticRelativeEfficiency .................605
10.3.2EstimatingEquationsBasedonSigned-RankWilcoxon...607
10.3.3ConfidenceIntervalfortheMedian ...............608
10.3.4MonteCarloInvestigation ....................609
10.4Mann–Whitney–WilcoxonProcedure ..................612
10.4.1AsymptoticRelativeEfficiency .................616
10.4.2EstimatingEquationsBasedontheMann–Whitney–Wilcoxon618
10.4.3ConfidenceIntervalfortheShiftParameterΔ .........618
10.4.4MonteCarloInvestigationofPower ..............619
10.5 ∗ GeneralRankScores ..........................621
10.5.1Efficacy ..............................624
10.5.2EstimatingEquationsBasedonGeneralScores ........626
10.5.3Optimization:BestEstimates ..................626
10.6 ∗ AdaptiveProcedures ..........................633
10.7SimpleLinearModel ...........................639
10.8MeasuresofAssociation... ......................645
10.8.1Kendall’s τ ............................645
10.8.2Spearman’sRho... ......................648
10.9RobustConcepts .............................652
10.9.1LocationModel ..........................652
10.9.2LinearModel ...........................659
11BayesianStatistics
11.1BayesianProcedures ...........................669
11.1.1PriorandPosteriorDistributions
11.1.2BayesianPointEstimation
11.1.3BayesianIntervalEstimation ..................676
11.1.4BayesianTestingProcedures
11.1.5BayesianSequentialProcedures
11.2MoreBayesianTerminologyandIdeas
11.3GibbsSampler
11.4ModernBayesianMethods..
11.4.1EmpiricalBayes...
B.2ProbabilityDistributions...
B.3RFunctions
B.4Loops...................................713
B.5InputandOutput
This page intentionally left blank
Preface
Wehavemadesubstantialchangesinthiseditionof IntroductiontoMathematical Statistics.Someofthesechangeshelpstudentsappreciatetheconnectionbetween statisticaltheoryandstatisticalpracticewhileotherchangesenhancethedevelopmentanddiscussionofthestatisticaltheorypresentedinthisbook.
Manyofthechangesinthiseditionreflectcommentsmadebyourreaders.One ofthesecommentsconcernedthesmallnumberofrealdatasetsintheprevious editions.Inthisedition,wehaveincludedmorerealdatasets,usingthemto illustratestatisticalmethodsortocomparemethods.Further,wehavemadethese datasetsaccessibletostudentsbyincludingtheminthefreeRpackage hmcpkg TheycanalsobeindividuallydownloadedinanRsessionattheurllistedonpage12. Ingeneral,theRcodefortheanalysesonthesedatasetsisgiveninthetext.
WehavealsoexpandedtheuseofthestatisticalsoftwareR.WeselectedR becauseitisapowerfulstatisticallanguagethatisfreeandrunsonallthreemain platforms(Windows,Mac,andLinux).Instructors,though,canselectanother statisticalpackage.WehavealsoexpandedouruseofRfunctionstocompute analysesandsimulationstudies,includingseveralgames.Wehavekeptthelevelof codingforthesefunctionsstraightforward.Ourgoalistoshowstudentsthatwith afewsimplelinesofcodetheycanperformsignificantcomputations.AppendixB containsabriefRprimer,whichsufficesfortheunderstandingoftheRusedinthe text.Aswiththedatasets,theseRfunctionscanbesourcedindividuallyatthe citedurl;however,theyarealsoincludedinthepackage hmcpkg.
WehavesupplementedthemathematicalreviewmaterialinAppendixA,placing itinthedocument MathematicalPrimerforIntroductiontoMathematicalStatistics Itisfreelyavailableforstudentstodownloadatthelistedurl.Besidessequences, thissupplementreviewsthetopicsofinfiniteseries,differentiation,andintegration(univariateandbivariate).Wehavealsoexpandedthediscussionofiterated integralsinthetext.Wehaveaddedfigurestoclarifydiscussion.
Wehaveretainedtheorderofelementarystatisticalinferences(Chapter4)and asymptotictheory(Chapter5).InChapters5and6,wehavewrittenbriefreviews ofthematerialinChapter4,sothatChapters4and5areessentiallyindependent ofoneanotherand,hence,canbeinterchanged.InChapter3,wenowbeginthe sectiononthemultivariatenormaldistributionwithasubsectiononthebivariate normaldistribution.Severalimportanttopicshavebeenadded.Thisincludes Tukey’smultiplecomparisonprocedureinChapter9andconfidenceintervalsfor thecorrelationcoefficientsfoundinChapters9and10.Chapter7nowcontainsa
discussiononstandarderrorsforestimatesobtainedbybootstrappingthesample. SeveraltopicsthatwerediscussedintheExercisesarenowdiscussedinthetext. Examplesincludequantiles,Section1.7.1,andhazardfunctions,Section3.3.In general,wehavemademoreuseofsubsectionstobreakupsomeofthediscussion. Also,severalmoresectionsarenowindicatedby ∗ asbeingoptional.
ContentandCoursePlanning
Chapters1and2developprobabilitymodelsforunivariateandmultivariatevariableswhileChapter3discussesmanyofthemostwidelyusedprobabilitymodels. Chapter4discussesstatisticaltheoryformuchoftheinferencefoundinastandardstatisticalmethodscourse.Chapter5presentsasymptotictheory,concluding withtheCentralLimitTheorem.Chapter6providesacompleteinference(estimationandtesting)basedonmaximumlikelihoodtheory.TheEMalgorithmis alsodiscussed.Chapters7–8containoptimalestimationproceduresandtestsof statisticalhypotheses.Thefinalthreechaptersprovidetheoryforthreeimportant topicsinstatistics.Chapter9containsinferencefornormaltheorymethodsfor basicanalysisofvariance,univariateregression,andcorrelationmodels.Chapter 10presentsnonparametricmethods(estimationandtesting)forlocationandunivariateregressionmodels.Italsoincludesdiscussionontherobustconceptsof efficiency,influence,andbreakdown.Chapter11offersanintroductiontoBayesian methods.ThisincludestraditionalBayesianproceduresaswellasMarkovChain MonteCarlotechniques.
Severalcoursescanbedesignedusingourbook.Thebasictwo-semestercourse inmathematicalstatisticscoversmostofthematerialinChapters1–8withtopics selectedfromtheremainingchapters.Forsuchacourse,theinstructorwouldhave theoptionofinterchangingtheorderofChapters4and5,thusbeginningthesecond semesterwithanintroductiontostatisticaltheory(Chapter4).Aone-semester coursecouldconsistofChapters1–4withaselectionoftopicsfromChapter5. Underthisoption,thestudentseesmuchofthestatisticaltheoryforthemethods discussedinanon-theoreticalcourseinmethods.Ontheotherhand,aswiththe two-semestersequence,aftercoveringChapters1–3,theinstructorcanelecttocover Chapter5andfinishthecoursewithaselectionoftopicsfromChapter4.
ThedatasetsandRfunctionsusedinthisbookandtheRpackage hmcpkg can bedownloadedfromthistitle’spageatthesite: www.pearsonglobaleditions.com
Acknowledgments
BobHoggpassedawayin 2014,sohedidnotworkonthiseditionofthebook. Often,though,whenIwastryingtodecidewhetherornottomakeachangeinthe manuscript,IfoundmyselfthinkingofwhatBobwoulddo.Inhismemory,Ihave retainedtheorderoftheauthorsforthisedition.
Aswithearliereditions,commentsfromreadersarealwayswelcomedandappreciated.Wewouldliketothankthesereviewersofthepreviousedition:James Baldone,VirginiaCollege;StevenCulpepper,UniversityofIllinoisatUrbanaChampaign;YuichiroKakihara,CaliforniaStateUniversity;JaechoulLee,Boise StateUniversity;MichaelLevine,PurdueUniversity;TingniSun,Universityof Maryland,CollegePark;andDanielWeiner,BostonUniversity.Weappreciated andtookintoconsiderationtheircommentsforthisrevision.Weappreciatethe helpfulcommentsofThomasHettmanspergerofPennStateUniversity,AshAbebe ofAuburnUniversity,andProfessorIoannisKalogridisoftheUniversityofLeuven. AspecialthankstoPatrickBarbera(PortfolioManager,Statistics),LaurenMorse (ContentProducer,Math/Stats),YvonneVannatta(ProductMarketingManager), andtherestofthestaffatPearsonfortheirhelpinputtingthiseditiontogether. ThanksalsotoRichardPonticelli,NorthShoreCommunityCollege,whoaccuracy checkedthepageproofs.Also,aspecialthankstomywifeMargeforherunwavering supportandencouragementofmyeffortsinwritingthisedition.
JoeMcKean
AcknowledgmentsfortheGlobalEdition
Pearsonwouldliketothankandacknowledgethefollowingpeoplefortheirwork onthisGlobalEdition.
Contributors
PolinaDolmatova,AmericanUniversityofCentralAsia TsungFeiKhang,UniversityofMalaya EricA.L.LI,TheUniversityofHongKong ChoungMinNg,UniversityofMalaya NeeleshS.Upadhye,IndianInstituteofTechnologyMadras Reviewers
SampritChakrabarti,ICFAIBusinessSchoolKolkata Rub´enGarciaBerasategui,JakartaInternationalCollege AneeshKumarK.,MahatmaGandhiCollege,Iritty JesperRyd´en,SwedishUniversityofAgriculturalSciences PoojaSengupta,InternationalManagementInstituteKolkata NeeleshS.Upadhye,IndianInstituteofTechnologyMadras
This page intentionally left blank
ProbabilityandDistributions
1.1Introduction
Inthissection,weintuitivelydiscusstheconceptsofaprobabilitymodelwhichwe formalizeinSecton1.3Manykindsofinvestigationsmaybecharacterizedinpart bythefactthatrepeatedexperimentation,underessentiallythesameconditions, ismoreorlessstandardprocedure.Forinstance,inmedicalresearch,interestmay centerontheeffectofadrugthatistobeadministered;oraneconomistmaybe concernedwiththepricesofthreespecifiedcommoditiesatvarioustimeintervals;or anagronomistmaywishtostudytheeffectthatachemicalfertilizerhasontheyield ofacerealgrain.Theonlywayinwhichaninvestigatorcanelicitinformationabout anysuchphenomenonistoperformtheexperiment.Eachexperimentterminates withan outcome.Butitischaracteristicoftheseexperimentsthattheoutcome cannotbepredictedwithcertaintypriortotheexperiment.
Supposethatwehavesuchanexperiment,buttheexperimentisofsuchanature thatacollectionofeverypossibleoutcomecanbedescribedpriortoitsperformance. Ifthiskindofexperimentcanberepeatedunderthesameconditions,itiscalled a randomexperiment,andthecollectionofeverypossibleoutcomeiscalledthe experimentalspaceorthe samplespace.Wedenotethesamplespaceby C .
Example1.1.1. Inthetossofacoin,lettheoutcometailsbedenotedby T andlet theoutcomeheadsbedenotedby H .Ifweassumethatthecoinmayberepeatedly tossedunderthesameconditions,thenthetossofthiscoinisanexampleofa randomexperimentinwhichtheoutcomeisoneofthetwosymbols T or H ;that is,thesamplespaceisthecollectionofthesetwosymbols.Forthisexample,then, C = {H,T }.
Example1.1.2. Inthecastofonereddieandonewhitedie,lettheoutcomebethe orderedpair(numberofspotsuponthereddie,numberofspotsuponthewhite die).Ifweassumethatthesetwodicemayberepeatedlycastunderthesameconditions,thenthecastofthispairofdiceisarandomexperiment.Thesamplespace consistsofthe36orderedpairs: C = {(1, 1),..., (1, 6), (2, 1),..., (2, 6),..., (6, 6)}
ProbabilityandDistributions
WegenerallyusesmallRomanlettersfortheelementsof C suchas a,b, or c.Oftenforanexperiment,weareinterestedinthechancesofcertainsubsetsof elementsofthesamplespaceoccurring.Subsetsof C areoftencalled events andare generallydenotedbycapitolRomanletterssuchas A,B, or C .Iftheexperiment resultsinanelementinanevent A,wesaytheevent A hasoccurred.Weare interestedinthechancesthataneventoccurs.Forinstance,inExample1.1.1we maybeinterestedinthechancesofgettingheads;i.e.,thechancesoftheevent A = {H } occurring.Inthesecondexample,wemaybeinterestedintheoccurrence ofthesumoftheupfacesofthedicebeing“7”or“11;”thatis,intheoccurrenceof theevent A = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1), (5, 6), (6, 5)}
Nowconceiveofourhavingmade N repeatedperformancesoftherandomexperiment.Thenwecancountthenumber f oftimes(the frequency)thatthe event A actuallyoccurredthroughoutthe N performances.Theratio f/N iscalled the relativefrequency oftheevent A inthese N experiments.Arelativefrequencyisusuallyquiteerraticforsmallvaluesof N ,asyoucandiscoverbytossing acoin.Butas N increases,experienceindicatesthatweassociatewiththeevent A anumber,say p,thatisequalorapproximatelyequaltothatnumberaboutwhich therelativefrequencyseemstostabilize.Ifwedothis,thenthenumber p canbe interpretedasthatnumberwhich,infutureperformancesoftheexperiment,the relativefrequencyoftheevent A willeitherequalorapproximate.Thus,although we cannot predicttheoutcomeofarandomexperiment,we can,foralargevalue of N ,predictapproximatelytherelativefrequencywithwhichtheoutcomewillbe in A.Thenumber p associatedwiththeevent A isgivenvariousnames.Sometimesitiscalledthe probability thattheoutcomeoftherandomexperimentisin A;sometimesitiscalledthe probability oftheevent A;andsometimesitiscalled the probabilitymeasure of A.Thecontextusuallysuggestsanappropriatechoiceof terminology.
Example1.1.3. Let C denotethesamplespaceofExample1.1.2andlet B be thecollectionofeveryorderedpairof C forwhichthesumofthepairisequalto seven.Thus B = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2)(6, 1)}.Supposethatthediceare cast N =400timesandlet f denotethefrequencyofasumofseven.Supposethat 400castsresultin f =60.Thentherelativefrequencywithwhichtheoutcome wasin B is f/N = 60 400 =0 15.Thuswemightassociatewith B anumber p thatis closeto0.15,and p wouldbecalledtheprobabilityoftheevent B .
Remark1.1.1. Theprecedinginterpretationofprobabilityissometimesreferred toasthe relativefrequencyapproach,anditobviouslydependsuponthefactthatan experimentcanberepeatedunderessentiallyidenticalconditions.However,many personsextendprobabilitytoothersituationsbytreatingitasarationalmeasure ofbelief.Forexample,thestatement p = 2 5 foranevent A wouldmeantothem thattheir personal or subjective probabilityoftheevent A isequalto 2 5 .Hence, iftheyarenotopposedtogambling,thiscouldbeinterpretedasawillingnesson theirparttobetontheoutcomeof A sothatthetwopossiblepayoffsareinthe ratio p/(1 p)= 2 5 / 3 5 = 2 3 .Moreover,iftheytrulybelievethat p = 2 5 iscorrect, theywouldbewillingtoaccepteithersideofthebet:(a)win3unitsif A occurs andlose2ifitdoesnotoccur,or(b)win2unitsif A doesnotoccurandlose3if
itdoes.However,sincethemathematicalpropertiesofprobabilitygiveninSection 1.3areconsistentwitheitheroftheseinterpretations,thesubsequentmathematical developmentdoesnotdependuponwhichapproachisused.
Theprimarypurposeofhavingamathematicaltheoryofstatisticsistoprovide mathematicalmodelsforrandomexperiments.Onceamodelforsuchanexperimenthasbeenprovidedandthetheoryworkedoutindetail,thestatisticianmay, withinthisframework,makeinferences(thatis,drawconclusions)abouttherandomexperiment.Theconstructionofsuchamodelrequiresatheoryofprobability. Oneofthemorelogicallysatisfyingtheoriesofprobabilityisthatbasedonthe conceptsofsetsandfunctionsofsets.TheseconceptsareintroducedinSection1.2.
1.2Sets
Theconceptofa set ora collection ofobjectsisusuallyleftundefined.However, aparticularsetcanbedescribedsothatthereisnomisunderstandingastowhat collectionofobjectsisunderconsideration.Forexample,thesetofthefirst10 positiveintegersissufficientlywelldescribedtomakeclearthatthenumbers 3 4 and 14arenotintheset,whilethenumber3isintheset.Ifanobjectbelongstoa set,itissaidtobean element oftheset.Forexample,if C denotesthesetofreal numbers x forwhich0 ≤ x ≤ 1,then 3 4 isanelementoftheset C .Thefactthat 3 4 isanelementoftheset C isindicatedbywriting 3 4 ∈ C .Moregenerally, c ∈ C meansthat c isanelementoftheset C .
Thesetsthatconcernusarefrequently setsofnumbers.However,thelanguage ofsetsof points provessomewhatmoreconvenientthanthatofsetsofnumbers. Accordingly,webrieflyindicatehowweusethisterminology.Inanalyticgeometry considerableemphasisisplacedonthefactthattoeachpointonaline(onwhich anoriginandaunitpointhavebeenselected)therecorrespondsoneandonlyone number,say x;andthattoeachnumber x therecorrespondsoneandonlyonepoint ontheline.Thisone-to-onecorrespondencebetweenthenumbersandpointsona lineenablesustospeak,withoutmisunderstanding,ofthe“point x”insteadofthe “number x.”Furthermore,withaplanerectangularcoordinatesystemandwith x and y numbers,toeachsymbol(x,y )therecorrespondsoneandonlyonepointinthe plane;andtoeachpointintheplanetherecorrespondsbutonesuchsymbol.Here again,wemayspeakofthe“point(x,y ),”meaningthe“orderednumberpair x and y .”Thisconvenientlanguagecanbeusedwhenwehavearectangularcoordinate systeminaspaceofthreeormoredimensions.Thusthe“point(x1 ,x2 ,...,xn )” meansthenumbers x1 ,x2 ,...,xn intheorderstated.Accordingly,indescribingour sets,wefrequentlyspeakofasetofpoints(asetwhoseelementsarepoints),being careful,ofcourse,todescribethesetsoastoavoidanyambiguity.Thenotation C = {x :0 ≤ x ≤ 1} isread“C istheone-dimensionalsetofpoints x forwhich 0 ≤ x ≤ 1.”Similarly, C = {(x,y ):0 ≤ x ≤ 1, 0 ≤ y ≤ 1} canberead“C isthe two-dimensionalsetofpoints(x,y )thatareinteriorto,orontheboundaryof,a squarewithoppositeverticesat(0, 0)and(1, 1).”
Wesayaset C is countable if C isfiniteorhasasmanyelementsasthereare positiveintegers.Forexample,thesets C1 = {1, 2,..., 100} and C2 = {1, 3, 5, 7,...}
ProbabilityandDistributions
arecountablesets.Theintervalofrealnumbers(0, 1],though,isnotcountable.
1.2.1ReviewofSetTheory
AsinSection1.1,let C denotethesamplespacefortheexperiment.Recallthat eventsaresubsetsof C .Weusethewordseventandsubsetinterchangeablyinthis section.Anelementaryalgebraofsetswillprovequiteusefulforourpurposes.We nowreviewthisalgebrabelowalongwithillustrativeexamples.Forillustration,we alsomakeuseof Venndiagrams.ConsiderthecollectionofVenndiagramsin Figure1.2.1.Theinterioroftherectangleineachplotrepresentsthesamplespace C.TheshadedregioninPanel(a)representstheevent A
Figure1.2.1: AseriesofVenndiagrams.ThesamplespaceCisrepresentedby theinterioroftherectangleineachplot.Panel(a)depictstheevent A;Panel(b) depicts A ⊂ B ;Panel(c)depicts A ∪ B ;andPanel(d)depicts A ∩ B
Wefirstdefinethecomplementofanevent A.
Definition1.2.1. The complement ofanevent A isthesetofallelementsinC whicharenotin A.Wedenotethecomplementof A by Ac .Thatis, Ac = {x ∈C : x/ ∈ A}.
Thecomplementof A isrepresentedbythewhitespaceintheVenndiagramin Panel(a)ofFigure1.2.1.
Theemptysetistheeventwithnoelementsinit.Itisdenotedby φ.Note that C c = φ and φc = C .Thenextdefinitiondefineswhenoneeventisasubsetof another.
Definition1.2.2. Ifeachelementofaset A isalsoanelementofset B ,theset A iscalleda subset oftheset B .Thisisindicatedbywriting A ⊂ B .If A ⊂ B and also B ⊂ A,thetwosetshavethesameelements,andthisisindicatedbywriting A = B .
Panel(b)ofFigure1.2.1depicts A ⊂ B .
Theevent A or B isdefinedasfollows:
Definition1.2.3. Let A and B beevents.Thenthe union of A and B istheset ofallelementsthatarein A orin B orinboth A and B .Theunionof A and B isdenotedby A ∪ B
Panel(c)ofFigure1.2.1shows A ∪ B .
Theeventthatboth A and B occurisdefinedby,
Definition1.2.4. Let A and B beevents.Thenthe intersection of A and B is thesetofallelementsthatareinboth A and B .Theintersectionof A and B is denotedby A ∩ B
Panel(d)ofFigure1.2.1illustrates A ∩ B
Twoeventsare disjoint iftheyhavenoelementsincommon.Moreformallywe define
Definition1.2.5. Let A and B beevents.Then A and B are disjoint if A ∩ B = φ
If A and B aredisjoint,thenwesay A ∪ B formsa disjointunion. Thenexttwo examplesillustratetheseconcepts.
Example1.2.1. Supposewehaveaspinnerwiththenumbers1through10on it.Theexperimentistospinthespinnerandrecordthenumberspun.Then C = {1, 2,..., 10}.Definetheevents A, B ,and C by
,
},
= {
, 3, 4},and C = {3, 4, 5, 6},respectively. Ac = {3, 4,..., 10}; A ∪ B = {1, 2, 3, 4}; A ∩ B = {2}
Thereadershouldverifytheseresults.
Example1.2.2. Forthisexample,supposetheexperimentistoselectarealnumber intheopeninterval(0, 5);hence,thesamplespaceis C =(0, 5).Let A =(1, 3),
B =(2, 4),and C =[3, 4.5).
,
ProbabilityandDistributions
Asketchoftherealnumberlinebetween0and5helpstoverifytheseresults.
Expressions(1.2.1)–(1.2.2)and(1.2.3)–(1.2.4)areillustrationsofgeneral distributivelaws.Foranysets A, B ,and C ,
Thesefollowdirectlyfromsettheory.Toverifyeachidentity,sketchVenndiagrams ofbothsides.
Thenexttwoidentitiesarecollectivelyknownas DeMorgan’sLaws.Forany sets A and B ,
Forinstance,inExample1.2.1,
while,fromExample1.2.2, (A ∩ B )c =(2, 3)c =(0, 2] ∪ [3, 5)=[(0, 1] ∪ [3, 5)] ∪ [(0
Asthelastexpressionsuggests,itiseasytoextendunionsandintersectionstomore thantwosets.If A1 ,A2 ,...,An areanysets,wedefine A1 ∪ A2 ∪···∪ An = {x : x ∈ Ai , forsome i =1, 2,...,n} (1.2.8) A1 ∩ A2 ∩···∩ A
Weoftenabbreviativetheseby ∪n i=1 Ai and ∩n i=1 Ai ,respectively.Expressionsfor countableunionsandintersectionsfollowdirectly;thatis,if A1 ,A2 ,...,An ... isa sequenceofsetsthen
Thenexttwoexamplesillustratetheseideas.
Example1.2.3. Suppose C = {1, 2, 3,...}.If An = {1, 3,..., 2n 1} and Bn = {n,n +1,...},for n =1, 2, 3,...,then
,
,...
Example1.2.4. Suppose C istheintervalofrealnumbers(0, 5).Suppose Cn = (1 n 1 , 2+ n 1 )and Dn =(n 1 , 3 n 1 ),for n =1, 2, 3,.... Then
, 3);
Weoccassionallyhavesequencesofsetsthatare monotone.Theyareoftwo types.Wesayasequenceofsets {An } is nondecreasing,(nestedupward),if An ⊂ An+1 for n =1, 2, 3,....Forsuchasequence,wedefine
Thesequenceofsets An = {1, 3,..., 2n 1} ofExample1.2.3issuchasequence. Sointhiscase,wewritelimn→∞ An = {1, 3, 5,...}.Thesequenceofsets {Dn } of Example1.2.4isalsoanondecreasingsuquenceofsets. Thesecondtypeofmonotonesetsconsistsofthe nonincreasing,(nested downward) sequences.Asequenceofsets {An } is nonincreasing,if An ⊃ An+1 for n =1, 2, 3,....Inthiscase,wedefine
Thesequencesofsets {Bn } and {Cn } ofExamples1.2.3and1.2.4,respectively,are examplesofnonincreasingsequencesofsets.
1.2.2SetFunctions
Manyofthefunctionsusedincalculusandinthisbookarefunctionsthatmapreal numbersintorealnumbers.Weareconcernedalsowithfunctionsthatmapsets intorealnumbers.Suchfunctionsarenaturallycalledfunctionsofasetor,more simply, setfunctions.Nextwegivesomeexamplesofsetfunctionsandevaluate themforcertainsimplesets.
Example1.2.5. Let C = R,thesetofrealnumbers.Forasubset A in C ,let Q(A) beequaltothenumberofpointsin A thatcorrespondtopositiveintegers.Then Q(A)isasetfunctionoftheset A.Thus,if A = {x :0 <x< 5},then Q(A)=4; if A = {−2, 1},then Q(A)=0;andif A = {x : −∞ <x< 6},then Q(A)=5.
Example1.2.6. Let C = R2 .Forasubset A of C ,let Q(A)betheareaof A if A hasafinitearea;otherwise,let Q(A)beundefined.Thus,if A = {(x,y ): x2 + y 2 ≤ 1},then Q(A)= π ;if A = {(0, 0), (1, 1), (0, 1)},then Q(A)=0;andif A = {(x,y ):0 ≤ x, 0 ≤ y,x + y ≤ 1},then Q(A)= 1 2
Oftenoursetfunctionsaredefinedintermsofsumsorintegrals.1 Withthisin mind,weintroducethefollowingnotation.Thesymbol A f (x) dx
1 PleaseseeChapters2and3of MathematicalComments,atsitenotedinthePreface,fora reviewofsumsandintegrals
ProbabilityandDistributions
meanstheordinary(Riemann)integralof f (x)overaprescribedone-dimensional set A andthesymbol
g (x,y ) dxdy
meanstheRiemannintegralof g (x,y )overaprescribedtwo-dimensionalset A Thisnotationcanbeextendedtointegralsover n dimensions.Tobesure,unless thesesets A andthesefunctions f (x)and g (x,y )arechosenwithcare,theintegrals frequentlyfailtoexist.Similarly,thesymbol
f (x)
meansthesumextendedoverall x ∈ A andthesymbol
g (x,y )
meansthesumextendedoverall(x,y ) ∈ A.Aswithintegration,thisnotation extendstosumsover n dimensions.
Thefirstexampleisforasetfunctiondefinedonsumsinvolvinga geometric series.AspointedoutinExample2.3.1of MathematicalComments, 2 if |a| < 1, thenthefollowingseriesconvergesto1/(1 a):
Example1.2.7. Let C bethesetofallnonnegativeintegersandlet A beasubset of C .Definethesetfunction Q by
Itfollowsfrom(1.2.18)that Q(C )=3.If A = {1, 2, 3} then Q(A)=38/27.Suppose B = {1, 3, 5,...} isthesetofalloddpositiveintegers.Thecomputationof Q(B )is givennext.Thisderivationconsistsofrewritingtheseriessothat(1.2.18)canbe applied.Frequently,weperformsuchderivationsinthisbook.
Inthenextexample,thesetfunctionisdefinedintermsofanintegralinvolving theexponentialfunction f (x)= e x .
2 DownloadableatsitenotedinthePreface
Example1.2.8. Let C betheintervalofpositiverealnumbers,i.e., C =(0, ∞). Let A beasubsetof C .Definethesetfunction Q by
Q(A)= A e x dx, (1.2.20)
providedtheintegralexists.Thereadershouldworkthroughthefollowingintegrations:
x
[(1, 3) ∪ [3, 5)]=
Q(C )= ∞ 0 e x dx =1
Ourfinalexample,involvesan n dimensionalintegral.
= Q[(1, 3)]+ Q([3, 5)]
Example1.2.9. Let C = Rn .For A in C definethesetfunction
(A)= A
,
providedtheintegralexists.Forexample,if A = {(x1 ,x2 ,...,xn ):0 ≤ x1 ≤ x2 , 0 ≤ xi ≤ 1, for i =2,3,...,n},thenuponexpressingthemultipleintegralas aniteratedintegral3 weobtain
If B = {(x1 ,x
(B )=
where n!= n(n 1) 3 2 1.
≤ 1},then
3 Foradiscussionofmultipleintegralsintermsofiteratedintegrals,seeChapter3of MathematicalComments.
ProbabilityandDistributions
EXERCISES
1.2.1. Findtheunion C1 ∪ C2 andtheintersection C1 ∩ C2 ofthetwosets C1 and C2 ,where
(a) C1 = {2, 3, 5, 7}, C2 = {1, 3, 5}
(b) C1 = {x :0 ≤ x ≤ 3}, C2 = {x :2 <x< 4}
(c) C1 = {(x,y ):0 <x< 1, 0 <y< 3}, C2 = {(x,y ):0 <x< 2, 2 ≤ y< 3}
1.2.2. Findthecomplement C c oftheset C withrespecttothespace C if
(a) C = {x :0 <x< 2}, C = x :0 <x< 2 3 .
(b) C = (x,y,z ): x2 +2y 2 +3z 2 ≤ 4 , C = (x,y,z ): x2 +2y 2 +3z 2 < 4
(c) C = (x,y ): x2 + y 2 ≤ 1 , C = {(x,y ): |x| + |y | < 1}
1.2.3. Listallpossiblearrangementsofthefourletters l , a, m,and b.Let C1 be thecollectionofthearrangementsinwhich b isinthefirstposition.Let C2 bethe collectionofthearrangementsinwhich a isinthethirdposition.Findtheunion andtheintersectionof C1 and C2 .
1.2.4. ConcerningDeMorgan’sLaws(1.2.6)and(1.2.7):
(a) UseVenndiagramstoverifythelaws.
(b) Showthatthelawsaretrue.
(c) Generalizethelawstocountableunionsandintersections.
1.2.5. BytheuseofVenndiagrams,inwhichthespace C isthesetofpoints enclosedbyarectanglecontainingthecircles C1 ,C2 , and C3 ,comparethefollowing sets.Theselawsarecalledthe distributivelaws.
(a) C1 ∩ (C2 ∪ C3 )and(C1 ∩ C2 ) ∪ (C1 ∩ C3 ).
(b) C1 ∪ (C2 ∩ C3 )and(C1 ∪ C2 ) ∩ (C1 ∪ C3 ).
1.2.6. Showthatthefollowingsequencesofsets, {Ck },arenondecreasing,(1.2.16), thenfindlimk →∞ Ck .
(a) Ck = {x :1/k ≤ x ≤ 3 1/k }, k =1, 2, 3,... .
(b) Ck = {(x,y ):1/k ≤ x2 + y 2 ≤ 4 1/k }, k =1, 2, 3,...
1.2.7. Showthatthefollowingsequencesofsets, {Ck },arenonincreasing,(1.2.17), thenfindlimk →∞ Ck .
(a) Ck = {x :2 1/k<x ≤ 2}, k =1, 2, 3,... .
(b) Ck = {x :2 <x ≤ 2+1/k }, k =1, 2, 3,... .
(c) Ck = {(x,y ):0 ≤ x2 + y 2 ≤ 1/k }, k =1, 2, 3,... .
1.2.8. Foreveryone-dimensionalset C ,definethefunction Q (C )= C f (x), where f (x)= 3 4 1 4 x , x =0, 1, 2,...,zeroelsewhere.If C1 = {x : x =0, 2, 4} and C2 = {x : x =0, 1, 2,...},find Q (C1 )and Q (C2 ) . Hint :Recallthat Sn = a + ar + + ar n 1 = a(1 r n )/(1 r )and,hence,it followsthatlimn→∞ Sn = a/(1 r )providedthat |r | < 1.
1.2.9. Foreveryone-dimensionalset C forwhichtheintegralexists,let Q(C )= C f (x) dx,where f (x)= 3 4 (1 x2 ), 1 <x< 1,zeroelsewhere;otherwise,let Q(C ) beundefined.If C1 = {x : 1 3 <x< 1 3 }, C2 = {0},and C3 = {x : 1 <x< 5}, find Q(C1 ), Q(C2 ),and Q(C3 ).
1.2.10. Foreverytwo-dimensionalset C containedin R2 forwhichtheintegral exists,let Q(C )= C (x2 + y 2 ) dxdy .If C1 = {(x,y ): 1 ≤ x ≤ 1, 1 ≤ y ≤ 1}, C2 = {(x,y ): 1 ≤ x = y ≤ 1},and C3 = {(x,y ): x2 + y 2 ≤ 1},find Q(C1 ),Q(C2 ), and Q(C3 ).
1.2.11. Let C denotethesetofpointsthatareinteriorto,orontheboundaryof,a squarewithoppositeverticesatthepoints(0,0)and(1,1).Let Q(C )= C dydx
(a) If C ⊂C istheset {(x,y ):0 <y/2 <x< 1/2},compute Q(C ).
(b) If C ⊂C istheset {(x,y ):0 <x< 1,x + y =1},compute Q(C ).
(c) If C ⊂C istheset {(x,y ):0 <x/2 <y ≤ x +1/4 < 1},compute Q(C ).
1.2.12. Let C bethesetofpointsinteriortoorontheboundaryofacubewith edgeoflength1.Moreover,saythatthecubeisinthefirstoctantwithonevertex atthepoint(0, 0, 0)andanoppositevertexatthepoint(1, 1, 1).Let Q(C )= C dxdydz
(a) If C ⊂C istheset {(x,y,z ):0 <x<y<z< 1},compute Q(C ).
(b) If C isthesubset {(x,y,z ):0 <x = y = z< 1},compute Q(C ).
1.2.13. Let C denotetheset {(x,y,z ): x2 + y 2 + z 2 ≤ 1}.Usingsphericalcoordinates,evaluate Q(C )= C x2 + y 2 + z 2 dxdydz.
1.2.14. Tojoinacertainclub,apersonmustbeeitherastatisticianoramathematicianorboth.Ofthe35membersinthisclub,25arestatisticiansand17 aremathematicians.Howmanypersonsintheclubarebothastatisticiananda mathematician?
1.2.15. Afterahard-foughtfootballgame,itwasreportedthat,ofthe11starting players,7hurtahip,5hurtanarm,7hurtaknee,3hurtbothahipandanarm, 3hurtbothahipandaknee,2hurtbothanarmandaknee,and1hurtallthree. Commentontheaccuracyofthereport.
