Contents
Preface
Chapter 1 Generation of Random Variables 1
1.1 The Inversion Method 3
1.1.1 Generating samples from continuous distributions 3
1.1.2 Generating samples from discrete distributions 7
1.2 The Grid Method 12
1.3 The Rejection Method 15
1.3.1 Generating samples from continuous distributions 15
1.3.2 The efficiency of the rejection method 18
1.3.3 Several examples 20
1.3.4 Log-concave densities 24
1.4 The Sampling/Importance Resampling (SIR) Method 27
1.4.1 The SIR without replacement 28
1.4.2 Theoretical justification 30
1.5 The Stochastic Representation (SR) Method.32
1.5.1 The‘d=’operator 32
1.5.2 Many-to-one SR for univariate case 34
1.5.3 SR for multivariate case 36
1.5.4 Mixture representation 39
1.6 The Conditional Sampling Method 42
Exercise 1 47
Chapter 2 Optimization 53
2.1 A Review of Some Standard Concepts 54
2.1.1 Order relations 54
2.1.2 Stationary points 57
2.1.3 Convex and concave functions 60
2.1.4 Mean value theorem 61
2.1.5 Taylor theorem 63
2.1.6 Rates of convergence 64
2.1.7 The case of multiple dimensions 64
2.2 Newton’s Method and Its Variants 66
2.2.1 Newton’s method and root finding 67
2.2.2 Newton’s method and optimization 71
2.2.3 The Newton–Raphson algorithm 72
2.2.4 The Fisher scoring algorithm 75
2.2.5 Application to logistic regression 76
2.3 The Expectation–Maximization (EM) Algorithm 80
2.3.1 The formulation of the EM algorithm 81
2.3.2 The ascent property of the EM algorithm 89
2.3.3 Missing information principle and standard errors 92
2.4 The ECM Algorithm 95
2.5 Minorization–Maximization (MM) Algorithms 100
2.5.1 A brief review of MM algorithms 100
2.5.2 The MM idea 101
2.5.3 The quadratic lower–bound algorithm 103
2.5.4 The De Pierro algorithm 106
Exercise 2 115
Chapter 3 Integration 125
3.1 Laplace Approximations 126
3.2 Riemannian Simulation 129
3.2.1 Classical Monte Carlo integration 129
3.2.2 Motivation for Riemannian simulation 132
3.2.3 Variance of the Riemannian sum estimator 133
3.3 The Importance Sampling Method 135
3.3.1 The formulation of the importance sampling method 135
3.3.2 The weighted estimator 138
3.4 Variance Reduction 141
3.4.1 Antithetic variables 141
3.4.2 Control variables 145
Exercise 3 146
Chapter 4 Markov Chain Monte Carlo Methods 149
4.1 Bayes Formulae and Inverse Bayes Formulae (IBF) 151
4.1.1 The point，function- and sampling-wise IBF 152
4.1.2 Monte Carlo versions of the IBF 160
4.1.3 Generalization to the case of three random variables 163
4.2 The Bayesian Methodology 163
4.2.1 The posterior distribution 165
4.2.2 Nuisance parameters 167
4.2.3 Posterior predictive distribution 169
4.2.4 Bayes factor 172
4.2.5 Estimation of marginal likelihood 173
4.3 The Data Augmentation (DA) Algorithm 175
4.3.1 Missing data mechanism 175
4.3.2 The idea of data augmentation 177
4.3.3 The original DA algorithm 178
4.3.4 Connection with the IBF 180
4.4 The Gibbs sampler 181
4.4.1 The formulation of the Gibbs sampling 182
4.4.2 The two–block Gibbs sampling 184
4.5 The Exact IBF Sampling 187
4.6 The IBF sampler 191
4.6.1 Background and the basic idea 191
4.6.2 The formulation of the IBF sampler 192
4.6.3 Theoretical justification for choosing θ0 =.θ 194
Exercise 4 196
Chapter 5 Bootstrap Methods 203
5.1 Bootstrap Confidence Intervals 203
5.1.1 Parametric bootstrap 203
5.1.2 Non-parametric bootstrap 213
5.2 Hypothesis Testing with the Bootstrap 219
5.2.1 Testing equality of two unknown distributions 219
5.2.2 Testing equality of two group means 223
5.2.3 One–sample problem 228
Exercise 5 231
Appendix A Some Statistical Distributions and Stochastic
Processes 233
A.1 Discrete Distributions 233
A.1.1 Finite discrete distribution 233
A.1.2 Hypergeometric distribution 234
A.1.3 Binomial and related distributions 235
A.1.4 Poisson and related distributions 237
A.1.5 Negative–binomial and related distributions 240
A.1.6 Generalized Poisson and related distributions 242
A.1.7 Multinomial and related distributions 243
A.2 Continuous Distributions 245
A.2.1 Uniform， beta and Dirichlet distributions 245
A.2.2 Logistic and Laplace distributions 248
A.2.3 Exponential， gamma and inverse gamma distributions 249
A.2.4 Chi-square， F and inverse chi-square distributions 251
A.2.5 Normal， lognormal and inverse Gaussian distributions 252
A.2.6 Multivariate normal distribution 254
A.2.7 Student’s t and multivariate t distributions 255
A.2.8 Wishart and inverse Wishart distributions 256
A.3 Stochastic Processes 258
A.3.1 Homogeneous Poisson process 258
A.3.2 Nonhomogeneous Poisson process 259
Appendix B R Programming
鄂公网安备 42010302001612号 