ICOMMUNICATIONandREGENERATION
1Heuristics
1.1ARangeofMarkovianEnvironments
1.2BasicModelsinPractice
1.3StochasticStabilityForMarkovModels
1.4Commentary
2MarkovModels
2.1MarkovModelsInTimeSeries
2.2NonlinearStateSpaceModels
2.3ModelsInControlAndSystemsTheory
2.4MarkovModelsWithRegenerationTimes
2.5Commentary
3TransitionProbabilities
3.1DefiningaMarkovianProcess
3.2FoundationsonaCountableSpace
3.3SpecificTransitionMatrices
3.4FoundationsforGeneral'StateSpaceChains
3.5BuildingTransitionKernelsForSpecificModels
3.6Commentary
4Irreducibility
4.1CommunicationandIrreducibility:CountableSpaces
4.2-Irreducibility
4.3-IrreducibilityForRandomWalkModels
4.4-IrreducibleLinearModels
4.5Commentary
5Pseudo-atoms
5.1Splitting-IrreducibleChains
5.2SmallSets
5.3SmallSetsforSpecificModels
5.4CyclicBehavior
5.5PetiteSetsandSampledChains
5.6Commentary
TopologyandContinuity
6.1FellerPropertiesandFormsofStability
6.2T-chains
6.3ContinuousComponentsForSpecificModels
6.4e-Chains
6.5Commentary
TheNonlinearStateSpaceModel
7.1ForwardAccessibilityandContinuousComponents
7.2MinimalSetsandIrreducibility
7.3Periodicityfornonlinearstatespacemodels
7.4ForwardAccessibleExamples
7.5Equicontinnityandthenonlinearstatespacemodel
7.6Commentary
IISTABILITYSTRUCTURES
8TransienceandRecurrence
8.1Classifyingchainsoncountablespaces
8.2Classifying-irreduciblechains
8.3Recurrenceandtransiencerelationships
8.4Classificationusingdriftcriteria
8.5ClassifyingrandomwalkonIR+
8.6Commentary
HarrisandTopologicalRecurrence
9.1Harrisrecurrence
9.2Non-evanescentandrecurrentchains
9.3Topologicallyrecurrentandtransientstates
9.4Criteriaforstabilityonatopologicalspace
9.5Stochasticcomparisonandincrementanalysis
9.6Commentary
10TheExistenceof
10.1StationarityandInvariance
10.2Theexistenceof:chainswithatoms
10.3Invariantmeasures:countablespacemodels
10.4Theexistenceof:-irreduciblechains
10.5InvariantMeasures:GeneralModels
10.6Commentary
11DriftandRegularity
11.1Regularchains
11.2Drift,hittingtimesanddeterministicmodels
11.3Driftcriteriaforregularity
11.4Usingtheregularitycriteria
11.5Evaluatingnon-positivity
11.6Commentary
12InvatianceandTightness
12.1Chainsboundedinprobability
12.2Generalizedsamplingandinvariantmeasures
12.3Theexistenceofa-finlteinvariantmeasure
12.4InvariantMeasuresfore-Chains
12.5Establishingboundednessinprobability
12.6Commentary
IIICONVERGENCE
13Ergodicity
13.1Ergodicchainsoncountablespaces
13.2Renewalandregeneration
13.3ErgodicityofpositiveHarrischains
13.4Sumsoftransitionprobabilities
13.5Commentary
14f-Ergodicityandf-Regularity
14.1f-Properties:chainswithatoms
14.2f-Regularityanddrift
14.3f-Ergodicityforgeneralchains
14.4f-Ergodicityofspecificmodels
14.5AKeyRenewalTheorem
14-6Commentary
15GeometricErgodicity
15.1Geometricproperties:chainswithatoms
15.2Kendallsetsanddriftcriteria
15.3f-Geometricregularityof∮and∮n
15.4f-Geometricergodicityforgeneralchains
15.5Simplerandomwalkandlinearmodels
15.6Commentary
16V-UniformErgodicity
16.1Operatornormconvergence
16.2Uniformergodicity
16.3Geometricergodicityandincrementanalysis
16.4Modelsfromqueueingtheory
16.5Autoregressiveandstatespacemodels
16.6Commentary
17SamplePathsandLimitTheorems
17.1Invariant-FieldsandtheLLN
17.2ErgodicTheoremsforChainsPossessinganAtom
17.3GeneralHarrisChains
17.4TheFunctionalCLT
17.5CriteriafortheCLTandtheLIL
17.6Applications
17.7Commentary
18Positivity
18.1Nullrecurrentchains
18.2Characterizingpositivityusingpn
18.3PositivityandT-chains
18.4Positivityande-Chains
18.5TheLLNfore-Chains
18.6Commentary
19GeneralizedClassificationCriteria
19.1State-dependentdrifts
19.2History-dependentdriftcriteria
19.3Mixeddriftconditions
19.4Commentary
IVAPPENDICES
MudMaps
A.1Recurrenceversustransience
A.2Positivityversusnullity
A.3ConvergenceProperties
BTestingforStability
B.1AGlossaryofDriRConditions
B.2ThescalarSETARModel:acompleteclassification
CAGlossaryofModelAssumptions
C.1RegenerativeModels
C.2StateSpaceModels
DSomeMathematicalBackground
D.1SomeMeasureTheory
D.2SomeProbabilityTheory
D.3SomeTopology
D.4SomeRealAnalysis
D.5SomeConvergenceConceptsforMeasures
D.6SomeMartingaleTheory
D.7SomeResultsonSequencesandNumbers
References
Index
SymbolsIndex