PREFACE
GLOSSARY OF NOTATIONS
1 VIDEO FORMATION, PERCEPTION, AND REPRESENTATION
l.l Color Perception and Specification
l.2 Video Capture and Display
l.3 Analog Video Raster
l.4 Analog Color Television Systems
l.5 Digital Video
l.6 Summary 30
l.7 Problems 31
1.8 Bibliography 32
2 FOURIER ANALYSIS OF VIDEO SIGNALS AND FREQUENCY RESPONSE OF THE HUMAN VISUAL SYSTEM
2.1 Multidimensional Continuous-Space Signals and Systems 33
2.2 Multidimensional Discrete-Space Signals and Systems 36
2.3 Frequency Domain Characterization of Vdeo Signals 38
2.4 Frequency Response of the Human Visual System 42
2.5 Summary 50
2.6 Problems 5l
2.7 Bib1iography 52
3 VIDEO SAMPLING
3.l Basics of the Lattice Theory 54
3.2 Sampling over Lattices 59
3.3 Sampling of Video Signals
3.4 Filtering Operations in Cameras and Display Devices
3.5 Summary 80
3.6 Problems 80
3.7 Bibliography 83
4 VIDEO SAMPLING RATE CONVERSION
4.l Conversion of Signals Sampled on Different Lattices
4.2 Sampling Rate Conversion of Video Signals
4.3 Summary l05
4.4 Problems l06
4.5 Bibliography 109
5 VIDEO MODELING
5.l Camera Model
5.2 Illumination Model
5.3 Object Model l20
5.4 Scene Model 125
5.5 Two-Dimensional Motion Models l28
5.6 Summary l37
5.7 Problems l38
5.8 Bibliography 139
6 TWO-DIMENSIONAL MOTION ESTIMATION
6.l Optical Flow l42
6.2 General Methodologies 145
6.3 Pixel-Based Motion Estimation
6.4 Block-Matching Algorithm
6.5 Deformable Block-Matching Algorithms
6.6 Mesh-Based Motion Estimation l69
6.7 Global Motion Estimation l77
6.8 Region-Based Motion Estimation 179
6.9 Multiresolution Motion Estimation 182
6.l0 Application of Motion Estimation in Video Coding
6.ll Summary 188
6.l2 Problems 189
6.l3 Bibliography l9l
7 THREE-DIMENSIONAL MOTION ESTIMATION
7.l Feature-Based Motion Estimation l95
7.2 Direct Motion Estimation 203
7.3 Iterative Motion Estimation 2l2
7.4 Summary 2l3
7.5 Problems 2l4
7.6 Bibliography 2l5
8 FOUNDATIONS OF VIDEO CODING
8.l Overview of Coding Systems 218
8.2 Basic Notions in Probability and Information Theory 22l
8.3 Information Theory for Source Coding 227
8.4 Binary Encoding 234
8.5 Scalar Quantization 24l
8.6 Vector Quantization 248
8.7 Sununary 257
8.8 Problems 259
8.9 Bibliography 26l
9 WAVEFORM-BASED VIDEO CODING
9.1 Block-Based Transform Coding 263
9.2 Predictive Coding 285
9.3 Video Coding Using Temporal Prediction and Transform Coding
9.4 Sununary 308
9.5 Problems 309
9.6 Bibliography 3ll
10 CONTENT-DEPENDENT VIDEOCODING
l0.l Two-Dimensional Shape Coding 3l4
l0.2 Texture Coding for Arbitrarily Shaped Regions 324
l0.3 Joint Shape and Texture Coding 326
l0.4 Region-Based Video Coding 327
l0.5 Object-Based Video Coding 328
l0.6 Knowledge-Based Video Coding 336
l0.7 Semantic Video Coding 338
l0.8 Layered Coding System 339
l0.9 Summary 342
l0.l0 Problems 343
l0.ll Bibliography
11 SCALABLE VIDEO CODING
ll.l Basic Modes of Scalability 350
ll.2 Object-Based Scalability 359
ll.3 Wavelet-Transform-Based Coding 36l
ll.4 Summary 370
ll.5 Problems 370
ll.6 Bibliography 371
12 STEREO AND MULTIVIEW SEQUENCE PROCESSING
l2.l Depth Perception 375
l2.2 Stereo Imaging Principle 377
l2.3 Disparity Estimation 385
l2.4 Intermediate View Synthesis 393
l2.5 Stereo Sequence Coding 396
l2.6 Summary 400
l2.7 Problems 402
l2.8 Bibliography 403
13 VIDEO COMPRESSION STANDARDS
l3.l Standardization 406
l3.2 Video Telephony with H.26l and H.263 4l3
l3.3 Standards for Visual Communication Systems 42l
13.4 Consume Video Communications with MPEG-1 423
13.5 Digital TV with MPEG-2 426
l3.6 Coding of Audiovisual Objects with MPEG-4 437
l3.7 Video Bit Stream Syntax 454
l3.8 Multimedia Content Description Using MPEG-7 458
l3.9 Summary 465
l3.l0 Problems 466
l3.ll Bibliography 467
14 ERROR CONTROL IN VIDEOCOMMUNICATIONS
l4.l Motivation and Overview of Approaches 473
l4.2 Typical Video Applicahons and Conununication Networks 476
14.3 Transport-Level Error Contro1 485
l4.4 Error-Resilient Encoding 489
l4.5 Decoder Error Concealment 498
l4.6 Encoder-Decoder Interactive Error Control 502
l4.7 Error-Resilience Tools in H.263 and MPEG-4 505
l4.8 Sununary 509
l4.9 Problems 5ll
l4.l0 Bibliography 5l3
15 STREAMING VIDEO OVER THE INTERNET AND WIRELESS IP NETWORKS
l5.l Architecture for Video Streaming Systems 520
l5.2 Video Compression 522