ContentsAbout the Author xiiiForeword xvPreface xixGlossary xxi1 Introduction 11.1 The Scene 11.2 Video Compression 31.3 MPEG-4 and H.264 51.4 This Book 61.5 References 72 Video Formats and Quality 92.1 Introduction 92.2 Natural Video Scenes 92.3 Capture 102.3.1 Spatial Sampling 112.3.2 Temporal Sampling 112.3.3 Frames and Fields 132.4 Colour Spaces 132.4.1 RGB 142.4.2 YCbCr 152.4.3 YCbCr Sampling Formats 172.5 Video Formats 192.6 Quality 202.6.1 Subjective Quality Measurement 212.6.2 Objective Quality Measurement 222.7 Conclusions 242.8 References 24
3 Video Coding Concepts 273.1 Introduction 273.2 Video CODEC 283.3 Temporal Model 303.3.1 Prediction from the Previous Video Frame 303.3.2 Changes due to Motion 303.3.3 Block-based Motion Estimation and Compensation 323.3.4 Motion Compensated Prediction of a Macroblock 333.3.5 Motion Compensation Block Size 343.3.6 Sub-pixel Motion Compensation 373.3.7 Region-based Motion Compensation 413.4 Image model 423.4.1 Predictive Image Coding 443.4.2 Transform Coding 453.4.3 Quantisation 513.4.4 Reordering and Zero Encoding 563.5 Entropy Coder 613.5.1 Predictive Coding 613.5.2 Variable-length Coding 623.5.3 Arithmetic Coding 693.6 The Hybrid DPCM/DCT Video CODEC Model 723.7 Conclusions 823.8 References 834 The MPEG-4 and H.264 Standards 854.1 Introduction 854.2 Developing the Standards 854.2.1 ISO MPEG 864.2.2 ITU-T VCEG 874.2.3 JVT 874.2.4 Development History 884.2.5 Deciding the Content of the Standards 884.3 Using the Standards 894.3.1 What the Standards Cover 904.3.2 Decoding the Standards 904.3.3 Conforming to the Standards 914.4 Overview of MPEG-4 Visual/Part 2 924.5 Overview of H.264 / MPEG-4 Part 10 934.6 Comparison of MPEG-4 Visual and H.264 944.7 Related Standards 954.7.1 JPEG and JPEG2000 954.7.2 MPEG-1 and MPEG-2 954.7.3 H.261 and H.263 964.7.4 Other Parts of MPEG-4 974.8 Conclusions 974.9 References 98
5 MPEG-4 Visual 995.1 Introduction 995.2 Overview of MPEG-4 Visual (Natural Video Coding) 1005.2.1 Features 1005.2.2 Tools, Objects, Profiles and Levels 1005.2.3 Video Objects 1035.3 Coding Rectangular Frames 1045.3.1 Input and Output Video Format 1065.3.2 The Simple Profile 1065.3.3 The Advanced Simple Profile 1155.3.4 The Advanced Real Time Simple Profile 1215.4 Coding Arbitrary-shaped Regions 1225.4.1 The Core Profile 1245.4.2 The Main Profile 1335.4.3 The Advanced Coding Efficiency Profile 1385.4.4 The N-bit Profile 1415.5 Scalable Video Coding 1425.5.1 Spatial Scalability 1425.5.2 Temporal Scalability 1445.5.3 Fine Granular Scalability 1455.5.4 The Simple Scalable Profile 1485.5.5 The Core Scalable Profile 1485.5.6 The Fine Granular Scalability Profile 1495.6 Texture Coding 1495.6.1 The Scalable Texture Profile 1525.6.2 The Advanced Scalable Texture Profile 1525.7 Coding Studio-quality Video 1535.7.1 The Simple Studio Profile 1535.7.2 The Core Studio Profile 1555.8 Coding Synthetic Visual Scenes 1555.8.1 Animated 2D and 3D Mesh Coding 1555.8.2 Face and Body Animation 1565.9 Conclusions 1565.10 References 1566 H.264/MPEG-4 Part 10 1596.1 Introduction 1596.1.1 Terminology 1596.2 The H.264 CODEC 1606.3 H.264 structure 1626.3.1 Profiles and Levels 1626.3.2 Video Format 1626.3.3 Coded Data Format 1636.3.4 Reference Pictures 1636.3.5 Slices 1646.3.6 Macroblocks 1646.4 The Baseline Profile 1656.4.1 Overview 1656.4.2 Reference Picture Management 1666.4.3 Slices 1676.4.4 Macroblock Prediction 1696.4.5 Inter Prediction 1706.4.6 Intra Prediction 1776.4.7 Deblocking Filter 1846.4.8 Transform and Quantisation 1876.4.9 4 × 4 Luma DC Coefficient Transform and Quantisation(16 × 16 Intra-mode Only) 1946.4.10 2 × 2 Chroma DC Coefficient Transform and Quantisation 1956.4.11 The Complete Transform, Quantisation, Rescaling and InverseTransform Process 1966.4.12 Reordering 1986.4.13 Entropy Coding 1986.5 The Main Profile 2076.5.1 B Slices 2076.5.2 Weighted Prediction 2116.5.3 Interlaced Video 2126.5.4 Context-based Adaptive Binary Arithmetic Coding (CABAC) 2126.6 The Extended Profile 2166.6.1 SP and SI slices 2166.6.2 Data Partitioned Slices 2206.7 Transport of H.264 2206.8 Conclusions 2226.9 References 2227 Design and Performance 2257.1 Introduction 2257.2 Functional Design 2257.2.1 Segmentation 2267.2.2 Motion Estimation 2267.2.3 DCT/IDCT 2347.2.4 Wavelet Transform 2387.2.5 Quantise/Rescale 2387.2.6 Entropy Coding 2387.3 Input and Output 2417.3.1 Interfacing 2417.3.2 Pre-processing 2427.3.3 Post-processing 2437.4 Performance 2467.4.1 Criteria 2467.4.2 Subjective Performance 2477.4.3 Rate–distortion Performance 2517.4.4 Computational Performance 2547.4.5 Performance Optimisation 2557.5 Rate control 2567.6 Transport and Storage 2627.6.1 Transport Mechanisms 2627.6.2 File Formats 2637.6.3 Coding and Transport Issues 2647.7 Conclusions 2657.8 References 2658 Applications and Directions 2698.1 Introduction 2698.2 Applications 2698.3 Platforms 2708.4 Choosing a CODEC 2708.5 Commercial issues 2728.5.1 Open Standards? 2738.5.2 Licensing MPEG-4 Visual and H.264 2748.5.3 Capturing the Market 2748.6 Future Directions 2758.7 Conclusions 2768.8 References 276Bibliography 277Index 279