Contents
Preface
Chapter 1 Introduction 1
1.1 Introduction 1
1.2 Research Problems and Motivations 5
1.2.1 Research Problems 5
1.2.2 Research Motivations 6
1.3 Aims and Contributions 10
1.4 The Outline 11
Chapter 2 Context-aware Sensing and Multisensor Data Fusion 14
2.1 Context-aware Sensing 14
2.1.1 Location-aware Sensing 16
2.1.2 Positioning Technologies 22
2.1.3 Spatial Interaction 25
2.2 Human Motion Capture and Analysis 27
2.2.1 Human Motion 28
2.2.2 Human Motion Capture Systems 29
2.2.3 Human Motion Analysis 35
2.3 Multisensor Data Fusion 36
2.3.1 Introduction 36
2.3.2 Probabilistic Approaches 37
2.3.3 Bayesian Filters and Sensor Fusion 38
2.4 Gaussian Processes and Sensor Fusion 40
2.4.1 Gaussian Processes 41
2.4.2 Sensor Fusion with Gaussian Processes 45
2.5 Conclusions 45
Chapter 3 Sensor Fusion with Multi-rate Sensors-based Kalman Filter 47
3.1 Introduction 47
3.2 The Kalman Filter and Multi-rate Sensors-based Kalman Filter 49
3.2.1 Background 49
3.2.2 Sensor Fusion with Multi-rate Sensors-based Kalman Filter 50
3.3 System Overview 53
3.3.1 Sensor Noise Characteristics 53
3.3.2 The Coordinate Systems 53
3.3.3 The Multi-rate Sensors-based Fusion System 56
3.4 Inertial Sensor Fusion 58
3.4.1 Orientation Estimation 58
3.4.2 Experiment: Comparison of Acceleration Estimated with Kinect Sensor and Inertial Sensors 60
3.5 Experiment: Fusing Kinect Sensor and Inertial Sensors with Multi-rate Sensors-based Kalman Filter 68
3.5.1 Experimental Set-up 68
3.5.2 Experiment Design 68
3.5.3 Position Estimation 69
3.5.4 Velocity Estimation 72
3.5.5 Acceleration Estimation 73
3.5.6 Conclusion 74
3.6 Conclusions 74
Chapter 4 The Sensor Fusion System 76
4.1 Introduction 77
4.1.1 Hand Motion Tracking with Kinect Sensor and Inertial Sensors 78
4.1.2 Challenges 79
4.1.3 Applications 80
4.2 System Overview 81
4.2.1 Augmenting the Kinect System with SK7 82
4.2.2 Augmenting the Kinect System with a Mobile Phone 82
4.3 Gaussian Process Prior Model for Fusing Kinect Sensor and Inertial Sensors 84
4.3.1 Problem Statement for Dynamical System Modelling 84
4.3.2 Transformations of GP Priors and Multi-rate Sensor Fusion 89
4.4 Alternative View of the Sensor Fusion—Multi-rate Kalman Filter 97
4.5 Experiment 101
4.5.1 Experiment Design 101
4.5.2 Experimental Method 102
4.5.3 Experimental Results 103
4.5.4 Conclusion 107
4.6 Conclusions 108
Chapter 5 Transformations of Gaussian Process Priors for User Matching 110
5.1 Introduction 110
5.2 Background 113
5.3 Fusing Kinect Sensor and Inertial Sensors for User Matching 114
5.3.1 Problem Statement for User Matching with GP Priors 115
5.3.2 Multi-rate Sensor Fusion for User Matching 116
5.4 User Matching System Overview 118
5.5 Simulation Experiment: Estimation of Position, Velocity and Acceleration with GP Priors 119
5.6 The User Matching Experiment I: Subtle Hand Movement 122
5.6.1 Experiment Design 122
5.6.2 Experimental Results 122
5.6.3 Summary of Subtle Movement Results 134
5.7 The User Matching Experiment II: Mobile Device in User's Trouser Pocket 134
5.7.1 Experiment Design 134
5.7.2 Experimental Results 136
5.7.3 Summary of Device-in-pocket Results 141
5.8 The User Matching Experiment III: Walking with Mobile Device in the Hand 141
5.8.1 Experiment Design 141
5.8.2 Experimental Results 141
5.8.3 Summary of Walking Results 148
5.9 Discussions 148
5.10 Conclusions 150
Chapter 6 Experiment—User Performance Improvement in Sensor Fusion System 153
6.1 Introduction 153
6.2 Background 155
6.2.1 Feedback Control System 155
6.2.2 Visual Feedback 156
6.3 Augmenting the Kinect System with Mobile Device in Spatially Aware Display 157
6.3.1 System Overview 157
6.3.2 Augmenting the Kinect System with a Mobile Device (N9) 158
6.4 Experiment: User Study—Trajectory-based Target Acquisition Task 162
6.4.1 Participants and Apparatus 162
6.4.2 Data Collection and Analysis 162
6.4.3 Experiment Design 163
6.4.4 Experimental Results 164
6.4.5 Conclusion 168
6.5 Conclusions 169
Chapter 7 Conclusions 171
7.1 Sensor Fusion with Multi-rate Sensors-based Kalman Filter 172
7.2 The Sensor Fusion System 173
7.3 First Application—User Matching and Identiˉcation 175
7.4 Second Application—Position Stabilisation and Lag Reduction 176
7.5 Combination of Two Applications in Proxemic Interaction 178
Appendix A Acronyms 180
References 181
Index 195