Contents Foreword by Admiral Edmund P. Giambastiani, Jr.
Foreword by Dr. William Nylin, Jr.
Preface
Acknowledgments
bout the Authors
Part One What Is Automated Software Testing and Why Shoul We Automate?
1 What Is Effective Automated Software Testing (AST)?
1.1 Automated Software Testing Definition
1.2 Automated Software Testing Recipes
1.3 Advances in AST Technologies
1.4 Automating Various Software Testing Types
1.5 Providing AST-Based Production Support
1.6 Automating Standards Assessments
2 Why Automate?
2.1 The Challenges of Testing Software Today
2.2 Reducing the Time and Cost of Software Testing
2.3 Impacting Software Quality
2.4 Improvements to Your Software Test Program
3 The Business Case
3.1 Definition of the Business Case
3.2 Identifying the Business Needs
3.3 Justifying Automation in Terms of Cost and Benefits
3.4 Risks
3.5 Other Considerations
4 Why Automated Software Testing Fails and Pitfalls to Avoid
4.1 R&D Does Not Generally Focus on Automated or Manual Testing Efforts
4.2 AST Myths and Realities
4.3 Lack of Software Development Consideration for AST
4.4 The Forest for the Trees-Not Knowing Which Tool to Pick
4.5 Lack of Automaton Standards across Tool Vendors
4.6 Lack of Business Case
Part Two How to Automate: Top Six Keys for Automation Payoff
5 Key 1: Know Your Requirements
5.1 Understand the AST Support Requirements
5.2 Additional Information in Support of AST Requirements
5.3 When Information Is Not Available
5.4 Start Implementing Your Requirements Traceability Matrix(RTM)
6 Key 2: Develop the Automated Test Strategy
6.1 The AST Strategy Document
6.2 Scope and Automated Test Objectives
6.3 Identify the Approach
6.4 Automated Software Test Framework(ASTF)
6.5 AST Environment/Configuration
6.6 Automating the RTM
6.7 Automated Defect Tracking
7 Key 3: Test the Automated Software Test Framework (ASTF)
7.1 Verify That the ASTF Meets Specified Requirements and That Features Behave As Expected
7.2 Peer-Review All ASTF-Related Artifacts, Including Design,Development, and Test Cases
7.3 Verify Requirements and Coverage
7.4 Hold a Customer Review
8 Key 4: Continuously Track Progress--and Adjust Accordingly
8.1 AST Program Tracking and Defect Prevention
8.2 AST Metrics
8.3 Root Cause Analysis
9 Key 5: Implement AST Processes
9.1 AST Phases and Milestones
9.2 AST Phase 1: Requirements Gathering-Analyze Automated testing Needs
9.3 AST Phase 2: Test Case Design and Development
9.4 AST Phase 3: Automated Software Testing Framework (ASTF) Test Script Development
9.5 AST Phase 4: Automated Test Execution and Results Reporting
9.6 AST Phase 5: Program Review and Assessment
9.7 Virtual Quality Gates
9.8 Process Measurement
10. Key 6: Put the Right People on the Project-Know the Skill Sets Required
10.1 Program Management
10.2 Systems Engineering
10.3 Software Development
10.4 Configuration Management
10.5 Quality Assurance
10.6 Subject Matter Experts(SMEs)
Appendices A Process Checklist
A.1 AST Phase 1: Requirements Gathering-Analyze Automated testing Needs
A.2 AST Phase 2: Test Case Design and Development
A.3 AST Phase 3: Automated Software Testing Framework (ASTF) and Test Script Development
A.4 AST Phase 4: Automated Test Execution and Results Reporting
A.5 AST Phase 5: Program Review and Assessment
B. AST Applied to Various Testing Types
B.1 Security Testing
B.2 Soak Testing
B.3 Concurrency Testing
B.4 Performance Testing
B.5 Code Coverage Testing
B.6 Unit Testing
C The Right Tool for the Job
C.1 Requirements Management (RM)
C.2 Unit Testing Frameworks--Example Evaluation
C.3 Configuration Management--Example Evaluation
C.4 Defect Tracking--Example Evaluation
C.5 Security Testing
C.6 Automated Software Testing Framework (ASTF)-Example Evaluation
C.7 Other STL Support Tools
D Case Study: An Automated Software Testing Framework(ASTF) Example
D.1 Key Design Features
D.2 Test Manager
D.3 More on Automated Test Case and Test Code Generation
D.4 Results Reporting
D.5 Automated Defect Reporting
Contributing Authors
Index