
Systems Testing and Quality Assurance |
|
Course Topics
Introduction
- What is quality?
- What causes quality problems?
- Definition of errors (per IEEE)
- Sources by phase of systems life-cycle
- What are the costs associated with a lack of quality?
- What are the costs associated with having quality?
- DIRFT: Doing It Right The First Time
Systems Life-Cycle Concepts
- Phases
- Costs by phase
- Establishing quality before design and coding
- Defining testable requirements
- Testing the concept-of-operations
- Planning for systems acceptance
- Using CASE technology
- The role of testing and debugging
- Configuration management
- Baseline concept
- Techniques for controlling changes
- Adjusting to changing requirements
- Documentation requirements
Testing Concepts
- Definition
- Economics of testing
- Relating priorities to risks
- Who does what testing
- Monitoring test effectiveness
- Paradox: purpose vs. commonly used measure of success ("no errors")
- Use of test matrices
Test-Case Design
- Defining specific test objectives
- Black box vs. white box techniques
- Black box techniques
- Equivalence partitioning
- Boundary-value analysis
- Cause-effect graphing
- Error guessing
- White box techniques
- Statement coverage
- Decision coverage
- Condition coverage
- Decision/condition coverage
- Multiple-condition coverage
- Using the techniques together for maximum effectiveness
- Generating test data
- Assembling the test cases into test specifications
Testing Strategies (Discussion of Advantages and Disadvantages)
- Big bang vs. incremental
- Top‑down vs. bottom-up
- Modified top-down
- How strategy affects test-case design
- Error seeding
- Regression Testing
- Definition
- Why it is critical
- When to do it
- Testing prepackaged software
Types of Testing
- Module testing and integration
- Smoke testing
- Procedure testing
- Function testing
- System testing
- Volume
- Stress
- Usability
- Recovery
- Sewiceabiiity
- Etc.
- Acceptance testing
- Installation testing
Creating the Test Plan
- Using a phased approach (affected by strategy employed)
- Contents
- Test specifications
- Objectives
- Completion criteria
- Schedules/cycles
- Responsibilities
- Methods/tools to be used
- Test-case libraries to be used
- Standards to be followed
- Computer resources
- Tracking procedures and error disposition
- Debugging procedures
- Reporting requirements
- Determining test-completion criteria (i.e. knowing when to stop)
- Who is to create the plan?
- When is the plan to be created?
Inspections, Walkthroughs, Types
- The concept of Peer Reviews
- Inspections
- Structured Walkthroughs
- When to use: determination of review milestones and objectives
Debugging
- Problem recognition and identification
- Specifying the problem or defining the deviation
- Searching for distinctions
- Generating possible causes: what's changed?
- Deducing the most probable cause(s)
- Testing for true “cause(s)"
Fixing Errors
- Traps and pitfalls
- How to test fixes
- The role of regression and smoke testing
- Designing changes into existing programs
- Error analysis: a tool for improvement
Getting Better
- Using the test plan
- Developing system metrics
- Monitoring system performance versus system objectives
|