(More of this stuff? Wake me up when it’s over. Ed)

Experienced-based testing
- Strengths
— Works in short timescales
— Enables reactive and flexible testing (tester learns from results of previous tests)
— Often applied where there is minimal documentation
- Weaknesses
— Poorly documented
— Relies on experience of tester
— Unpredictable application by different testers
- Examples
— Exploratory testing
— Error guessing

Scripted vs unscripted testing
- Exploratory testing, while often considered an unscripted approach, is a systematic technique, while ad-hoc testing is the testing equivalent of ‘hacking’.

Non-functional testing
- Strengths
— Evaluates quality characteristics
- Weaknesses
— Often overlooked
— Poorly understood
— Often needs specialists
- Examples
— Performance testing
— Usability testing
— Reliability testing

Specifying Software Reliability
- Probability of Failure on Demand (POFOD)
— the likelihood that the system fails on demand
— POFOD = 0.01 (or 1% means that at least 99 out of every 100 requests must be dealt with successfully
— The Sizewell B Primary Protection System has a requirement of POFOD = 0.001
- Mean Time to Failure (MTTF)
— the time between system failures
— MTTF = 10 to the power 3 hours means that the average time between failures is no less than 1000 hours
— the Flight Control Software for the Airbus A320 and Boeing 777 has a requirement of MTTF of 10 to the power 9 hours

TPI
- Based on:
— Experience
— TMap
— Other process improvement models
— Boehm’s 1979 rework stats
- “..more improvement steps, practical details and instructions”
- Continuous model
— equivalent to a 13 stage model
- Includes optional improvement suggestions
- Survey results (113 responses)
— 81% felt TPI had a positive effect
— 66% felt TPI resulted in a product with less failures

TPI Structure
Key Areas -> Levels -> Checkpoints or Improvement Suggestions

TPI Assessment Activities
Example TPI checkpoint:
- Level A
— A test strategy is defined for a single high-level test
- Checkpoints – Level A
— A motivated product risk analysis
— There is a variation in test depth, depending on risks
— One or more test design techniques are used, in line with required depth of test
— A re-test strategy is defined

Example TPI improvement suggestions:
- Level A
— A test strategy is defined for a single high-level test
- Improvement suggestions Level A
— Involve proper parties when defining test strategy
— If only one test design technique available, make variations to vary depth of rigour
— Define a re-test strategy, consider full re-test, a thin re-test, or even no re-test
— Identify subsystems and quality characteristics, with defined relative importance

‘Ideal’ Test Process Improvement
- Easy to use
- Continuing publicity
- Publicly available
- Available consultants
- Not marketing
- Accepted by professional bodies
- Include provision for improvement
- Provide many small evolutionary improvements
- Sound basis:
— Practical
— Empirical
— Theoretical
— Published and justified
- Detail of how to:
— Assess
— Identify improvements
— Make improvements
- Quantifiable improvements
- Tailorable (project specific)

Comparison of TPI and TMMI:
TMMi will look at the whole organisation, strictly staged improvement jumps levels 1-5
TPI allows improvements of parts of the organisation, small continuous improvement moving A-B-C-D with a quality profile.

Maturity Levels -> Process Areas -> Specific Goals -> Specific Practices or
Maturity Levels -> Process Areas -> Generic Goals -> Generic Practices

TMMi Define the test approad
1. Sekect the test design techniques to be used
2. Define the approach to reviewing test work products
3. Define the approach for re-testing
4. Define the approach for regression testing
5. Define the supporting test tools to be used
6. Identify significant constraints regarding the test approach
7. Align the test approach with the defined organization-wide or programme-wide test strategy
8. Identify any non-compliances to the test stratgey and its rationale
9. Review the test approach with stakeholders
10 Revise the test approach as appropriate

TMMI Assessment Activities
Preparation Phase -> Meeting Phase -> Reporting Phase