What type of content headings we might need. The following list provides possible headings that could be used to support the development of a Test Strategy. It is highly unlikely that all headings would be needed in any one Test Strategy. Text in italics is from the Test Policy document.

1. Objectives of Testing
Support for risk mitigation
An objective of testing is to provide sufficient information to determine the current quality of the system under test. As such all activities aimed at achieving this are considered to be software testing activities e.g. reviews, static analysis, integration, system, acceptance and regression testing.

2. Scope of Testing
All software that is delivered to customers will be tested

3. Test Methodology

4. Test Process
The organisation will follow the test process defined in ISO 29119-2
The organisation will follow the static test process defined in IEEE 1012

5. Tester Responsibility
Decide which tasks testers should perform

6. Degree of Independence
All software will be tested by a testing group that is technically, managerially and financially independent from the development group
The software testing shall be based on the Fundamental Test Process and be aligned with the development approach, but performed by an independent group within the organisation.

7. Test Organisation Structure
Testing shall be resourced within the organisation from a central pool of testers who may be assigned to one or more projects at a time. In addition, a central ‘expert’ software testing resource led by the Head of Testing shall provide test consultancy services to projects as necessary.

8. Tester Education
All testers must be able to program
All testers must be ISTQB Foundation certified or must have achieved an equivalent qualification or be able to demonstate an equivalent level of experience
All members of testing teams are expected to have achieved the minimum level of ISEB qualification in software testing (or equivalent) within 6 months of joining a testing team.

9. Test Process Improvement
Test Process Improvement shall be performed and is the responsibility of the Software Testing Team. TPI shall be used in this area. The organisation shall improve in no less than three key areas annually.

10. Standards
ISO 29119 will be followed for all test documentation produced within the organisation
Where relevant international (ISO/IEC) software testing standards are available then these standards will be used on all projects. Identification of these relevant standards is the responsibility of the Software Testing Team, as is checking compliance of projects with these standards.

11. Testing Lifecycle Model
Testing will follow a defined lifecycle model that will be aligned with the development lifecycle model

12. Other Organisational Strategies

13. Measuring the Value of Testing
The organisation will measure the return on investment of testing

14. Test Types
Functional (white-box testing, black-box testing etc), non-functional (reliability testing, usability testing etc)

15. Test Techniques

16. Test Levels

17. Entry & Exit Criteria

18. Test Completion Criteria

19. Retesting & Regression Testing

20. Test Selection/Prioritisation

21. Test Environments

22. Test Asset Reuse

23. Test Automation & Tools
The approach to test automation
Which testing tools are to be used

24. Generic Risks to be Addressed

25. Approach to incident management (not actually in the draft ISO 29119, but is in the ISEB syllabus).

(Good headings but the content’s a bit thin. Ed)

Now we will consider how we might add detail to the Test Policy in a top-down manner to create a first draft Organisational Test Strategy, that will, due to the top-down approach, be exactly aligned with the Test Policy.

There are three ways of generating a Test Strategy: top-down, bottom-up and a mixture of the two. A top-down approach bases the strategy on high-level requirements i.e. what should happen, while a bottom-up approach bases the strategy on what is already happening on projects and extracts the common themes. Idealists genrally favour the top-down approach, while consevatives would probably favour the bottom-up approach as this involves the minimum amount of change, and perhaps effort, by those who have to implement the strategy. Pragmatics would go for a mix of the two.

Working from the Test Policy will enable a first draft to be created, but the result is unlikely to be the finished article, as Orgasitional Test Strategies will also typically include some content not directly related to Test Policy. The next stage will be to go through the list of 25 Organisational Test Strategy headings, as suggested by the ISO 29119 Software Testing standard, and brainstorm how these apply to the organisation. Having worked from the Test Policy and the list of headings then enough information should have been collected to allow the Test Strategy to be defined.

A list of the 25 Organisational Test Strategy headings, as suggested in ISO 29119, are in the following entry.

Thought I’d write up what I’m learning about the ISEB Practitioner Certificate and this blog seems as good a place as any.

Today it’s the initial testing document that all companies should have, the company wide Test Policy document (Buried in a filing cabinet in the basement or on a companies intranet site with no links to it. Ed)

A Test Policy is:



The ISO Working Group Test Policy Rules are:



Other considerations
Place in hierarchy
Top level planning document – all subsequent planning documents should implement the test policy or justify why they don’t. The policy will be driven by business risk and the organization’s business culture. It may not always be formally documented.

Role/Purpose
To describe the organizations philosophy towards software testing so that all members of the organization know what is expected (this is particularly important to newcomers and 3rd parties). To describe the test-related activities that are expected to happen as a matter of course in all projects.

Content
Can include: Definition of testing, the test process to be used, what is meant by ‘effective’ testing and how it is evaluated, the level of software quality to be achieved and the organisational approach to test process improvement.

Benefits/Pitfalls
Benefits – everyone knows what is expected of them – both in-house and 3rd parties.
Pitfalls – The policy doesn’t always exist and may divert resources on creating one if it doesn’t exist. The policy may be ignored if there is no will in the organization to implement it. It may be out of synch with the business drivers of an organization.

Example Test Policy document
The headings have been taken from the current draft of the ISO 29119 Software Testing Standard which is due in 2011.

Begin Test Policy document————————————————————————————
Objectives of Testing
An objective of testing is to provide sufficient information to determine the current quality of the system under test. As such all activities aimed at achieving this are considered to be software testing activities (e.g. reviews, static analysis, integration, system, acceptance and regression testing).

Scope of Testing
All software that is delivered to customers will be tested.

Test Process
The organization will follow the test process defined in ISO 29119-2.

Test Levels
The levels of testing to be undertaken e.g. unit testing, system testing.

Success Factors
The number of issues outstanding on release, code and coverage requirements to be achieved.

Degree of Independence
The software testing shall be based on the Fundamental Test Process and be aligned with the development approach, but performed by an independant group within the organization.

Test Organisation Structure
Testing shall be resourced within the organization from a central pool of testers who may be assigned to one or more projects at a time.In addition, a central ‘expert’ software testing resource led by the Head of Testing shall provide test consultancy services to projects as necessary.

Tester Education



Test Process Improvement
Test process improvement shall be performed and is the responsibility of the Software Testing team. TPI or TMMI shall be used in this area. The organization shall improve in no less than three key areas annually.

Standards
Where relevant international software or systems testing standards are available then these standards will be used on all projects. Identification of these relevant standards is the responsibility of the Software Testing Team, as is checking compliance of projects with these standards.

Testing Lifecycle Model
Testing will follow a defined lifecycle model that will be aligned with the development lifecycle model.

Tester Ethics
Testers shall follow the same code of ethics as other IT staff within the organization as outlined in the organization’s IT Code of Practice.

Other Organisational Policies
Any other relevant organisation policies.

Measuring the Value of Testing
The organization will measure the return on investment of testing.

End Test Policy document————————————————————————————–

What a tedious blog post (Yep, I think you have done that subject to death. Ed).