Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Lesson #8: Test Planning and Strategy

No description
by

Richard Shy

on 25 February 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Lesson #8: Test Planning and Strategy

Test Planning and Strategy
Parts of a Test Plan
1
.
Test Plan Identifier
identifies test plan, its level, and level of software it is related to
means for version tracking

2.
References
all documents that support the test plan
documents that can be referenced include, but are not limited to:
Project Plan
Requirements Specifications
High Level Design Document
Detailed Design Document
Development and Test Process Standards
Methodology Guidelines and Examples
Corporate Standards and Guidelines
Parts of a Test Plan
3.
Introduction
the
executive summary
of the test plan
states purpose, level, and scope of the test plan
may refer to other relevant documents that support the test plan like:
Resource and Budget Constraints
Scope of the Testing Effort
How testing relates to other evaluation activities (Analysis & Reviews)
Process to be used for change control, and communication and coordination of key activities.
4.
Test Items
list of functions (from a technical point of view) intended to be tested within the scope of the test plan
may also include key delivery schedule issues for critical elements
Types of Test Strategies
Conclusion
What is a Test Plan?
the “project plan” for testing work to be done
derived from:
Business Requirements Document
Software Requirements Specification
Use Case Documents
Ask yourself "What?"
How to Choose Your Test Strategy
3. Objectives
gear your testing approach to the objectives that you need to meet
If the objective is to find as many defects as possible with a minimal amount of up-front time and effort invested – for example, at a typical independent test lab – then a dynamic strategy makes sense

4. Regulations
regulations are requirements that you need to meet, along with those of your stakeholders
How to Choose Your Test Strategy
5. Product
Some products like weapons systems and contract-development software tend to have well-specified requirements. This leads to synergy with a requirements-based analytical strategy

6. Business
Business considerations and business continuity are often important. If you can use a legacy system as a model for a new system, you can use a model-based strategy
What is a Test Strategy Document?
a high level document usually developed by the Project Manager
defines software testing approach to achieve testing objectives
static document
sets the standards for testing processes and activities
Test Plans
a set of guidelines that describe test design, and HOW testing is to be performed
the choice of test strategy / approach dictates the success / failure of the test effort, as well as the accuracy of test plans and estimates
Types of Test Plans
1.
Master Test Plan
a single, high-level test plan for a project / product that unifies all other test plans

2.
Level-Specific Test Plans
low-level test plan for each level of testing:
Unit Test Plan
Integration Test Plan
System Test Plan
Acceptance Test Plan

3.
Type-Specific Test Plans
low-level test plan for major types of testing, both functional and non-functional
Regression Test Plan
Performance Test Plan
Security Test Plan
How to Choose Your Test Strategy
1. Risks
consider risks and level of risk in performing risk management
examples:
For a well-established application that is evolving slowly, regression is an important risk, so regression-averse strategies make sense.
For a new application, a risk analysis may reveal different risks if you pick a risk-based analytical strategy.
2. Skills
consider the skills of the team, both possessed and lacking
A standard compliant strategy is a smart choice when you lack the time and skills in your team to create your own approach
Types of Test Strategies
Types of Test Strategies
Objectives
Introduction

Definition of a Test Plan
Types of Test Plans
Importance of a Test Plan
Parts of a Test Plan

Definition of Test Strategy
Types of Test Strategies
How to Choose Your Test Strategy
Test Strategy Outline
Introduction
Different organizations have different practices and standards in implementing Test Planning and Strategy.

Depending on the maturity of the organization, details of Test Plan and Strategy may vary, and sometimes even combined in one document.

Generally, the Test Plan contains details which focus on “
What?
” questions while Test Strategy focuses “
How?
” questions within Software Test Project.
usually prepared by the
Test Lead
or
Test Manager
describes the entire testing process, and
WHAT
is to be tested
Why Write a Test Plan?
Guides thinking and perspective
in terms of incorporating enough testing time in the project schedule

Serves as
means of communication
feedback should drive the creation and modification of the document

Helps to
manage change
keeps testing aligned with project needs
Parts of a Test Plan
Parts of a Test Plan
6.
Features to be tested
list of what is to be tested from a USER’s point of view
set the level of risk per feature, .i.e., high, medium, or low
users do not understand technical terms; they understand functions and processes
7.
Features not to be tested
list of what is NOT to be tested both from a user’s point of view and configuration point of view
not a technical description
identify the reasons why the feature is not to be tested
Not to be included in this release of the software.
Low risk, has been used before and is considered stable.
Will be released but not tested or documented as a functional part of the release of this version of the software.
5.
Software Risk Issues
identify software to be tested and its critical areas
Delivery of a third party product
New version of interfacing software
Ability to use and understand a new package/tool, etc.
Extremely complex functions
Modifications to components with a past history of failure
Poorly documented modules or change requests
identify inherent software risks
complexity
safety
multiple interfaces
impact on client
government regulations
misunderstanding of original requirements
beware of vague requirements and requirements that cannot be tested
Parts of a Test Plan
Parts of a Test Plan
9.
Pass / Fail Criteria
what are the completion criteria for the plan?
examples:
Unit Test Level:
All test cases completed.
A specified percentage of cases completed with a percentage containing some number of minor defects.
Code coverage tool indicates all code covered.
Master Plan Level:
All lower level plans completed.
A specified number of plans completed without errors and a percentage with minor defects.
10.
Suspension Criteria & Resumption Requirements
know when to pause in a series of tests if continuing further has no value
8.
Approach / Strategy
states overall test strategy
identify rules and processes to be followed
answer the following questions:
Are any special tools to be used and what are they?
Will the tool require special training?
What metrics will be collected?
Which level is each metric to be collected at?
How is Configuration Management to be handled?
How many different configurations will be tested?
Hardware
Software
Combinations of HW, SW and other vendor packages
What levels of regression testing will be done and how much at each test level?
Will regression testing be based on severity of defects detected?
How will elements in the requirements and design that do not make sense or are untestable be processed?
Note
“Features to be tested” and “Features NOT to be tested” are directly related to “Software Risk Issues” and “Planning Risks and Contingencies” section.

What will and will not be tested are directly affected by the
levels of acceptable risk
within the project, and what does not get tested affects the level of risk of the project.
Parts of a Test Plan
Parts of a Test Plan
13.
Environmental Needs
any special requirements for the test plan such as:
Special hardware such as simulators, static generators, etc.
How will test data be provided? Are there special collection requirements or specific ranges of data that must be provided?
How much testing will be done on each component of a multi-part feature?
Special power requirements
Specific versions of other supporting software
Restricted use of the system during testing
14.
Staffing and Training Needs
training on the application / system
training for any test tools to be used
15.
Responsibilities
who is in charge?
Parts of a Test Plan
Parts of a Test Plan
18.
Assumptions and Dependencies
list all assumptions made during the plan, as well as dependencies

19.
Approvals
identify who can approve the process as complete and allow the project to proceed to the next level

20.
Glossary
used to define terms and acronyms used in the document, and testing in general, to eliminate confusion and promote consistent communication

11.
Test Deliverables
what is to be delivered as part of this plan?
Test plan document
Test cases
Test design specifications
Tools and their outputs (If required and applicable)
Simulators (If required and applicable)
Static and dynamic generators (If required and applicable)
Error logs and execution logs
Problem reports and corrective actions
12.
Remaining Tasks
Multi-phase / Incremental Project
identify remaining tasks to avoid confusion should defects be reported on future not-yet-developed functions
Multi-party Project / Process
identify accountability for functions so that responsibility for defects may be traced properly
16.
Schedule
should be based on realistic and validated estimates
identify how to handle schedule slippages
all relevant milestones should be identified with their relationship to the development process

17.
Planning Risks and Contingencies
identify overall risks to the project with emphasis on the test process, and specify actions to be taken in response to various events
if X happens, the following actions will be taken…
1.
Analytical
uses formal or informal analytical techniques, usually during requirements and design stages
Risk-based
performing risk analysis using projects documents and stakeholder input in planning, estimating, designing and prioritizing tests based on risk
Requirements-based
analysis of requirements specification forms the basis for planning, estimating, and designing tests
2.
Model-Based
involves the creation or selection of formal/informal models for critical system behaviors during requirements and design stages
example:
building (mathematical) models to which the system under test is compared with
if the system conforms to what the model predicts, then the system is deemed working
3.
Methodical
methodically designing, implementing, and executing tests following a specific industry standard for software quality, or some defined outline of things needed to be done
example:
ISO 9126
4.
Process or Standard Compliant
relies on an externally developed approach to test, often with little to no customization
may have early or late point of involvement for testing
example:
Agile’s Extreme Programming
5.
Dynamic
involves concentrating efforts to finding as many defects as possible during test execution and adapting to the realities of the system under test upon delivery based on a lightweight set of guidelines
emphasizes later stages of testing
example: exploratory testing
6.
Consultative or Directed
relies on a group of non-testers to guide or perform testing efforts
emphasizes later stages of testing simply due to the lack of recognition of the value of early testing
example:
asking users or developers for instructions on what and how to test
relying on users and/or developers to do testing

7.
Regression-averse
involves creating a set of automated procedures that allow for the detection of regression defects
automated tests are (re)run every test to ensure that nothing has broken
Preventive vs. Reactive
Some of these tests are
preventive
, while some are
reactive
.

Preventive
identifies problems prior to test execution
allows early and cheap removal of bugs

Reactive
focuses on test execution period
allows location of defects and defect clusters that might have been hard to anticipate until you have the actual system on your hands

Don’t view these as either-or strategies. Instead,
adopt whatever approach suits your needs
as a team in a particular situation.

`
If included in your test plan, this is your overall test strategy for this test plan; it should be appropriate to the level of the plan (master, acceptance, etc.) and should be in agreement with all higher and lower levels of plans. Overall rules and processes should be identified.

Are any special tools to be used and what are they?
Will the tool require special training?
What metrics will be collected?
Which level is each metric to be collected at?
How is Configuration Management to be handled?
How many different configurations will be tested?
Hardware
Software
Combinations of HW, SW and other vendor packages
What levels of regression testing will be done and how much at each test level?
Will regression testing be based on severity of defects detected?
How will elements in the requirements and design that do not make sense or are untestable be processed?
Specify if there are special requirements for the testing.
Only the full component will be tested.
A specified segment of grouping of features/components must be tested together
If implemented in a separate document(or for a large scale of project), here are the common components outlined in the Test Strategy Document specifically describing the strategy or approach.

Scope and Objectives
Business Issues
Roles and Responsibilities
Communication and Status Reporting
Test Deliverability
Industry Standards to Follow
Test Automation and Tools
Testing Measurements and Metrics
Risks and Mitigation
Defect Reporting and Tracking
Change and Configuration Management
Training Plan
Test Strategy or Test Approach Outline
Difference between Test Plan and Test Strategy
Different types of Test Plans
what they are
when to use them
Different types of Test Strategies
how to choose
outline
Full transcript