Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

Testing

Week 4

Sheetal S

Principles

TDD and BDD

Principles of Software testing

Rules by BCS

Exhaustive testing is impossible.

Defect Clustering

Pesticide Paradox

Testing shows presence of Bugs

Early Testing

Absence of error is Fallacy

Testing is context dependent

Principles of Software testing

Rules by BCS

1 Exhaustive testing is impossible

Exhaustive testing is not possible. Instead, we need the optimal amount of testing based on the risk assessment of the application.

Real time implementation is a problem, time consuming, cannot test all data. so to avoid it we apply testing technique.

And the million dollar question is, how do you determine this risk?

To answer this let's do an exercise

In your opinion, Which operation is most likely to cause your Operating system to fail?

I am sure most of you would have guessed, Opening 10 different application all at the same time.

So if you were testing this Operating system, you would realize that defects are likely to be found in multi-tasking activity and need to be tested thoroughly

Principles of Software testing

Rules by BCS

2 Defect Clustering

Defect Clustering which states that a small number of modules contain most of the defects detected. This is the application of the Pareto Principle to software testing: approximately 80% of the problems are found in 20% of the modules.

By experience, you can identify such risky modules. But this approach has its own problems

If the same tests are repeated over and over again, eventually the same test cases will no longer find new bugs.

Principles of Software testing

Rules by BCS

3 Pesticide paradox

Repetitive use of the same pesticide mix to eradicate insects during farming will over time lead to the insects developing resistance to the pesticide Thereby ineffective of pesticides on insects. The same applies to software testing. If the same set of repetitive tests are conducted, the method will be useless for discovering new defects.

To overcome this, the test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects.

Testers cannot simply depend on existing test techniques. He must look out continually to improve the existing methods to make testing more effective. But even after all this sweat & hard work in testing, you can never claim your product is bug-free.

Principles of Software testing

Rules by BCS

4 Testing shows presence of Bugs

Testing talks about the presence of bugs and don’t talk about the absence of bugs. i.e. Software Testing reduces the probability of undiscovered bugs remaining in the software but even if no bugs are found, it is not a proof of correctness.

But what if, you work extra hard, taking all precautions & make your software product 99% bug-free. And the software does not meet the needs & requirements of the clients.

Principles of Software testing

Rules by BCS

5 Absence of error is Fallacy

It is possible that software which is 99% bug-free is still unusable. This can be the case if the system is tested thoroughly for the wrong requirement. Software testing is not mere finding defects, but also to check that software addresses the business needs. The absence of Error is a Fallacy i.e. Finding and fixing defects does not help if the system build is unusable and does not fulfill the user's needs & requirements.

Principles of Software testing

Rules by BCS

6 Early Testing

Testing should start as early as possible in the Software Development Life Cycle.

So that any defects in the requirements or design phase are captured in early stages. It is much cheaper to fix a Defect in the early stages of testing.

But how early one should start testing? It is recommended that you start finding the bug the moment the requirements are defined.

Principles of Software testing

Rules by BCS

7 Testing is context dependent

Testing is context dependent which basically means that the way you test an e-commerce site will be different from the way you test a commercial off the shelf application. All the developed software’s are not identical. You might use a different approach, methodologies, techniques, and types of testing depending upon the application type. For instance testing, any POS system at a retail store will be different than testing an ATM machine.

Difference Between

TDD and BDD

BDD

TDD

  • Behaiviour Driven Development
  • Test Driven Development
  • BDD gives clear understanding as to what the sytem should do from the prospective of developer and customer

  • TDD is a process of writing tests before the code and making the test fail. change the code so that the test can pass and at least re-factor the code as per the requirement and make sure the test passes again
  • TDD focus on implementation aspect of the system
  • BDD focuses on behavioured aspect of the system, exhibits then implements details of it

  • TDD only gives the developer an understanding of what the system should do
  • BDD is where the step of making it more customer focus comes in

  • TDD continues on until the integration tests
  • BDD is usually written in very ennglish like language and very frequently with further language like cucumber, specflow to make it easier for known tester

Types Of Review

Code review

Pair programming

Inspection

Walkthrough

Technical review

Types Of Reviews

Code Review

Code review is systematic examination (often as peer review) of computer source code.

Code Review

  • Systematic examination (often as peer review) of computer

source code intended to find and fix mistakes overlooked in

the initial development phase, improving both the overall

quality of software and the developers' skills.

• Code reviews can often find and remove common

vulnerabilities such as format string exploits, race conditions,

memory leaks and buffer overflows, thereby improving

software security.

More about

Pair programming

Is a type of code review where two persons

develop code together at the same workstation.

Pair Programming

  • Two programmers work together at one keyboard.

One types in code, the other reviews each line of code as it's typed

The person typing is called the driver

The person reviewing the code is called the observer or

navigator.

The two programmers switch roles frequently.

  • While reviewing, the observer also considers the strategic

direction of the work, coming up with ideas for improvements

and likely future problems to address.

  • Freeing the driver to focus all of his or her attention on the

"tactical" aspects of completing the current task, using the

observer as a safety net and guide.

More about

Benefits

• Design quality

• Reduced cost of development

• Learning and training

• Overcoming difficult problem

• Improved morale

• Decreased management risk

• Increased discipline and better time management

• Resilient flow

• Fewer interruptions

• Fewer workstations required

Drawbacks

• Developer egos

• Developer intimidation

• Developer work preference

• Tutoring cost

• Potential conflict

• Chat sessions

Benefits and Drawbacks

Inspection

Is a very formal type of peer review where the

reviewers are following a well-defined process to find defects.

Inspection

  • Refers to peer review of any work product by trained

individuals who look for defects using a well defined

process.

• An inspection might also be referred to as a Fagan

inspection after Michael Fagan, the inventor of the

process

More about

Walkthrough

Is a form of peer review where the author leads

members of the development team and other interested parties through a software product and the participants ask questions and make comments about defects.

Walkthrough

Technical Review

Is a form of peer review in which a team of

qualified personnel examines the suitability of the software product for its intended use and identifies discrepancies from specifications and standards.

Technical Review

Requirement Traceability Matrix

RTM

RTM

More About

Requirement Traceability Matrix (RTM) is a table (mostly a spreadsheet) which shows if each requirement has a respective Test case / cases to make sure if the requirement is covered for testing.

It is basically used to ensure that ALL the requirements and Change Requests are or will be tested.

Advantages of Requirement Traceability Matrix (RTM):

Gives Overview of ALL the requirements

Shows how requirements are linked to Test Cases

Makes sure 100% coverage of requirements

Easy to prepare

No special tool is required

  • Get all available requirement documents. For eg. Business Requirement Document(BRD), Functional Requirement Document(FSD), Technical Requirement Document(TSD)
  • First list down All the requirements from BRD one by one with requirement ID#
  • Now go to FSD, and list all respective functional requirements for each Business Requirements
  • Open Test Scenario or Test Case document and link available TC IDs to respective Functional Requirements

How to create a matrix

Project: Online Flight Booking Application

Lets create a RTM for this project

Example

Business Requirement Document (BRD) :

This document is provided by Client with high level business Requirements. Suppose for Flight Booking Application it shows below 2 requirements

BR_1 Reservation Module :

It should allow user to book one or more tickets, one way or round way for future dates

BR_2 Payment Module:

User should able to make payment for booked tickets via Credit / Debit Card or through Reward Points

BRD

Functional Specification Document (FSD) :

This document is prepared by Technical team which further elaborate business requirements into functional requirements that can be implemented in a software.

Suppose above 2 business requirements in BRD have more detailed functional requirements:

BR_1 Reservation Module :

FR_1 : One Way Ticket booking

It should allow user to book one way ticket

FR_2 Round Way Ticket

It should allows user to book round way ticket

FR_3 Multicity Ticket booking

It should allows user to book one way or round way ticket for multiple cities

BR_2 Payment Module:

FR_4: By Credit Card

It should allows user to make payment by Credit Cards

FR_5 By Debit Card

It should allows user to make payment by Debit Cards

FR_6 By Reward Points

It should allows user to make payment by Reward Points

FRD

And you have written some test cases or test scenarios for each functional requirement.

Test Case ID Test Case Description

TC#001 Verify if user is able to book

one way ticket

TC#002 Verify if user is able to nook multiple

one way ticket

TC#003 Verify is user is able to book round

way ticket

Test Case

Requirement Traceability Matrix

BRD FSD Test Case Document

BR_1 Reservation Module FR_1 One way ticket booking TC#001

FR_2 Round way ticket TC#002

BR_2 Payment Module FR_4 By Credit card TC#009

RTM

Mobile Testing

Mobile Testing

Mobile Applications

Native

Mobile Web

Hybrid

Mobile Operating system

Mobile Applications

IOS

Android

Windows

Blackberry

Native App

Native

They are the applications which are downloaded from the app store or any web store. It will be developed separately for each platform.

Mobile app has to be fast and rich looking. Native mobile app can access all the hardware and data from the mobile, camera,contacts, etc.

They have high cost of maintenance.

They sit on the device and use the phone memory.

They can provide the flexibility to use without internet

Mobile Web Apps

Access website from mobie device as safari, chrome, Android web.

We need internet to access mobile web app

Mobile web pages are developed through HTML, CSS, Java script

They can fit into any sizes(5s, 6s, 8, 10) based on resolution it can fit in the device size. You cannot access contacts and other things

Hybrid Applications

Hybrid Applications

Hybrid apps are also available on your app store same like native apps.

They are part native and part mobile.

They are available in both ios and android.

They are compatible in both operating system of mobiles.

They are HTML, Native both mixed.

By using hybrid app tool, some native app can also be written. They can add some native app features to app store

Tools used to develop hybrid apps are - Phone Gap, Sencha touch

In house app

These apps are not available in apps store, but they can be downloaded by their website to limited user

In house apps

Emulators

Simulators

Real Devices

Simulators and Emulators

EMULATORS, SIMULATORS, OR REAL DEVICES

About

These tools can be categorised into three broad categories: emulators, simulators, or real devices. Well, few may say that emulators and simulators can be clubbed in a single group. But, real device testing does what the name implies – it tests the application as they run on the users’ smartphones.

Let’s take a deeper look at the categories.

Real device mobile testing requires the use of the actual smartphone. But emulators and simulators are based on the concept of virtual testing. Virtual testing involves testing the software that provides the same functionality as provided by the real phone.

For testing mobile apps, an emulator is a desktop application that mimics the hardware and OS of the applications that should be tested. A simulator does not mimic the hardware/OS, but rather it mimics the basic behavior of a device. While simulators are usually simpler in function than emulators, they are not as useful as emulators. Real device testing checks the functionality of mobile apps thoroughly and ensures the appropriate working of the apps.

Emulators provide better results compared to simulators as they can be used to test specific situations or cases, and also can mimic multiple devices. Emulators are often used more as they are relatively cheaper – specifically if compared to real devices.

Compare Emulator/ Simulator vs Real devices

Compare

Situation-based scenarios: In situation-based scenarios, buying required mobile devices may be tough – while using the emulator/simulator may take care of the need. This means, not all the scenarios can be tested in simulators/emulators – this is possible only if testing is done on the mobile.

Ease-of-use scenarios: Emulators or Simulators make testing the bunch of apps on the multiple mobile devices available today much easier, but they miss out the UI and UX aspects of an app, including the color and brightness display.

Ease-of-access scenarios: Testing on mobile devices is reliable – however, testing on Emulators and Simulators makes it easier as it requires only the URL for the app.

Validating battery scenarios: Mobile devices can perform this test as required whereas emulators and simulators usually are not able to mimic the issues related to the battery.

Validating Performance: This is a very crucial part of testing a mobile application. Testing the performance of an app on a mobile device usually gives a more accurate result as compared when tested on emulators or simulators.

DISADVANTAGES OF EMULATORS AND SIMULATORS

Disadvantage

While both Emulators and Simulators look like great alternatives for testing apps, since having a wide range of real devices also accounts for the cost involved. But, they have issues that may generate both false positive and negative results, which will surely have a negative impact on the business ROI. Another disadvantage of emulator/simulator is that howsoever much close their feature may be to the real app, emulator/simulator may still not cover all features/scenarios and the results obtained may not be as accurate as we would want them to be.

Real device testing wins the challenge over either of the two. With an efficient testing process, real devices can validate for every possible scenario with almost 100% accuracy.

However, having all three options have their own benefits, and can be used according to the need, helping organisations meet their goals.

App distribution tools

Developers develops apps

How it is distributed to QA?

Tools used are Hockey app- for both android and IOS

Test flight - Mostly IOS

Developer will push build file in hockey app, tester will take the hockey app build from test devices and takes up the code.

IOS- developers license is required, they will add your profile or office to developer organization and will give the certificate

Android - apk file is emailed and downloaded by email attachment

Analytics

Adobe Analytics

Google Analytics

Fluory Analytics

2 factors to determine the list of device to test

  • Based on analytics team
  • confidence on quality of application

Crashlytics

Analytics

Crash can happen in foreground and background

Its a third party software which is embedded into the mobile application.

This tool gives a crash report to the email address. you can clearly see on what screen the user was navigating to before getting the crash.

This will give you device name, battery, time, Ram details etc.

Sometimes crash can happen because of RAM, battery

and all. This crash report has logs also

(Technical details)

There is a tool called LemonJar in Xcode to see crash logs in IOS

Abdroid studio is for Android device

Challenges

1. Timings

2. Screen size

3. Number of Devices

4. OS Versions

5. Connection Types

6. Battery Life

7. Internationalisation

8. Different Mobile App types

What kind of testing we can do on mobile app which may not be done in web app

Challenges

  • Cross device and OS(Based on Analytics)
  • Connection testing/ Network testing(2g,3g ..)
  • Battery Testing(How much battery is consumed by an app)
  • App upgrade testing(data or functionality should not loose on upgrade)
  • GUI Testing - (look and feel of app, taps and all buttons)
  • Orientation testing(Portrait/Landscape)
  • Interruption testing (Try to interrupt usage of application, try to cancel app eg, phone message and how it should behave, bank app should get logged out)

Push Notification

Tap on notification to take you to right page

This is taken care by CRM Team. (Customer relation manager)

There is a common tool for Push Notifications (Adobe Neolane)

Push Notification

Learn more about creating dynamic, engaging presentations with Prezi