LOGO

Software Testing
Nguyn t Thng Faculty of Mathematics and Informatics University of Science, VNU-HCM

Course
Name: Software Testing 1 Lecturer: Nguyn t Thng Credit: 3 Duration: Presentation: 13 weeks On-the-fly practice: 02 weeks Lab practice: 10 weeks Prerequisites: Software Engineering
Nguyn t Thng Software Testing 2

Content
1

Introduction Test Principles Test Types Test Techniques Test Methodologies Test Automation Test Management Test Tools Test a Software Project
Software Testing 3

Nguyn t Thng

Practice

Test Analysis

Test Report

Test Case

Tester

Bug Tracking

Bug Report

Nguyn t Thng

Software Testing

Evaluate
Mid-term exam: 30% Type: Writing Time: Up to chapter 6 Lab practice: 30% Type: On lab Time: 15th week Final exam: 50% Type: Multiple choice Time: (Depend)
Nguyn t Thng Software Testing 5

Reference
1. Internet Source. Keyword: Software Testing, Test Methodology, Test Principles, Test Tool, Test Technique. 2. Dorothy Graham, Erik van Veenendaal, Isabel Evans, Rex Black Foundations of Software Testing 3. Software Testing Help http://www.softwaretestinghelp.com 4. The UML Testing Profile http://www.omg.org
Nguyn t Thng Software Testing 6

Reference (2)
6. Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki

Nguyn t Thng

Software Testing

Chapter 1

Introduction

Nguyn t Thng

Software Testing

Content
1

Introduction Test Principles Test Types Test Techniques Test Methodologies Test Automation Test Management Test Tools Test a Software Project
Software Testing 9

Nguyn t Thng

Introduction
What is a bug? Quality and Reliability Why is testing necessary? Purposes of testing No bugs found How much testing is enough? Test approaches Which factors influence test? Independence in test Who are Testers?
Nguyn t Thng Software Testing 10

What is a bug?
Error: a human action that produces an incorrect result Fault: a manifestation of an error in software also known as a defect or bug if executed, a fault may cause a failure Failure: deviation of the software from its expected delivery or service (found defect)
Nguyn t Thng Software Testing 11

What is a bug? (2)

A person makes an error ... that creates a bug in the software ... that can cause a failure in operation

Nguyn t Thng

Software Testing

12

Quality and Reliability

Software quality:
Reliability: the probability that software will not cause the failure of the system for a specified time under specified conditions. Efficiency: the ability to ensure high performance once the application is in runtime mode. Security: the ability to encounter critical vulnerabilities that damage the business. Maintainability: includes adaptability, portability and transferability
Nguyn t Thng Software Testing 13

Quality and Reliability (2)

The less failures system causes, the more reliable the system is. No system is bug-free. So a system can be reliable but still have bugs. Bugs occur in software since:
Software is written by human beings
Know something, but not everything Have skills, but are not perfect Make mistakes

Software is developed under increasing pressure and strict deadlines

Nguyn t Thng Software Testing 14

Why is testing necessary?

Software failures cost: Huge sums
Ariane 5: $7,000,000,000 Mariner space probe to Venus: $250,000,000 American Airline: $50,000,000

Very little or nothing at all

Minor inconvenience No visible

Software is not linear: Small input may have very large effect
Nguyn t Thng Software Testing 15

Why is testing necessary? (2)

Software failures can cause death or injury: Radiation treatment kills patients Train driver killed Aircraft crashes (Airbus & Korean Airlines) Bank system overdraft letters cause suicide

Nguyn t Thng

Software Testing

16

Why is testing necessary? (3)

Testing is necessary:
because software is likely to have bugs because failures can be very expensive to learn about the reliability of the software to avoid being sued by customers to stay in business

Testing is not:
to prove that the software has no bugs to fill the time before the release date because testing is included in the project plan
Nguyn t Thng Software Testing 17

Purposes of testing
Purposes of testing are to: find bugs/faults build confidence demonstrate conformance to requirements reduce costs show system meets user needs assess the software quality improve the software quality: reliability, efficiency, security, usability, maintainability, reusability, testability, etc.
Nguyn t Thng Software Testing 18

No bugs found

Confidence

Bugs found

Time

No bugs found = confidence?

Nguyn t Thng Software Testing 19

No bugs found (2)

High
You think you are here

Many Bugs

Few Bugs

Low Software Quality

Few Bugs

High

Test Quality

Few Bugs

You may be here

Nguyn t Thng

Low
Software Testing 20

How much testing is enough?

Average 4 menus 3 options / menu

system has 20 screens

Average: 10 fields / screen 2 types input / field (date as Jan 3 or 3/1) (number as integer or decimal) Around 100 possible values

Total for 'exhaustive' testing: 20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests 1 sec/test-> 8000 mins ~ 133 hrs ~ 17.7 days (not counting finger trouble, bugs or retest) 10 secs-> 34 wks, 1 min-> 4 yrs, 10 min-> 40 yrs
We cannot test everything
Nguyn t Thng Software Testing 21

How much testing is enough? (2)

Exhaustive testing: is exercising all combinations of inputs and preconditions consumes impractical amount of time We do not run an exhaustive test not enough resources not enough time not necessary

Nguyn t Thng

Software Testing

22

How much testing is enough? (3)

its never enough when you have done what you planned when your customer/user is happy when you have proved that the system works correctly when you are confident that the system works correctly it depends on the risks for your system

Nguyn t Thng

Software Testing

23

How much testing is enough? (4)

So little time vs. So much to test: test time will always be limited base on RISK, we have to determine:
what to test first what to test most how thoroughly to test each item what not to test (this time)

we have to allocate the time available for testing by prioritizing testing

Nguyn t Thng

Software Testing

24

How much testing is enough? (5)

Other factors that influence testing contractual requirements legal requirements industry-specific requirements
e.g. pharmaceutical industry (FDA), compiler standard tests, safety-critical or safety-related such as railroad switching, air traffic control

Nguyn t Thng

Software Testing

25

How much testing is enough? (6)

Most important principles:
It is difficult to determine how much testing is enough but it is not impossible Prioritize tests so that, whenever you stop testing, you have done the best testing in the time available.

Nguyn t Thng

Software Testing

26

Test approaches
Traditional testing approach Show that the system:
does what it should doesn't do what it shouldn't

Easy test cases

Goal: show working Success: system works Result: bugs left in

Nguyn t Thng

Software Testing

27

Test approaches (2)

A better testing approach Show that the system:
does what it shouldn't doesn't do what it should

Difficult test cases

Goal: find bugs Success: system fails Result: fewer bugs left in

Nguyn t Thng

Software Testing

28

Test approaches (3)

The testing paradox Purpose of testing is to destroy confidence:
Purpose of testing is to try to find bugs Finding bugs destroys confidence

Purpose of testing is to build confidence

The best way to build confidence is to try to destroy it

Nguyn t Thng

Software Testing

29

Independence in test
Test your own work? find 30% - 50% of your own bugs same assumptions and thought processes see what you meant or want to see, not what is there emotional attachment
dont want to find bugs actively want NOT to find bugs

Independence in test is very important.

Nguyn t Thng Software Testing 30

Independence in test (2)

Levels of independence Tests designed by the person who wrote the software Tests designed by a different person Tests designed by someone from a different department or team (e.g. test team) Tests designed by someone from a different organisation (e.g. agency) Tests generated by a tool (low quality tests?)
Nguyn t Thng Software Testing 31

Who are Testers?

Testers are the persons who: Are usually not welcomed, Do a boring repeated work, Do a destructive process, Bring bad news, Describe bugs to authors, Work under worst (time) pressure, Need to take a different view, Need a different mindset,
Nguyn t Thng Software Testing 32

Who are Testers? (3)

Testers have responsibilities to:
follow the test plans, script, etc. as documented report bugs objectively and factually, no abuse check tests are correct before reporting bugs remember it is the software, not the programmer, that you are testing assess risk objectively prioritise what you report communicate the truth
Nguyn t Thng Software Testing 33

Who are Testers? (2)

Testers have the rights to: be regarded as a professional, request specifications and test plans, access to accurate progress and changes, be trained in working domain, find bugs, report bugs seriously (non-reproducible), make predictions about future bug levels,

Nguyn t Thng

Software Testing

34

Questions

The important thing is not to stop questioning. Curiosity has its own reason for existing.
(Albert Einstein)

Nguyn t Thng

Software Testing

35