Exploring the Factors that Affect Smartphone Testing

AbstractThe starting point of this study is based on the

assumption that the complexity in smart phones is unavoidable
since buyers of these products strongly prefer high-feature
models. At the same time manufacturer aims to achieve higher
levels of user satisfaction by improving usability The user
interface having friendly and efficient environment is attracting
most of the customers. However, the aim of this study is to
identify and analyze various limitations of testing Smart Phone.
KeywordsSmartphone, Testing,
Softare Development Life Cycle, SDCL

I.

Userinterface,

usablity,

INTRODUCTION

Today is the age of competitive software market. Different

tasks are solved by almost identical functionality programs.
The user interface having friendly and efficient environment is
attracting most of the customers. However, the aim of this
study is to identify and analyze various limitations of testing
Smart Phone.
This study is based on survey. The starting point is based
on the assumption that the complexity in smart phones is
unavoidable since buyers of these products strongly prefer
high-feature models. At the same time manufacturer aims to
achieve higher levels of user satisfaction by improving
usability. Because smart phones integrate PDA and cell phone
features, conflicting design goals arise [1, 2, 3]. Smart phone
differs a lot in every sense from display screen and input data
to phone functionality. Smart phone uses its own operating
system to operate with phone functionality.
II.

LITERATURE REVIEW

Literature always plays an important role to know the state

of art or to know that other researchers say about any specific
topic.
A detailed literature review was done on testing of smart
phone and desktop application testing. The breakdown of
literature review is as follows:
Testing
Traditional Testing in Desktop Applications
Testing in Smart Phone Applications
A. Testing
Testing is an activity performed for evaluating product
quality, and for improving it, by identifying defects and
problems [13]. Software Testing is a part of Software
Development Life Cycle. Testing is one of the oldest forms of
verification [16]. Software Testing is one of the broad
categories of software engineering. In which any particular
software is tested against the documented specifications.
Testing is an activity performed for evaluating product
quality, and for improving it, by identifying defects and
problems [6]. Software Testing is one of the broad categories

of software engineering. In which any particular software is

tested against the documented specifications. Software testing
is a process which is used to measure the quality of software
developed. It is also a process of uncovering errors in a
program and makes it a feasible task. It is useful process of
executing program with the intent of finding bugs [7].
According to [8], testing is a phase which is further
divided into mentioned levels:
1. Unit Testing
2. Integration Testing
3. Code Coverage Analysis
4. System Testing
5. Acceptance Testing
Testing is an essential activity in software engineering that
is used to validate whether the system behaves as intended and
identifies the malfunctions. It is widely used in industry for
quality assurance, as it provides a realistic feedback of the
behavior of the system. Nowadays the software systems are
being used to perform critical tasks where the margin of error
is really low. So these systems should be error free and
contain a high quality. Software testing is an important
process that can help to achieve software quality assurance.
Companies are spending huge amount of money on testing
activities. Research shows that more than 50% of the total cost
of development is devoted to software testing [10]. One of the
aims of testing is to reveal as much potential for failure as
possible, and many techniques have been developed to do this,
which attempt to break the program, by running one or more
tests drawn from identified classes of executions deemed
equivalent [13].
Software testing is also used to test the software for other
software quality factors like reliability, usability, integrity,
security, capability, efficiency, portability, maintainability,
compatibility etc. [18].
B. Traditional Testing in Desktop Applications
According to Nick von Dadelszen [12] two different
approaches to testing are commonly used: White box testing in
which full information and source code provided and Black
box testing in which no code or information is provided and it
works only with downloadable applications.
White Box and Black Box are further divided into number
of techniques. Some techniques are extracted from some
literature. These are as follows:
Research
Papers

Techniques
________________________________________
Black Box
White Box

A Survey of
Software
Testing Tools
for
Computational
Science[8]

(Used to attain good inputoutput data coverage)

Equalance Partitioning
Boundary
Value
Analysis
Cause effect graphing
Error guessing

(Focuses on program
logic coverage)
Statement Coverage
Decision coverage
Condition coverage
Decision-Condition
Coverage
Multiple Condition
Coverage

Software
Testing
Techniques [17]

Functional Testing
External Behavior

Structural Testing
Internal Structure
Path Coverage
Data Flow coverage
Branch Coverage

SWEBOK [13]

Specification based testing

techniques
Decision tables
Finite State Machine
Based
Testing from Formal
Specifications
Random Testing

Code based techniques

Control Flow
Data Flow

Different Forms
of
Software
Testing
Techniques for
Finding Errors
[19]

Correctness Testing
Ad-hoc testing
Load testing
Stress testing
Functional testing
Exploratory testing
Usability testing

Correctness Testing
- Control flow
- Branch testing
- Path testing
- Data flow testing
- Loop

Similarly, the general steps involved in traditional testing are:

1. Designing test cases
2. Preparing test data
3. Execution of test data
4. Comparison of test cases [11]

C. Testing in Smart Phone Applications

One of the most dynamic and competitive in consumer
electronics industry is market of mobile communication
products [1]. Latest trend in cell phones is smart phone.
According to [20], Application development for mobile
devices is a fast growing phenomenon. According to
eWeek.com there were more than 450 million mobile Internet
users in 2009, and the number is expected to surpass 1 billion
by the end of 2013.
Todays mobile applications deliver complex functionality
on platforms that have limited resources for computing. This
diversity in mobile computing environments presents unique
challenges in application development, quality assurance, and
deployment, requiring unique testing strategies [9]. Further it
says, mobile applications, although they have limited
computing resources, are often built to be as agile and reliable
as PC-based applications. In order to meet the challenge,
mobile application testing has evolved as a separate stream of
testing.
Increasing rate of introducing new features and
applications is also increasing the speed of adoption of smart
phone but less focus is paid on testing because of integration
of PDA and cell phone to get smart phone features, due to
which conflicting goals also arises. [2,3] and hence, testing
issues also occur.
Testing Techniques of Smart Phone differ with
Software Testing Techniques. Some of the similarities and
differences in literature are given below:
According to [14], there are some techniques which are
different from Software Testing Techniques. These are:
Power Consumption Testing uncovers defects
related to battery drainage caused by the application.

 Database Testing is very important for all

applications. We check for data integrity and errors
while editing, deleting and modifying the forms and
all other DB related functionality. This testing is done
manually, without the use of any testing tools.
In [20], mobile application testing is considered to be different
from desktop application testing in following areas:
User Interface Testing: Mobile devices have unique
interfaces like smaller screens that can be re oriented,
touch screens and soft keyboards and navigation
methods like hard keys and trackballs.
External Factor Testing: Mobile device applications
must also contend with interactions and interruptions
from other device features like various network
connection types SD cards, Phone Calls, and assorted
device settings.
Stress Testing: Mobile device applications have
much less overall device memory and power
available so must handle them very efficiently.
Security Testing: It is important when user base
grows, security of mobile web applications, sensitive
data storage, data behavior under various device
permission schemes.
Emulator Use: emulators can be a great asset when it
comes to achieving testing coverage on multiple
devices.
According to Mark & Stephen [5], these devices share a
common problem: attempting to give users access to powerful
computing services and resources through small interfaces,
which typically have tiny visual displays, poor audio
interaction facilities and limited input techniques. They also
introduce new challenges such as designing for intermittent
and expensive network access, and design for position
awareness and context sensitivity.
Further [10] adds, a major draw of any smartphone is its
ability to run applications, thus users are increasingly relying
on smartphones for computing needs, rather than using laptops
or desktops. Reliability is particularly important for sensitive
mobile applications such as online banking, business
management, health care, or military domains. The physical
constraints of mobile devices (e.g., low-power CPU, small
memory, small display), as well as developers' unfamiliarity
with mobile platforms (due to their novelty), makes mobile
applications prone to new kinds of bugs.
In [15] top 7 quality issues with mobile are discussed that
can affect the testing process in smart phones. These issues
are: Time, Inadequate processes, Lack of Test Plans, Data
entry, Validation issues, Lack of physical devices and lack of
test automation. When we talk about testing in Mobile
domain, it is not only confined to Mobile Apps, but also
includes mobile handset and mobile website testing. As
discussed in [15], similar to general Software Testing, Mobile
Software Testing also includes:
User Interface Testing (Color scheme, Menu styles,
Consistency of UI over various Devices)
Functional Testing (Testing core functionality of Mobile
App as per specification)

Performance & Stress Testing [Behavior of Mobile

Application in Low resources(Memory/Space), Behavior
of mobile website when many mobile user simultaneously
access mobile website)]
Usability Testing (Testing of usability aspects of Mobile
Apps)
Apart from above mentioned testing types, some key testing
types may include the following.
Testing for Compatibility: Testing the compatibility
of your application with native device features
Certification
Compliance
Testing:
For
downloadable mobile applications, there are various
Third party Mobile Quality Certification program for
various platforms.
Submission Guidelines Compliance Testing: The
application needs to adhere to the specified
submission guidelines to publish it in any mobile
application store. Failure to meet these guidelines
may result in rejection of your app on mobile
application stores
Interruption Testing (Voice Calls, SMS, Charger,
Low memory Notification) while application is
running.
Monkey Testing: Continual key pad entries via tools
to test application stability with various key press
events.
Low Network/No Network case Testing:
Application behavior when there is no network
coverage or Low network strength.
According to [9], the unique challenges in testing mobile
applications arise from the diversity of the device
environment, hardware and networking considerations, and
Rapid Application Development (RAD) methodologies. Some
of the challenges are:
1. Diversity of the device environment
a. Display size
b. Application run times
c. Lesser memory
d. Processing power
e. Network landscape
f. Processing speed
g. Network latency
2. RAD Methodologies
a. Testing of mobile application is traditionally
done by manual execution of test cases and
visual verification of the results.
b. Use of emulators and actual devices
III.

EXPECTED OUTCOME

When it comes to testing desktop applications, literature

shows that many researchers have proved its importance and
execution through various methodologies. [7,8,11,18]
In literature, identified testing techniques mainly targeted
desktop application.
So, there is a need to identify the factors affecting testing in
smart phone in order to reduce budget spent on testing and

time to market. That is the reason; testing smart phone is a

whole new changed process.
Testing smart phone is different from testing desktop
applications. Smart Phone Testing is a new and evolving
domain having limited resources in the devices as compare to
full fledge computers.
The research question of our study is:
What are the factors affecting smart phone testing?
In this Research Question, we will identify the factors
which are affecting the Testing Process of Smart Phones. For
this purpose literature survey will be conducted. .
As a research method, literature survey will be done. The
process of focusing on research question helps us to determine
the use, purpose and structure of this study. Other elements
like time availability, resources and skills will also be
considered. In software engineering we use different research
methods to reach a conclusion. Survey will be conducted by
following its proper structure and procedure.
Expected outcome will be in the form of suggestions and
guidelines. Literature Survey will be done to compare the
traditional software testing techniques of smart phones, in order
to get the factors that affect testing smart phone.

[3].
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].

[14].

REFERENCES

[15].
[16].

[1]. Jeroen Keijzers, Elke Den Ouden and Yuan Lu, Usability Benchmark
Study of Commercially AvailableSmart Phones: Cell Phone Type
Platform, PDA Type Platform and PC Type Platform, Proceedings of
the 10th Int. Conference on HCI with Mobile Devices and Services,
ACM, 2008, pp. 265-272.
[2]. Nyberg, M. Bjrk, S. Goldstein, Redstrm J., Handheld applications
design: Merging information appliances without affecting usability,
Proceedings IFIP TC.13 Conference on HCI, IOP Press 2001, pp. 391398.

[17].
[18].
[19].

Zheng, P. and Ni, L.M., Smart Phone & Next Generation Mobile
Computing, Elsevier Inc., San Francisco, 2006.
Cuixiong Hu, Iulian Neamtiu, Automating GUI Testing for Android
Applications, Proceedings of 6th Int. Workshop on Automation of
Software Test, ACM, 2011, pp. 77-83.
Mark Dunlop, Stephen Brewster, The challenge of Mobile Devices for
Human Computer Interaction, Springer Verlag London Ltd 2002
Tanya Dumaresq, Matt Villeneuve, Testing Strategies for Smart Phones
and Mobile Devices, Macadamian, 2010.
M. Ehmar Khan, Different Forms of Software Testing Techniques for
Finding Errors, IJCSI International Journal of Computer Science
Issues, Vol. 7, Issue 3, No 1, May 2010.
L.S. Chin, D.J. Worth, and C. Greenough, A Survey of Software
Testing Tools for Computational Science, Journal Article DOI: RAL
Technical Reports, RAL-TR 2007.
White Paper by Keane Testing Business Mobile Applications, [OnLine] Available: http://www.keane.com , January 2009
Ahsan Nawaz & Kashif Masood Malik, Software Testing Process in
Agile Development, Master Thesis Computer Science Thesis no: MCS2008-25, Blekinge Institute of Technology, June 2008.
Engr. Ali Javed, Software Quality Assurance: Dynamic Testing, Black
Box Testing, April 2012, 2013
Nick Von Dadelszen, Testing Mobile Application, OWASP NZ Day
Internet: www.lateralsecurity.com , July 2011, [April 2012].
Borque P., Dupuis R., (2004), Guide to the Software Engineering Body
of Knowledge.
SWEBOK, [On-Line] Available:
http://www.softwareengineeringhelp.com/software-engineering-body-ofknowledge/.
Lindiwe Vinson, Mobile Application Testing: Process, Tools &
Techniques,
Internet:
http://www.threeminds.com/mobileapplicationtesting.htm , May 2011,
[May 2012].
Top Seven Quality Issues with Mobiles, Internet:
http://zapt.blogspot.com/2011/03/top-7-quality-issues-with-mobile.html , March
2011, [June 2012].
Mary Jean Harrold, Testing: A Roadmap, In Future of Software
Engineering, 22nd International Conference on Software Engineering,
June 2000.
Lu Luo, Software Testing Techniques Technology Maturation and
Research Strategy, Class Report for 17-939A, 2003.
Sheikh Umar Farooq, S.M.K Quadri, Software Testing Goals,
Principles, and Limitations, International Journal of Computer
Applications (0975 8887) Volume 6 No.9, September 2010.
Ray
Robinson,
Automation
Test
Tools,
Internet:
http://www.telerik.com/automated-testing-tools/ , Sept 2001 [Sept
2012].

[20] XBOSoft Inc., Mobile Application Testing Challenges and Solutions",

XBOSoft White Paper-Mobile Phone Proliferation, 1989.