Ubiquitous Interactions

Ilya Shmorgun

http://inqb8tor.com, accessed 28.11.2012

New Wave of HCI

We are increasingly placed in dynamic
congurations of technology. devices and software. of our activities.

Activities are supported by a wide range of Devices are capable of sensing the context

Camera GPS and GLONASS Ambient light sensor Proximity sensor Accelerometer Three-axis gyroscope Digital compass WiFi 3G and EDGE Bluetooth

http://www.apple.com/iphone/specs.html, accessed 28.11.2012

Question
How to deliberately design to facilitate this new type of interaction in the best possible way?

Topics
Ubiquitous computing Ubiquitous interactions Key constructs Research questions Future steps

Ubiquitous Computing

Post-Desktop Model of HCI

Information processing is integrated into
everyday objects [1].

Ubiquitous computing is subsuming

traditional computing paradigms, for example desktop and mobile computing [2].

1. Zhao, R., Wang, J.:Visualizing the research on pervasive and ubiquitous computing. Scientometrics 86(3), 593612 (2011) 2. Greeneld, A.: Everyware: The Dawning Age of Ubiquitous Computing. New Riders Publishing, 1st edn. (Mar 2006)

Invisible Computing
Interaction with computers should be
what directly interests them [2]. more like interaction with the physical world [1].

Users should be able to sense and control Our devices remain the focus of attention
instead of fading into the background.
1. Abowd, G., Mynatt, E.: Charting past, present, and future research in ubiquitous computing. ACM Transactions on Computer-Human Interaction (TOCHI) 7(1), 2958 (2000) 2. Roussos, G., Musolesi, M., Magoulas, G.D.: Human behavior in ubiquitous environments: Experience and interaction design. Pervasive and Mobile Computing 6(5), 497498 (Oct 2010)

People and Computers

Previously a one-to-one relationship, one
technology - one application - one user. changing congurations of devices, applications, and users.

Nowadays many-to-many, with ever-

Brodersen, C., Bdker, S., Klokmose, C.: Ubiquitous substitution. Human-Computer InteractionINTERACT 2007 pp. 179192 (2007)

Context
Any information, which characterizes the
situation of a person, a place or an object, relevant to the interaction between a user and an application [1]. context [2].

Ubiquitous computing as a technology of

1. Chen, G., Kotz, D.: A survey of context-aware mobile computing research (2000) 2. Dourish,P.: Where the Action Is: The Foundations of Embodied Interaction (Bradford Books). The MIT Press, new ed edn. (Aug 2004)

Types of Context
Where you are. Who you are with. What resources are nearby.

Schilit, B., Adams, N., Want, R.: Context-aware computing applications (1994)

Modus Operandi

All the time everywhere information processing.

Saha, D., Mukherjee, A.: Pervasive computing: a paradigm for the 21st century. Computer 36(3), 2531 (2003)

Ubiquitous Interactions

Denition
Interactions in the context of ubiquitous computing, which include multiple, dynamic, and distributed interfaces.

Klokmose, C.N.: An Instrumental Paradigm for Ubiquitous Interaction. DHRS 2006 p. 33 (2006)

Properties
Occurring all the time Occurring everywhere Occurring on any device with ease of
substituting devices when necessary

Being transparent Being informed by context

All the Time Everywhere

Enabled by the current generation of
mobile technology. reality.

Ubiquitous access to information is a

Satyanarayanan, M.: Mobile computing: the next decade. Proceedings of the 1st ACM Workshop on Mobile Cloud Computing & Services: Social Networks and Beyond p. 5 (2010)

Research Focus
No clear way of addressing the issues of occurring on any device, being transparent, and informed by context.

Research Problem

Ubiquitous interaction design offers little coherent theory to guide the design process.

Klokmose, C.N.: An Instrumental Paradigm for Ubiquitous Interaction. DHRS 2006 p. 33 (2006)

Research Question

How to design ubiquitous interactions?

Research Goal

Provide theoretically supported guidelines for designing ubiquitous interactions.

Key Constructs

Activity Theory
Facilitates the analysis of human activity in terms of three layers: activity, action, and operation.

Brodersen, C., Bdker, S., Klokmose, C.: Ubiquitous substitution. Human-Computer InteractionINTERACT 2007 pp. 179192 (2007)

Activity Theory
Activity layer focuses on why something
is it that is taking place [1]. occurs and analyzes the motivation for an activity [1].

Action layer focuses on understanding what Operations layer focuses on understanding

how an activity is carried out [2].
1. Brodersen, C., Bdker, S., Klokmose, C.: Ubiquitous substitution. Human-Computer InteractionINTERACT 2007 pp. 179192 (2007) 2. Bdker, S., Klokmose, C.N.: The HumanArtifact Model: An Activity Theoretical Approach to Artifact Ecologies. HumanComputer Interaction 26(4), 315371 (2011)

Activity Theory
Considers situations in which humans act
without conscious planning [1].

A breakdown occurs when something that

seems possible does not adhere to what is actually possible [2].

1. Bdker, S.: A Human Activity Approach to User Interfaces. HumanComputer Interaction 4(3), 171195 (Sep 1989) 2. Klokmose, C.N.: An Instrumental Paradigm for Ubiquitous Interaction. DHRS 2006 p. 33 (2006)

Artifacts
Objects designed with a particular purpose
of use in mind [1].

Do not have meaning without being

concrete use activities [2].

incorporated into social practice [2].

Need to be studied in use in the context of

1. Bdker, S., Klokmose, C.N.: The HumanArtifact Model: An Activity Theoretical Approach to Artifact Ecologies. HumanComputer Interaction 26(4), 315371 (2011) 2. Bannon, L.J., Bdker, S.: Beyond the interface: Encountering artifacts in use. DAIMI PB 18(288) (1989)

Artifacts
Historical devices or crystallized
knowledge.

Reect the state of practice up to the point

when they were created.

New artifacts take advantage of the

operations, which were developed during use of the previous generation of artifacts.

Bannon, L.J., Bdker, S.: Beyond the interface: Encountering artifacts in use. DAIMI PB 18(288) (1989)

Artifact Ecologies
A set of all physical artifacts with some
level of digital interactivity that a particular person owns, has access to, and uses [1].

Needs to include non-digital artifacts [2]. Dynamic and evolving [3]. All artifacts are destined to become part of
someones ecology [1].
1. Jung, H., Stolterman, E., Ryan, W., Thompson, T., Siegel, M.: Toward a framework for ecologies of artifacts: how are digital artifacts interconnected within a personal life? In: NordiCHI 08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges. ACM Request Permissions (Oct 2008) 2. Bdker, S., Klokmose, C.N.: The HumanArtifact Model: An Activity Theoretical Approach to Artifact Ecologies. HumanComputer Interaction 26(4), 315371 (2011) 3. Bdker, S., Klokmose, C.N.: Dynamics in artifact ecologies (2012)

Mediator
An artifact, which helps users act on the
artifacts [2]. object of their interest in a way that would not be possible without the mediator [1].

All actions in the world are mediated by In HCI everything from components of a
user interface to hardware on which software operates [3].
1. Bdker, S.: Use is everywhere and changing: analysis and design with the human-artifact model. Proceedings of the 29th Annual European Conference on Cognitive Ergonomics (2011) 2. Bannon, L.J., Bdker, S.: Beyond the interface: Encountering artifacts in use. DAIMI PB 18(288) (1989) 3. Brodersen, C., Bdker, S., Klokmose, C.: Ubiquitous substitution. Human-Computer InteractionINTERACT 2007 pp. 179192 (2007)

Ubiquitous Substitution
An approach that focuses on continuous Not about replacing one mediator with
another. switching of mediators with the purpose of enriching a users repertoire of action possibilities.

Understanding which mediator applies

better in which conditions.
Brodersen, C., Bdker, S., Klokmose, C.: Ubiquitous substitution. Human-Computer InteractionINTERACT 2007 pp. 179192 (2007)

Functional Organ
A combination of a tool with natural
human abilities, which allows a person to perform a new function or an existing function in a more efcient way [1]. achieving a state of transparency or seamlessness [2].

Designing functional organs facilitates

1. Kaptelinin,V.: Activity theory: implications for human-computer interaction. Context and consciousness: Activity theory and humancomputer interaction pp. 103 116 (1996) 2. Bdker, S., Klokmose, C.N.: The HumanArtifact Model: An Activity Theoretical Approach to Artifact Ecologies. HumanComputer Interaction 26(4), 315371 (2011)

Research Questions

Formulated Research Questions

How to design for ubiquitous substitution? How to design artifacts as functional
organs? mind?

How to design with context-awareness in

Ubiquitous interaction property

On any device

Research question
How to design for ubiquitous substitution? How to design digital artifacts as functional organs? How to design with context-awareness in mind?

Transparent

Informed by context

Research Question Priority

Designing for ubiquitous substitution is considered the primary research question as ubiquitous substitution is considered the main driver of ubiquitous interaction.

Research Question Priority

Facilitating transparency through designing artifacts as functional organs is considered a secondary objective as using artifacts in a seamless and transparent way is an important aspect of ubiquitous interaction.

Research Question Priority

Designing with context-awareness in mind is not considered critical for this research, however if possible this question will be addressed in future work with the goal of understanding how context-awareness can inuence ubiquitous interaction.

Future Steps

Future Steps Include

Mobile devices survey analysis mTLU

Mobile Devices Survey Overview

Semi-structured survey. Conducted over a period of 3 weeks. 101 full responses.

Mobile Devices Survey Goals

Which types of devices people use? Which types of services are used on those
devices?

How important is it for people to have

access to the same services across their devices?

Mobile Devices Survey Goals

Which specic features of those services
are used on which devices?

How aware are people of context-related

features of those services and how much do they use them?

Mobile Devices Survey Outcomes

A glimpse of how people are currently dealing with the issues related to ubiquitous substitution and if and how they are using context-related features provided by the current generation of digital services.

mTLU Overview
Design a digital artifact to leverage existing information sources at Tallinn University to provide better support for daily universityrelated activities of students, lecturers, and staff by leveraging the principles of ubiquitous substitution, transparency, and context-awareness.

mTLU Outcomes
A proof-of-concept, which demonstrates how the principles of ubiquitous interaction can be applied in a design project.

Conclusion

The New Reality

People use a plethora of different artifacts
to support their activities.

Artifacts compose artifact ecologies. Artifacts compete for the users attention. Artifacts inuence the users expectations. Artifacts can collect and leverage
contextual information.

Considerations
How to better support ubiquitous
substitution? organs?

How to design artifacts as functional What kind of contextual information is

needed?

Complex Challenges
We currently do not have a clear
understanding of how to design digital artifacts to provide answers to these issues. account to design solutions that better t into the changing conditions of the new wave of HCI.

We need to take these notions into

Thank you