Pervasive Computing

Lecture-2, 2012-2013

BITS Pilani
Pilani Campus

Rahul Banerjee, PhD Department of Computer Science & Information Systems

Dr. Rahul Banerjee, BITS Pilani, India

Interaction Points
Typical Problems and Issues related to technology intervention in the Pervasive Computing environments
HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and Project Communicate Summary of Concepts learnt
Dr. Rahul Banerjee, BITS Pilani, India

Typical Problems and Issues

Lets have a brief look at the select examples and then identify the problems related to the technology intervention in form of the Pervasive Computing solutions!
Examples taken so far:
InSTARTM Augmented Reality-based Car Navigation System Steering Display-based Systems Digital Graffiti Systems

Examples to be taken for brief exposure during the second and third lectures:
The GeorgiaTech experiment on child behaviour during game-playing in presence of a caregiver and child-to-child interactions in absence of adults The Project-Connect at BITS Pilani for the elderly-assistance The Project Communicate at BITS Pilani for the Children suffering from Autism The BITS Life-Guard Wearable Computing project for alerting drivers in case of reflex-level deterioration towards levels considered safe for driving
Dr. Rahul Banerjee, BITS Pilani, India

The Georgia Tech Experiment

Sensor-Compute Nodes
Dr. Rahul Banerjee, BITS Pilani, India

Interaction Points
Typical Problems and Issues related to technology intervention in the Pervasive Computing environments
HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and Project Communicate Summary of Concepts learnt
Dr. Rahul Banerjee, BITS Pilani, India

Introduction to the Touch Lives Initiative at BITS Pilani

The Touch Lives Initiative is an outreach initiative aimed at assisting the underserved segments of the society with technology-driven simple solutions. http://www.touch-lives.org/ How does it operate?
Idea > need-analysis > research > solution
The Ideas-Crucible: Research being carried out by the consenting researchers at BITS Pilani works as the seed to the outreach projects Ideas get generated through multiple routes:
Classroom discussions based on real-life engineering applications Idea-fests / Project Competitions / Literature-survey Keen observations of faculty/student researchers about peoples hardships Project-related brainstorming sessions
Dr. Rahul Banerjee, BITS Pilani, India

Projects Under the Touch Lives Initiative

Project Connect
for the elderly

Project Communicate
for the children suffering from autism

Project Smile
for orphans

Project Light
for the illiterate adults
A few more candidate projects are in various stages of conceptualization and initial feasibility analysis

Focus of the current presentation: Project Connect

How did it all begin?
Observed the difficulties faced by my own parents once their fitness level got affected partly due to normal age-related factors and partly due to accidents and prolonged illness This led to realization of many difficulties that the elderly may have to face
Dr. Rahul Banerjee, BITS Pilani, India

A brief look at related issues

Select Problems of the Elderly that warrant non-intrusive technology intervention
Unavailability of someone nearby to support them if they feel too weak to walk, use bathroom or operate a gadget etc. Unavailability of any support in the times like the dead of the night, noisy functions / parties / events etc. when they may need assistance but some to take their care may not even get to know Social issues like moving from the place of their years of association to a new setting with children and / or grandchildren and finding themselves almost out of place due to: Not knowing enough people with who they could relate, share and connect socially and mentally In-job children not being able to spend the expected time either due to genuine reasons or personal preferences Not having the kind of say, they may have enjoyed when they were financially, physically and socially on their own Some of these problems at times might bring the elderly to the fringes of society where they might begin feeling uncomfortable, unwanted and helpless
Dr. Rahul Banerjee, BITS Pilani, India

BITS Pilani
Pilani Campus

It is this set of problems and issues that the Project Connect seeks to address so as to be able to provide a healing touch to the lives of the elderly and bring them a better quality of life, make them feel useful and thus feel happier.
Dr. Rahul Banerjee, BITS Pilani, India Dr. Rahul Banerjee, BITS Pilani, India

BITS Pilani
Pilani Campus

Our approach:
Multi-pronged, but primarily technology-assisted Keeps people of different capabilities, backgrounds and needs in mind Is partly, pro-active and partly reactive, by design
Dr. Rahul Banerjee, BITS Pilani, India Dr. Rahul Banerjee, BITS Pilani, India

About Project Connect

This project has several elements including:
Context-sensitive Voice-based Search:
This project aims at providing voice-driven and voice-search-enabled multi-functional support system to the elderly residents in the Old Age Homes and on different floors or non-contiguous areas on the same floor of a building with their family members and / or caretakers. The idea of the project came from two young undergraduate students: Arpit Agrawal and Shrainik Jain; who were interested in voice-search algorithms. Having the backdrop of my observation on the needs of the elderly, I thought to link their desire to work in this area with a project of wider (social) significance

Purpose:
building a solution allowing them to seamlessly interface with the external world for multiple relevant services Examples may include: right from assistance for bodily support from the volunteers / attendants to their communication with their physicians and even connecting them automatically with their children and grand children anywhere around the globe --- all through a simple voice request and without any textual entry.
Dr. Rahul Banerjee, BITS Pilani, India

About Project Connect

Other elements of this project include:
Non-intrusive Sensor Network involving sensors of several types like those for sensing Temperature Ambient Light-level Humidity level Oxygen-level Pressure Touch based gesture recognition units Thermal Imaging sensor units in areas like bathrooms Video-sensor units for gaze-detection, expression-recognition and gazedirected activation of services / gadgets Medicine-reminder and supply-level threshold alerts Social event reminders and associated assistive messaging (Audio/Video/Textual)
Dr. Rahul Banerjee, BITS Pilani, India

About Project Connect

Other involved elements include:
Computer Networks and internetworks involving: Fixed and mobile computational devices including
Regular computers Smartphones Low-end Mobile phones Sensor-Compute Nodes

Wired and wireless modes of connectivity support for allowing networking of various kinds between the involved devices

Dr. Rahul Banerjee, BITS Pilani, India

Typical Problems and Issues related to technology intervention

Typical Problems and Issues related to technology intervention in the environments of the elderly people
HCI issues, HCI modes (eHCI and iHCI etc.), User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues

Solution approaches
Pre-programmed in part, if historical data is available Sensor-driven in part, for collecting data in real-time and using it by relating it to the context of the person-in-question Designed to be preferably transparent to the end-user
Dr. Rahul Banerjee, BITS Pilani, India

Current Status of Project Connect

Currently, context-sensitive voice search module is nearly ready but needs about another years work for correcting certain shortcomings and improving results.
A research communication about this part of the work has been published in the Proceedings of the HCII international conference A related piece of work was presented in another international conference CiC

Gesture-recognition has been well-known in literature and is being adapted for providing more precise recognition of what the elderly might actually wish to communicate through a gesture that might not exactly fit in the standard framework
<e.g. elderly at times may wish to say two things through a single indicative gesture or may inadvertently send slightly ambiguous signals for assistance>
Dr. Rahul Banerjee, BITS Pilani, India

Ongoing Work
Use of Body Area Networks, Wireless and Wired Sensor Networks along with Personal Area Networks like Bluetooth and Wireless Local Area Networks like Wi-Fi has been utilized in synchronism with 2G/2.5G/3G/4G networks for identifying mechanisms and service elements that the final solution would be able to provide. There are a few areas of concern involving seamless handoff, battery-power depletion, safety of the wearer / user in case of wearable computing devices and mobile computing/telephony devices etc. which are being addressed now. Expression-recognition and associated event/device actuation part of the work is in its very early stages.
Dr. Rahul Banerjee, BITS Pilani, India

Interaction Points
Typical Problems and Issues related to technology intervention in the Pervasive Computing environments
HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and Project Communicate Summary of Concepts learnt
Dr. Rahul Banerjee, BITS Pilani, India

A Wearable Computer Architecture for Saving Human Lives

Dr. Rahul Banerjee, BITS Pilani, India

The BITS-Lifeguard Wearable Computing Initiative

Began in 1999, this research aims to protect human lives from those road accidents that result from the reduced levels of the physical fitness or mental alertness of the driver. Initially, it is focusing on light vehicles and their drivers / occupants. However, the concept is easily extensible to large vehicles and their drivers / occupants as well. This research also draws on the works done by life scientists on human sensory system, brain and select externally measurable parameters (that can be measured, calibrated or accurately estimated without piercing human body).

Dr. Rahul Banerjee, BITS Pilani, India

The Vision behind the BITS-Lifeguard System (1 of 2)

The overall life-saving environment in which the BITSLifeguard is envisioned to work shall have two core components:
The wearable computing component: The BITS-Lifeguard The vehicular computing component

The scenario of action would include:

Part-I:
sensing of select critical parameters that help estimate the current level of alertness and physical ability to drive safely, comparing these with the pre-fed threshold levels and generate an alert to the driver; in case, driver fails to respond quickly enough, send and SoS signal to the vehicular computer wirelessly

These responsibilities are handled by the wearable computer

Dr. Rahul Banerjee, BITS Pilani, India

The Vision behind the BITS-Lifeguard System (2of 2)

The scenario of action would include:
Part-II
Taking over control from the driver, Safely attempting to move the vehicle as per the pre-fed GIS map and GPS data Stopping the vehicle on a side Sending information wirelessly to the rescue / recovery agencies providing the location details, vehicles details and drivers details Intimating to the pre-registered relative / friend about the event and location These steps are taken by the vehicles computer which forms part of the ITS and Vehicular Computing aspects of this pervasive computing initiative

Dr. Rahul Banerjee, BITS Pilani, India

Elements of the BITS-Lifeguard NonIntrusive Wearable Computing System A wearable computing system of this category needs at least five basic elements:
Non-Intrusive Sensory elements to sense the wearers environment, Computing elements to take care of computational needs; Communication elements to interconnect these computing elements (with mobility) Body safe Power Supply / Generation elements to provide the necessary power to the wearable computing system Fabric or placeholder elements to allow interconnected elements in place <could server other purposes also>
Dr. Rahul Banerjee, BITS Pilani, India

Intelligent / Smart Spaces / Environments

How does context awareness / context predic1on combined with other elements like actua1on mechanisms etc. allow Smart or Intelligent Pervasive Compu1ng Environments?

Driver With body mounted wearable sensors as part of the BITS Lifeguard

Vehicular Computer

Outside world, preferably ITS-enabled (or at least 2.5G / 3G / 4G / Wi-Fi supported

Dr. Rahul Banerjee, BITS Pilani, India

So, how do the Pervasive and Wearable Computing systems go together?

Pervasive computing systems know about the context of the environment of the user(s) and involve multiple compute-sense-communicate elements embedded in users environment. Wearable computing systems know more about the context of an individual wearer and involve computesense-communicate elements mounted / embedded on / in the body of the user in various ways. A typical wearable computing system may either work independently or may form part of an inclusive larger pervasive computing solution.

Dr. Rahul Banerjee, BITS Pilani, India

Interaction Points
Typical Problems and Issues related to technology intervention in the Pervasive Computing environments
HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and Project Communicate Summary of Concepts learnt
Dr. Rahul Banerjee, BITS Pilani, India

Summary of the Concepts Learnt

Pervasive computing systems know more:
about the context of the environment of the user(s) and involve multiple compute-sense-communicate elements embedded in users environment.

Wearable computing systems know more:

about the context of an individual wearer and involve compute-sense-communicate elements mounted / embedded on / in the body of the user in various ways.

Dr. Rahul Banerjee, BITS Pilani, India

References to the Research at BITS Pilani (1 of 2)

Rahul Banerjee: From Research to Classroom: A Course in Pervasive Computing, IEEE Pervasive Computing, JulySept. 2005, Vol. 4, No. 3, pp. 83-86. Sailesh Conjeti, Rajiv Ranjan Singh, & Rahul Banerjee: Bio-Inspired Wearable Computing Architecture and Physiological Signal Processing for On-Road Stress Monitoring, Proceedings of the IEEEEMBS International Conference on Bio-medical and Health Informatics, Shenzhen, Hong Kong, Jan 5-7, 2012, pp. 479-482. Available on IEEE Xplore: pp. 479 - 482, doi: 10.1109/BHI.2012.6211621. Rajiv Ranjan Singh, Sailesh Conjeti & Rahul Banerjee: An Approach for Real-Time Stress-Trend Detection in Physiological Signals in Wearable Computing Systems for Automotive Drivers, Proceedings of 14th International IEEE Conference on Intelligent Transportation Systems - ITSC 2011, October 5-7, 2011, George Washington University, Washington, DC, USA, pp. 1477-1482. Mahesh M. Bundele & Rahul Banerjee: ROC Analysis of a Fatigue Classifier for Vehicular Drivers, Proceedings of the 2010 IEEE International Conference on Intelligent Systems (IS 2010), London, UK; <an IEEE international conference>< also available through the IEEE Explore> <July7-9, 2010>. Mahesh M. Bundele & Rahul Banerjee: Design of Early Fatigue Detection Elements of a Wearable Computing System for the Prevention of Road Accidents, Proceedings of the 2nd International Workshop on Intelligent System and Applications (ISA2010), Wuhan, China; <an IEEE international conference>. <Proceedings by the IEEE, also available through the IEEE Explore> <May 22-23, 2010>.

Dr. Rahul Banerjee, BITS Pilani, India

References to the Research at BITS Pilani (1 of 2)

Rajiv Ranjan Singh & Rahul Banerjee: Multi-parametric Analysis of Sensory Data collected from Automotive Drivers for Building a Safety-Critical Wearable Computing System, Proceedings of the 2nd International Conference on Computer Engineering and Technology (ICCET 2010), April 16-18, 2010, Chengdu, China; <an IEEE international conference>. <Proceedings by the IEEE, also available through the IEEE Explore> Mahesh M. Bundele & Rahul Banerjee: Detection of Fatigue of Vehicular Driver using Skin Conductance and Oximetry Pulse: A Neural Network Approach, ACM-sponsored International Conference: iiWAS2009, December 1416, 2009, Kuala Lumpur, Malaysia. Proceedings of the iiWAS2009, pp. 725-730 <Proceedings on paper by the ACM > Mahesh M. Bundele & Rahul Banerjee: An SVM Classifier for Fatigue-Detection using Skin Conductance for Use in the BITS-Lifeguard Wearable Computing System, Proceedings of the IEEEsponsored ICETET-2009, Nagpur, December 16-18, 2009, pp. 936-941. <Proceedings by IEEE CS on CD and available through the IEEE Explore>

Dr. Rahul Banerjee, BITS Pilani, India

E-Fabric of a Wearable Garment

Source: ETH, Zurich

Dr. Rahul Banerjee, BITS Pilani, India

Wireless Communication in eFabric

Source: ETH, Zurich

Dr. Rahul Banerjee, BITS Pilani, India

User Interface & Sensors

Elek Tex belongs to Eleksen. Peratech Ltd.

Dr. Rahul Banerjee, BITS Pilani, India

About the Cloud-Assisted Pervasive Computing

Several highly computationally intensive applications of pervasive computing benefit from the Cloud-assisted Pervasive Computing paradigm. Cloud-assistance in such cases often comes at the backend for handling various compute and data-intensive tasks related to a set of pervasive services. However, there is a distinct possibility of multi-layer / multi-level cloud support at even intermediate levels of support particularly in terms of collaboration between heterogeneous cloud units or between private clouds via the public cloud.

Dr. Rahul Banerjee, BITS Pilani, India

Interaction Points
About the Course Pervasive Computing: What is it? Examples of Pervasive Computing Environments Situations that warrant Pervasive Computing Solutions Typical Problems and Issues related to technology intervention in the Pervasive Computing environments
HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and Project Communicate Summary of Concepts learnt and the Exercise for the day

Dr. Rahul Banerjee, BITS Pilani, India

Pervasive Computing with AR

Dr. Rahul Banerjee, BITS Pilani, India

A Scent-emi6ng Display System

An innova1ve display system developed recently by the NTT Communica1on Corpora1on had been installed at an underground Mall situated in the Tokyos train sta1on on an experimental basis in July 2008. IT emits appe1zing aromas along with the adver1sing videos being displayed on a 42-inch LCD Display panel. This commercial tex1ng was done by Recruit Co. Ltd for adver1sing cafes and restaurants.

NEC, Japan, 2008

Dr. Rahul Banerjee, BITS Pilani, India

DIGITAL GRAFFITI ON SEE-THROUGH DISPLAYS IN AR SYSTEMS

Source: PERVASIVE 2006-7. May 2006 (c) Alois Ferscha

Dr. Rahul Banerjee, BITS Pilani, India

Source: PERVASIVE 2006-7. May 2006 (c) Alois Ferscha

DISPLAYS WHICH YOU CAN STEER

Steerable Displays

Dr. Rahul Banerjee, BITS Pilani, India

Summary of the Concepts Learnt

Pervasive Computing is all about designing and deploying solutions
that would use an inherently distributed infrastructure Such an infrastructure comprises of compute-sense-communicate elements and is often used to support various types of customizable services via appropriate software and services.

A pervasive computing system has an ability to recognize context of the target environment for the purpose of offering desired set of pre-defined as well as adaptive services, preferably without the users being required to be aware of its presence. The design principles of Pervasive Computing are often driven by the desirable characteristics like: Simplicity Versatility Pleasurability
Dr. Rahul Banerjee, BITS Pilani, India

Your exercise for the day!

Study Project Aura (http://www.cs.cmu.edu/~aura/) at CMU and identify the involved elements. Consider using the basic framework of the Project Oxygen for applications other than those conceived by the CMU researchers. Conceptualize your own model of a simple pervasive computing system that could allow a given room to be converted into a simple pervasive computing unit by using commonly available off-the-shelf devices and other elements.

Dr. Rahul Banerjee, BITS Pilani, India

References
<other than the Text Book and E-Notes>

Dr. Rahul Banerjee, BITS Pilani, India

Please take a look at the reading advisory at Nalanda LMS Portal for additional information!

Thank you very much for you time and attention!

My Home page may have some additional information of possible interest to you. You may want to access it at the URL: http://www.bits-pilani.ac.in/~rahul Just in case, you may wish to contact me by Email, my Email address is: rahul@bits-pilani.ac.in
Dr. Rahul Banerjee, BITS Pilani, India