REPORT PROCET HUMAN (HCI) ON VIRTUAL AGENT

PREAPRED TO : PN SUZANA BAHARUDIN & CIK HAZRATI

PREPARED BY :
NADYA BT YUSOFF

2010684346

IZZNIE BT IZZUDDIN
NUR HIDAYAH BT ZAFFRIE

201062
2010631294

Contents
1.0 Introduction to Ayu The Virtual Assistant..............................................................5
2.0 The Literature review of Ayu The Virtual Agent....................................................7
2.1 The Making of Ayu The Virtual Agent.................................................................7
Cropping and Slicing............................................................................................ 7
2

Drawing................................................................................................................ 8
Painting................................................................................................................ 8
Measuring and Navigation.................................................................................... 8
Selection.............................................................................................................. 8
2.2 The Story Telling and The Virtual Assistant........................................................9
3.0 Ayu The Virtual Agent conceptual model............................................................10
4.0 Ayu The Virtual Agent Illustration......................................................................11
................................................................................................................................. 11
................................................................................................................................. 11
5.0 Ayu The Virtual Agent......................................................................................... 12
6.0 Ayu The Virtual Agent Behavior..........................................................................13
7.0 Ayu The Virtual Agent Usability Testing..............................................................14
7.1 Usability Testing.............................................................................................. 14
7.2 The method chosen......................................................................................... 20
8.0 The sample of the Interview and Observation....................................................23
9.0 Conclusion.......................................................................................................... 25
10.0 Reference........................................................................................................ 26
Bibliography............................................................................................................. 26
11.0 Appendixes..................................................................................................... 27

1.0 Introduction to Ayu The Virtual Assistant

Ayu is a story teller or the narrator of our story telling courseware. Ayu will
be a guidance along the story will be held. The reason why we were
choosing Ayu as our virtual assistant is because Ayu has a personality that
can attract kids to read the story. Ayu act as a sister at age 12 years old
suitable for telling story to kids that younger than her. Ayu are presented to
be a Malay girl where she wears a scarf and she will be fully tell the whole
story in English.
3

Children especially seven and above are tend to be developing their mind at
this kind of ages, so that this is probably a perfect time to help them in
developing their mind. That is why we create Ayu in helping developing their
mind much easier. Kids are usually easily to learn in what theyve seen and
heard. Despite Ayu telling them a story, Ayu also teach them how to read by
listening in what she said. Since we want to tell an ancient story or Malay
fairytale story Ayu come up into our mind at that time, she shows the
characteristic that Malay girl have. She soft spoken, a good story teller,
pretty, and has ability to attract kids.
The look and feel that she has seems to be really nice and attractive. Ayu
also being create to make the story become real and fun since Ayu can
interact with the user. Even though, Ayu has limited movement she shows a
really good expression on her face that seems simple but it worth to attract
users view.
Our usability goals is that this virtual agent is effective to use where Ayu the
agent is easily recognize by children . Ayu as a narrator will make children
easily how to read or understand what the story all about. Before this
children are not being exposed with digital story telling with an agent read to
them so here we create Ayu to make the story more attractive and
effective. Ayu only given a task in giving instruction and explanation in the
story.
Ayu is efficient where the user or children only need to listen with what she
will say no need to press or in computational requirement . The lack is Ayu
has limitation in interact with the user , she could not really interact well with
user since she only giving an instruction and tell a story to the user. Ayu only
given a simple instruction so that children will easily understand what she
saying.
Ayu is safe to use since there is no highly command required that will harm
the user. There is no complicated error that will make all the data in the

system being deleted. Our product has system that allow user the
opportunity to consider their intention if they want to delete the file it will be
a pop up instruction if they want to remove the item or not.
Our agent does not need any key press to interact with her. Only listen what
the instruction given by her. She does not need any button like if press will
pop up instruction she completely being program to give all the instruction
without waiting command from user.

2.0 The Literature review of Ayu The Virtual Agent

2.1 The Making of Ayu The Virtual Agent
Flash
We create character Ayu by using Flash. Adobe Flash (formerly Macromedia
Flash) is a multimedia platform used to add animation, video, and
interactivity to web pages. Flash is frequently used for advertisements,
games and flash animations for broadcast. Flash manipulates vector and
raster graphics to provide animation of text, drawings, and still images. It
supports bidirectional streaming of audio and video, and it can capture user
input via mouse, keyboard, microphone, and camera. Flash contains an
object-oriented language called Action Script and supports automation via
the Java script Flash language (JSFL).
Flash content may be displayed on various computer systems and devices,
using Adobe Flash Player, which is available free of charge for common web
browsers, some mobile phones and a few other electronic devices (using
Flash Lite).
Photoshop
We also use Adobe Photoshop in editing Ayus appearance . The tools that
provided inside this software are :
Cropping and Slicing
The crop tool can be used to select a particular area of an image and discard
the portions outside of the chosen section. This tool assists in creating a
focus point on an image and excluding unnecessary or excess space.

The "slice" and slice select tools, like the crop tool, are used in isolating parts
of images. The slice tool can be used to divide an image into different
sections, and these separate parts can be used as pieces of a web page
design once HTML and CSS are applied. The slice select tool allows sliced
sections of an image to be adjusted and shifted.
Drawing
Photoshop includes a few versions of the pen tool. The pen tool creates
precise paths that can be manipulated using anchor points. The freeform
pen tool allows the user to draw paths freehand, and with the magnetic
pen tool, the drawn path attaches closely to outlines of objects in an image,
which is useful for isolating them from a background.
Painting
A few of the painting tools in Photoshop are
the brush, pencil, gradient and paint bucket tools. These can be used to
retouch images by altering and adding colored pixels. Original paintings can
be created using the brush tool, since it has many different default brushtips, and brush strokes be easily adjusted in size, shape and texture. There is
an option to create custom brushes to suit the user's needs, and original
brushes and brush packages can be found on the web.
The pencil tool allows the user to paint rough strokes on the canvas, creating
a similar effect to a real pencil on paper. The gradient, fill and paint bucket
tools are similar to each other, as they each fill areas of the canvas with
color.
Measuring and Navigation
The eyedropper tool selects a color from an area of the image that is clicked,
and samples it for future use. The hand tool navigates an image by moving it
7

in any direction, and the zoom tool enlarges the part of an image that is
clicked on, allowing for a closer view.
Selection
The marquee tool can make selections that are single row, single column,
rectangular and elliptical. Once an area of an image is highlighted,
the move tool can be used to manually relocate the selected piece to
anywhere on the canvas. The lasso tool is similar to the marquee tool,
however, the user can make a custom selection by drawing it freehand. In
addition, the lasso tool can make magnetic and polygonal selections.

2.2 The Story Telling and The Virtual Assistant

The story is basically based on the ancient Malay fairytale. Actually we
change the plot of the story a little bit. The conclusion of this story we
change to make children mere interested to read and enjoy it. The agent Ayu
will read trough out the story and that make this story a different from
children usually read. The graphic of the story we make it look more
attractive and we also include dialog box , every dialog box will be a different
picture and each picture will be a voice that will parallel with the images and
word. User will be able to choose a different conclusion of the story where
there will be one of two item (seed) they need to choose and the item will be
a key for a good and bad conclusion.
The story is about one old King known as Sultan Alam Syah that does not
has any heir that will take his place. So he decided to take one og the village
son to be his son. With one condition each of them must grow one of two
seed that he give. Adam will be the one of our focus he will choose which
bean he want to grow, Adam is the user. If Adam choose green bean he will
not been crown as a prince because the seed has been color to be seen
beautiful, but if he choose brown he will be crown as a prince since he pass
the king test which want to test them does them choose the bean according
to their physical look or not.

3.0 Ayu The Virtual Agent conceptual model

Conceptual
Model

Metaphors
And
analogies

Digital and
interactive
story telling.

Agent will
read through
all the story.

Concepts

Relationships
between
concepts

Mapping

Agent will
give
instruction to
choose.

Giving
instruction
what to so
next.

Narrator

4.0 Ayu The Virtual Agent Illustration

11

5.0 Ayu The Virtual Agent

Ayu is the name of our virtual agent for this application. She is about the age
of 12 years old. She is a Malay and came from Kedah. We describe her with a polka
dots shawl and a pair of spectacles. She wears a long sleeves t-shirt and about
145cm tall. She is described as a soft spoken person where she act as the narrator
for this story entitled Pasu Tanpa Bunga. She also helps the children in reading the
story and giving instructions in this application.

6.0 Ayu The Virtual Agent Behavior

According to the interaction design first basic activity which is the
establishing the requirements, Ayus behavior need to be clarified and re-scoped
according to the children needs. We can put here the behavior of Ayu is helping the
children in reading the story. Besides that, Ayu helps giving instruction to the
children about what they need to do to proceed the application.
The second basic activity is designing alternatives. In designing alternatives,
there are two phase, the conceptual design and the physical design. In the
conceptual phase, we design the appearance of Ayu the virtual design, with the
look of a malay girl that wears a long sleeves t-shirts with a polka dots shawl and a
cute face. In the physical phase, we describe that Ayu is a soft spoken person which
interacts cutely with the children.
The third activity is the prototyping. We done the prototyping by drawing Ayu in the
computer. We draw Ayu in different movement. After that we combine all the edited
pictures in Flash to make it an animation.
The last activity is the evaluation. This activity is very important as we need
to evaluate the virtual agent as to improve it to be a better agent. In evaluation, we
used the method interview and observation.

13

7.0 Ayu The Virtual Agent Usability Testing

7.1 Usability Testing

Involves recording performance of typical users doing typical tasks.

Controlled settings.

Users are observed and timed.

Data is recorded on video & key presses are logged.

The data is used to calculate performance times, and to identify &

explain errors.

User satisfaction is evaluated using questionnaires & interviews.

Field observations may be used to provide contextual understanding.

Four basic activities in Interaction Design

1. Establishing requirements
2. Designing alternatives
3. Prototyping
4. Evaluating
Establishing requirements

What do users want? What do users need?

Requirements need clarification, refinement, completion, re-scoping

Input:

requirements document (maybe)

Output: stable requirements

Why establish?

Requirements arise from understanding users needs

Requirements can be justified & related to data.

Different kinds of requirements

Functional:
What the system should do
Historically the main focus of requirements activities

(Non-functional: memory size, response time...)

Data:
What kinds of data need to be stored?
How will they be stored (e.g. database)?

Environment or context of use:

physical: dusty? noisy? vibration? light? heat? humidity? .
(e.g. OMS insects, ATM)
social: sharing of files, of displays, in paper, across great
distances, work individually, privacy for clients
organisational: hierarchy, IT departments attitude and
remit, user support, communications structure and
infrastructure, availability of training
Users: Who are they?
Characteristics: ability, background, attitude to computers
System use: novice, expert, casual, frequent
Novice: step-by-step (prompted), constrained, clear information
Expert: flexibility, access/power
Frequent: short cuts
Casual/infrequent: clear instructions, e.g. menu paths

15

Emotions and the user experience

HCI has traditionally been about designing efficient and effective

systems

Now more about how to design interactive systems that make people
respond in certain ways

e.g. to be happy, to be trusting, to learn, to be motivated

Emotional interaction is concerned with how we feel and react when

interacting with technologies

Emotional interaction

What makes us happy, sad, annoyed, anxious, frustrated, motivated,

delirious and so on - translating this into different aspects of the user
experience

Why people become emotionally attached to certain products (e.g.

virtual pets)

Can social robots help reduce loneliness and improve wellbeing?

How to change human behavior through the use of emotive feedback

DECIDE
D - Determine the goals.
The main usability goals we would like for our system are effectiveness
and efficient. We not sot bothered about whether the system is safety
as we will focus on is the application easy to use by user or not. From
the list of user experience goals, there are definitely some that we
dont wish to be associated with our Pasu Tanpa Bunga Story Telling
application plus the agent such as annoying ,frustrating and boring,
enjoyable and fun would be additional bonuses as the main purpose of
using the application is to help children understand how to read digital
story book.

E - Explore the questions.

There are questions need to be answered in order to make goals
operational. We have broken down into number of relevant
questions for investigation. Why are children should be helping by
virtual agent while reading the graphic animated story book? Perhaps
an application that can be replayed and that can guide children until
they are expert at reading so that they could easily read some other
sources.

C - Choose the evaluation method.

The choice will depend on what is needed to answer the questions and which
theories or frameworks are appropriate to our context. Combinations of
methods are used as they enabled richer data to be gathered. Data collected
using different methods gives a broad picture of how well the design meets
the usability and user experience goals that were identified during the
17

requirements activity. Methods chose are the DECIDE framework and

Heuristic evaluation method. While the sotwares used are Flash, ActionScript,
Audacity, and Photoshop.
I - Identify the practical issues.
We started doing this was the need of kindergarten ,pre-school and
primary school to learn how read the story book with an agent helping
them. This children they are not seems easily with the simple not
animate story book. They need something that can develop their
imagination with the agent helping them and with this application they
can use it anywhere without anyone read to them.
D - Decide how to deal with the ethical issues.
We develop an informed consent form the children have a right to
know the goals of the study, what will happen with the findings and
privacy of personal information. The next thing we focused on was the
interactivity that the children will have with the system. We would want
them to be taught and understood better than when learning in school.
For example they can go to the next and previous slide. This would
make it easier for those who are slow or fast at catching up.

E - Evaluate, analyze, interpret, and present the data

Data gathering
5 key issues :
1. Setting goals
Easy to learn
In addition to easily understanding functionality of
multimedia software, multimedia systems should also be

easy to remember. Casual users should have no problems

in remembering how to use and navigate in the system
after periods of non-use.
Memoribility could give users the ability to transfer their
knowledge of use and navigation of one information based
on the use of another information base with same engine
(Nielsen, 1990a, 1990b).

Effective to use
Multimedia products should be designed to achieve a high
level of productivity.
Effectiveness measured in terms of speed and error, refers
to level of users performance (Lingard, 1994, Shackel,
1991).
After learning the multimedia software, users should
become more expert at using them over time (Robertson,
1994).

2. Identifying participant - Target user : Children age 5 to 10.

3. Relationship with participant - Between developer and
participant, make sure the data gathering from participants must
be clear and professional.
4. Triangulation - Term used to refer to investigation of a
phenomenon from at least two different perspective (Jupp, 2006).
We will observe and collect data from various sources. Example :
observe and interview the kids from each age.
5. Pilot studies - Test the application to a small group of participants
which range from the age 4 to 6 - year old in kindergarten, and
test also on 7-year old in primary school.

19

7.2 The method chosen

Data Recording
We have decided to use notes, audio, and autographs in order to record the
data.
Techniques to do the data gathering
Interview
Observation
Interview
We have came out with a few questions for the interview session based on
our justification on learnability and effectiveness.
Effectiveness
In order to get the data from this usability goal, we would like to know
from participant based on these :
1. Do you immediately understand the function of each button in
2.
3.
4.
5.

the application?
Is Ayu helping you understand how to read story book?
Do you understand every words and sentences that Ayu said?
Does Ayu help you understand how the application works?
Is the tell this application easy to use?

The question would look like these:

1.
2.
3.
4.

Adakah
Adakah
Adakah
Adakah

adik
adik
adik
adik

faham kegunaan button yang disediakan?(Ya/Tidak)

tahu yang mana satu button kembali ?(Ya/Tidak)
suka pada penampilan Ayu? (Ya/Tidak)
faham apa yang Ayu ceritakan dalam cerita ini?

(Ya/Tidak)
5. Adakah suara Ayu jelas? (Ya/Tidak)
6. Adakah adik faham bagaimana jalan cerita ini? (Ya/Tidak)

 Efficiency
In order to get the data from this usability goal, we would like to know
from participant based on these:
1.
2.
3.
4.
5.

Are the text transition too fast/slow?

Is Ayu talk too fast/slow?
Does Ayu help you in choosing the seed?
What about the font of the text?
Does the transition between one page to another page take a
long time to load?

The question would look like these:

1.
2.
3.
4.
5.

Adakah
Adakah
Adakah
Adakah
Adakah

adik tahu button mula yang mana? (Yes/No)

Ayu pencerita yang baik? (Yes/No)
semua tulisan senang dibaca? (Yes/No)
Ayu bercakap terlalu laju/perlahan? (Yes/No)
slide dimana agent keluar terlalu perlahan/laju? (Yes/

No)

Observation
In order to handle the observation, we adopted Robson (2002) framework
which encourages observers to pay greater attention to the context of the
activity.
Actors : What are the names and relevant details of the actors

involved?
Activities: What are the actors doing and why?
Acts: What are specific individual actions?
Events: Is what we observe part of a special event?
Goals: What are the actors trying to accomplish?
Feelings: What is the mood of the group and of individuals?

Thus we will find a group of children in the range of 4 - 7 and will be

introduced to our application. We will find the answers to the question on
How are these children going to use the application? How will they behave
when running the activity? and such. We will naturally pick up a pen and
paper and note down what we observe from their behaviour. We will also
21

take some photographs or videos and keep on observing them when they are
learning from the application themselves.
OBSERVATION
Based on framework suggested by Robson(2002), we have gathered the
results of the observation. (2 examples)
First :
Actors : Danial , 7 years old, moderate reader.
Activities: Focusing, at beginning he slightly confused with the story
but then he could figure out what happen in the story and he like the

agent.
Act : Eager to drag and drop the seed.
Events: Yes, because we test on the first timer.
Goals: He trying his best to understand the conclusion.
Feelings: He enjoy the story and want some other story like this story.

Second :
Actors : Adriana , 9 years old, advance reader.
Activities: Focusing, easily to figure out what to do, and a lot of

questioning. Able to understand what does the agent says.

Act : Listening carefully, and try to understand what the story about.
Events: Yes, to look to the reactions and feedbacks.
Goals: Could finish reading in minimum time.
Feelings: Enjoy doing the activity, rushes to finish them.

8.0 The sample of the Interview and Observation

Interview :
1.
2.
3.
4.

Adakah
Adakah
Adakah
Adakah

adik
adik
adik
adik

faham kegunaan button yang disediakan?(Ya/Tidak)

tahu yang mana satu button kembali ?(Ya/Tidak)
suka pada penampilan Ayu? (Ya/Tidak)
faham apa yang Ayu ceritakan dalam cerita ini?

(Ya/Tidak)
5. Adakah suara Ayu jelas? (Ya/Tidak)
6. Adakah adik faham bagaimana jalan cerita ini? (Ya/Tidak)

7. Adakah adik tahu button mula yang mana? (Yes/No)

8. Adakah Ayu pencerita yang baik? (Yes/No)
9. Adakah semua tulisan senang dibaca? (Yes/No)
10.
Adakah Ayu bercakap terlalu laju/perlahan? (Yes/No)
11.
Adakah slide dimana agent keluar terlalu perlahan/laju?
(Yes/ No)
Observation:
First :
Actors : Danial , 7 years old, moderate reader.
Activities: Focusing, at beginning he slightly confused with the story
but then he could figure out what happen in the story and he like the

agent.
Act : Eager to drag and drop the seed.
Events: Yes, because we test on the first timer.
Goals: He trying his best to understand the conclusion.
Feelings: He enjoy the story and want some other story like this story.

Second :
Actors :

Adriana , 9 years old, advance reader.

23

 Activities: Focusing, easily to figure out what to do, and a lot of

questioning. Able to understand what does the agent says.

Act : Listening carefully, and try to understand what the story about.

Events: Yes, to look to the reactions and feedbacks.

Goals: Could finish reading in minimum time.
Feelings: Enjoy doing the activity, rushes to finish them.

9.0 Conclusion
We conclude that children are tend to read more on the animated
graphic with text transition and sound included in story book with an agent
helping them. They really enjoy it since their mind are develop more on
what they see and hear. Ayu the agent is be able to guide along the story.
She read and giving instruction and that help kids more interested towards
this application. The children seems to like more if there the existence of the
agent. Therefore, our agent Ayu are completely helping in delivering the
story and her attractive appearance could be able attract children to have
read and understand well the story.

25

10.0 Reference
Bibliography
Suhaiza. (2011).Warisan Kisah Klasik Melayu. Malaysia; Edukid Publication Sdn. Bhd.
Yvonne Rogers,Helen Sharp,Jennifer Preece.(2011).Interaction Design,United
Kingdom;John Wiley & Sons Ltd.

http://cerita-nazihah.blogspot.com/2012/02/pasu-tanpa-pokok-bunga.html
http://en.wikipedia.org/wiki/Adobe_Photoshop
http://en.wikipedia.org/wiki/Adobe_Flash
http://www.chatbots.org/virtual_agent/
http://en.wikipedia.org/wiki/Conceptual_model_(computer_science)
http://en.wikipedia.org/wiki/Conceptual_model

11.1

Appendixes

27

Brown Seed

29

Green Seed