A Conceptual Virtual Reality Model
John N. Latta
irtual reality is an advanced human-computer interface
U that simulates a realistic environment and allows partici-
pants to interact with it. Experimental psychology provides a
foundation for applying systems concepts to human participa-
tion in an environment. The complexity of the human interface
makes it impossible to completely characterize this participa-
tion, whether the environment is real or computer generated.
However, we use this foundation here to propose a conceptual
model that examines both the human and technical elements of
a VR system.
We feel that the current fixation on immersive VR systems
fails to encompass the potential of the technology. The core of
VR is the human-computer interface. A VR system might range
from a flight simulator to a synthetic environment to telepres-
ence. The definition and conceptual model for the technology
should be robust enough to encompass all three forms, as well
as others that develop as the technology evolves.
Definition of virtual reality
Virtual reality involves the creation and experience of envi-
ronments. Its central objective is to place the participant in an
environment that is not normally or easily experienced. This
objective is satisfied by establishing a relationship between the
participant and the created environment.
Accordingly, we developed a three-tiered definition of VR,
January l Y Y 4
4th Wave
David J. Oberg
Global Outpost
This conceptual model
isolates and describes the
human and technical
elements that create the
participatory environments
of virtual reality systems.
shown in Figure 1 on page 24, that addresses respectively what
VR is, how it is accomplished, and its effect-both of VR on the
participant and of the participant on the environment. The dis-
tinguishing what of VR is its extension of the human-com-
puter interface. Engineering discipline requires precise and
consistent definitions of each interfaced system. However, de-
spite numerous studies,-3there are no such definitions of human
perceptual systems. We must therefore rely on more qualitative
definitions. As a basis for our conceptual VR model, we adopted
J.J. Gibsons descriptions of human perceptual and muscle sys-
temsJ (see Tables 1 and 2 on pages 24 and 25). Gibson began as
an experimental psychologist and throughout his career devel-
oped a systems view of perception. In one of his last works, he
described the visual system in the context of what he called eco-
logical psychology. This contextual concept helps in under-
standing a participants interaction with VR environments.
Comparing the list of perceptual systems in Table 1with the
list of muscle systems in Table 2 suggests the degree of integra-
tion possible between them. We believe that VR interface tech-
nology must integrate both systems. Again, the complexity of
the human interface makes it impossible to define an environ-
ment that fully interfaces to all the perceptual and muscle sys-
tems. Our model will therefore examine VR systems for only
some perceptual systems, not all.
The how tier in Figure 1 addresses the creation of a realis-
23
 Virtual Reality

Figure 1. VR objective and

definition.
To place the j Computer-based
participant in an ~ interface to
environment that is j human perceptual
not normally or ; and muscle
easily experienced ; systems

-
(Experiential)
To have a significant
personal experience
while participating in
an environment
tic interactive sensory environment. This requires linking the
sensory stimulation defined in tier 1 with the effect of VR par-
ticipation defined in tier 3. Realism is a qualitative judgment,
achieved when the participant perceives equivalence between in-
teractions with an artificially created environment and interac-
tions with a real environment.
Complete perceptual equiva-
lence is impossible, but there
is a subjective threshold at
which interactions within the
VR system feel natural and
(Operational) the environment seems real.
To perform The compelling aspect of
operations while
in an environment VR is the experience it cre-
ates, yet many VR applica-
tions have no experiential
element. They involve simply
accomplishing a task; the feel-
ing from the process is secondary at best. We have therefore di-
vided the effect tier into two components, experiential and
operational. We can define an operational VR system as one
that provides a computer interface to specific human perceptual
and muscle systems for the purpose of allowing the participant

Table 1. Perceptual systems.

System Mode of activh Receptive units Ewtemai information

Orienting Posturing and Mechanical Vestibular Forces of gravity Direction of gravity

orienting and gravity organs equilibrium and acceleration or acceleration

1 1
receptors and balance

Auditory Listening Mechanical Cochlear organs Orienting to Vibrations in Nature and location
receptors sounds he,t of vibratory events

I I

Haptic Touching Mechanical, Skin, joints, Exploration of Deformation Contact with object
thermal, and muscles, and many kinds of tissues, surfaces and shapes,
kinesthetic tendons including configuration material states,
receptors appendages, of joints, stretchins solidity and viscosity,
skin, and tongue of muscle fibers heat and cold

Taste-smell Tasting Chemical and Oral cavity Savoring Chemistry Nutritive and
mechanical (mouth) of ingested biochemical values
receptors objects

Smelling Chemical Nasal cavity Sniffing Chemistry Nature of odors

receptors (nose) of vapors

I I
Visual Seeing Photo receptors Ocular Accomodation, Light Size, shape, distance,
mechanism pupillary location, color, texture,
including eyes adjustment, and movement
and whole-body fixation,
movement convergence,
scanning

Adapted from J.J. Gibson, The Senses Considered us Perceptirul S y s t e m , Houghton-Mifflin. Boston, 1966.

24 IEEE Computer Graphics & Applications

- -

System Purpose
Postural Orientation with gravity and
-
acceleration forces
Orienting- Movement of body parts to
investigating obtain external stimulus
~
Locomotor Movement of body or body parts
to other parts of environment
Appetitive Movement of body parts to take
from or give to environment
Performatory I Movement beneficial to the individual I Take action, move items, protect self
Expressive Movement to express self, display
emotion, or identify self
Sematic Movements to signal action,
state, or expression
Adapted from J.J. Gibson, The Senses Considered (IS Percc,ptirn/ Systems. Houghton-Miftlin, Boston. 1966.
to perform operations that would not normally be possible.
Flight simulators are an excellent illustration of this.
Obtaining precision in the implementation of experiential
VR systems is an art form-comparable, for example, to the
ability of theater or film directors to evoke audience emotions.
VR systems for entertainment rely on the experiential effect
as the objective of the participation. It is likely that the lessons
learned from game design will provide some of the best guide-
lines for VR experiential design.(
Our conceptual model examines both a human and a techni-
cal view of a VR system, as shown in Figure 2. The human per-
spective centers o n the physical and psychological issues of
providing the stimulus to and detecting the actions of the par-
ticipant. The technical view refers to the physical space, func-
tions, processes, equipment, and concepts that define the
environment. The environment provides the stimulus that cre-
ates sensation while the individual takes action through move-
ment. The environment, in the VR context of this model. is the
total space, both real and artificial.
Human view
Figure 3 o n page 26 shows the participant within an environ-
ment, interfacing with the VR system through effectors and
sensors. The VR system uses effectors to stimulate the receptors
of the human perceptual system. These effectors can include, for
example, a helmet-mounted display, a force-feedback device. or
even an odor emitter. The system detects the participants ac-
tions, as expressed by the muscle systems, with sensors.
This human-centered view characterizes the participant in
the center of the environment. It highlights both the perceptual
and muscle systems as functional flow elements, showing cog-
nitive and psychological processes operating on output from
the perceptual systems and generating input to the muscle sys-
tems. This schematic also includes emotional responses that
could be reflected in actions. Further, because no individual is
January 1994
A Conceptual Virtual Reality Model
Application Other systems
Maintain body equilibrium Vestibular organs
Sense information or explore All other senses
Go from one location to another Orienting-investigative and
postural
Ingest or relieve Taste, ingestion, and
other body functions
I Locomotor and others
Make postural, facial, and vocal Voice, hearing, and facial muscles
movements
Voice expression Any other system based on
signal intents
Figure 2. VR system: Participant and environment.
I Human view Technical view
Environment
Participant
as simple as a series of functional flow blocks, we have listed
other factors on the left. including individual experience, er-
gonomics, and sensual sensitivity. Our ability to quantify the re-
lationship between stimulation from effectors and responses
from sensors with either an individuals personal experience of
participation or their performance is limited, except in special
cases. Without such quantification, the design of V R systems
with specific objectives will be heuristic.
A participant might consciously or unconsciously use differ-
ent criteria to evaluate or evoke the experience of participa-
tion. We categorize these criteria as either psychological,
anatomical, or individual. The anatomical criteria include, for
example, individual ergonomics. Such criteria are the easiest
to define. because they involve measuring the participants body
and fitting it with sensors and effectors. The psychological cri-
teria include the general foundations for defining human in-
teractions (for example, experimental. cognitive, and ecological
psychology). These criteria and those addressing the specific
profile and background of individual participants are much
more difficult issues. Csikszentmihalyi described an optimal
human experience, which he called flow. It is based on six
factors that address such things as the sense a person has of
time passing while engaged in a flow activity. However, flow is
a post-activity assessment. It does not provide clear rules to de-
25
Virtual Reality
Environment
I ronment varies, as it does, for
sign a potential experience that will meet the flow criteria. It
might nevertheless prove useful in VR development. HeeteF
used flow to describe the responses of participants to Bat-
tleTech. a location-based entertainment attraction.
Sensors and effectors
VR systems use sensors and effectors to create the partici-
patory environment. We define three classes of effectors:
1. Shell, a stationary enclosure designed with an environ-
mental and participatory objective.
2. Fixture, an environment-fixed device used to create sen-
sory stimulations.
3. Appliance, a participant-fixed component worn on or near
the body to stimulate one or more senses.
Shells are usually large and special-purpose. Fixtures can
range from C R T displays to a room-size tracking system. Ap-
pliances can range from a body suit to a pair of goggles. Al-
though appliances close the distance between the participant
and the stimulation, they are intrusive and people generally do
not like wearing them. Currently, the so-called immersive VR
systems are based on personal appliances that block stimulation
from the actual environment.
The immersive experience is but one class of VR systems
that meet the definition in Figure 1. VR could fittingly be called
environment interface technology. However, computer sci-
ence today emphasizes object technology and thus tends to clas-
sify VR systems on the basis of the interface equipment. We feel
that this classification masks the fundamental contribution of
the technology. Our definitions of appliances, fixtures, and
shells focus on providing an interface to perceptual systems,
not on describing what the interface looks like.
Sensors work in conjunction with effectors to enable inter-
action with the environment. For example, tracking devices in-
terface with a muscle system to detect participant actions.
26
Figure 3. VR system:
Human view.
Fixtures and appliances often
Virtual include both sensors and ef-
reality fectors operating indepen-
Sensors 4 I dently. Shells provide a
common interface between
the orienting perceptual sys-
tem and the postural and ori-
enting-investigating muscle
systems (see Tables 1and 2).
The degree to which the shell
represents the actual envi-
example, in flight simulators.
T o understand the rela-
tionship between the partici-
pant and the sensors and
effectors, we must define the
partipatory environment.
Space definition and environment
management
The definition and integration of the real and artificial envi-
ronments defines the participatory environment. The artificial
environment comprises two elements:
1. Space definition,including volume, spatial, object, and time
definitions; physical laws; and material flow; and
2. Environment management, including dynamics, elimina-
tion, creation, scaling, sampling, and time control.
Space definition elements exist independent of the experi-
ence, and environment management elements exist dynami-
cally during the experience. The definition of the artificial
environment includes all space definition elements; however, it
might also include the rules for environment management. The
flight simulator industry has developed technology to define
worlds as databases. The scope of VR worlds can include struc-
tures, such as buildings; the human body; and environments of
varying scales. Worlds associated with different scientific disci-
plines will likely be quantified by different methods. For ex-
ample, the tools used to describe C A D objects from 3 D
cross-sectional scans will have limited application in describ-
ing the human body.
Both the quality of the experience in VR and the partici-
pants ability to function effectively are determined by how well
the individual perceives the participatory environment. The
participatory environment is created through a spectrum of
stimulations to perceptual systems and actions by muscle sys-
tems. This spectrum of stimulations and responses may cross be-
tween real and artificial environments. For example, visual
displays use a combiner to superimpose an artificial image onto
the real field of view.
The interaction of perceptual and muscle systems is espe-
cially complex with haptic stimulation. As Table 1showed, the
lEEE Computer Graphics & Applications

Figure 4. VR system:
Technical view.
organ anatomy for the haptic
perceptual system includes
the postural and orienting-in-
vestigating muscle systems.
Cutaneous sensation comes
from the skin, while proprio-
ceptive sensation originates
with muscles, tendons, and
other internal organs. The
complexity of this interaction
is revealed in a recent driving simulator that claimed VR at-
tributes. The simulator generated acceleration motion cues by
delivering force impulses to pads within the seat. However. the
resulting proprioceptive sensation has little relation to the G
forces normally experienced during driving. Thus. this imple-
mentation does not interface with the muscle systems in a nat-
ural fashion and does not achieve sensory equivalence.
It is unrealistic to expect perceptual equivalence. The human
sensory system is too complex. For years, developers have tried
to create a full-field-of-view display with either a fixed instal-
lation or head-mounted display. Such a display must match the
resolution of the eye in the context of head and eye movements.
A number of compromises are required to achieve effective re-
sults that are affordable and. in the case of personal appliances.
comfortable. Thus. the V R engineering task requires an un-
derstanding of the human perceptual systems to guide the de-
sign. Certainly the literature on the visual system is rich, and the
foundation for the human orienting system is well defined for
flight simulation.""' However. that same level of understanding
does not exist for the haptic perceptual system."
Technical view
Robinett" developed a taxonomy for synthetic experience.
This taxonomy is the first thorough classification of experiences
that uses the concept of technical mediation. Zeltzer" has used
an autonomy, interaction. and presence (AIP) cube to describe
the taxonomy of graphic simulation systems. Our VR model
builds on the work of Robinett. It includes not only mediation.
but also creation and interaction within a participatory
environment.
Figure 4 illustrates the technical view of a VR system. It in-
cludes the real environment and a new component called the
fechnical corifection. Confection is the process of preparing or
making, especially by combining. In VR we are confecting a
participatory environment by combining a real environment
with an artificial one. The technical confection includes a con-
fection model that achieves interface control. defines the arti-
ficial environment, and mediates between the participant and
the real environment.
Technical confection also establishes the relationship between
the sensors and effectors for the participant and the environ-
ment. Some VR implementations, such as telepresence. can be
seen more clearly from a technical view that highlights the in-
termediary function between the participant and the environ-
January 1994
A Conceptual Virtual Reality Model
~ Virtual reality system
1 I environment
Rea'
~~
Figure 5. Functional components of the confection model.
interface interface
Artificial Environment Real
environment control environment
Representation
definition
1
ment. Also. the technological view reveals another participant
sensor, a body state sensor. It is not a part of the muscle sys-
tems: rather. it monitors involuntary body conditions, such as
temperature, pulse rate, and blood pressure. Certain high-stress
or critically functional VR environments may require such sen-
Tors. which are commonly used with astronauts, for example.
Figure 5 presents the functional elements of the confection
model. There are two control functions: one for interfaces (at
the top of the figure) and one for environments (in the center).
Environment control implements the combination of the real
and artificial environments. The functions under environment
control include space definition and environment management,
which operate as described earlier. Mediation functions apply
when the confection model operates as an interface between the
participant and the real environment. (Mediation can apply to
artificial environments, but we see little practical use for such
functions there.) Representation functions refer to different
forms the participant can take within the artificial or real envi-
ronment. These include agents. robots, or personifications with
such design parameters as autonomy. intelligence, and quantity.
We feel that the V R model must support representation. How-
ever. the scope of representation and its impact on participation
await further research.
27
, -
Virtual Reality

Interface
mapping Participant Model Environment
Participant participant Environmental
perceptual effectors state determination
systems via sensors
Visual Displays Camera
Auditory Speakers Microphone
Orienting Motion platform Accelerometer
The confection model can Haptic Switches Pressure
support independent models Taste-smell Pressure applicator Thermometer
for each perceptual system. It Others Infrared imaging
Others
also supports independent
models for each muscle sys- Participant Participant Environmental
tem, but the participants de- muscle sensors state modification
tection of action is usually systems via effectors
correlated with perceptual Postural Trackers Remote manipulator
systems. The confection Orienting/ Glove Motion platform
investigating Foot pedal Lighting
model is the central element Locomotor Body suit Sound
in determining the participa- Appetitive Microphone Material flow
tory environment. The pa- Performatory Switch Others
rameters that define its Expressive Others
Sematic
properties are interface map-
ping, model source, and time
and space. Figure 6. Interface mapping.
Figure 6 presents example
interface mappings for each Figure 7. Model source.
of the perceptual and muscle
systems identified in Tables 1
and 2, listed in order of most Model
common usage. For the source Artificial Real
model, we have listed exam- Dynamic Direct
ple effectors and sensors. We Model dynamically changes 1:1 mapping between the
d o not intend this list to be during the participation space or interface
based on the actions of the parameters as experienced
limiting; rather, it illustrates participant or other events. by the participant.
types of shells, fixtures, and Model database changes
appliances. For the environ- dynamically. Sampled
ment, we have listed sensors Limited spatial or interface
Constructed parameter resolution.
and effectors that interface to Model is defined a priori as a
the real environment. Inter- fixed space and objects. Modified
face mapping establishes the Model database is static. Modified space or interface
functional connection be- parameters, such as gain,
tween these elements. Most Recorded frequency response, or time
Time recording of the space variable.
mappings are as simple as a or interface parameters.
horizontal line connecting the Recorded
elements. Time recording of the space
Figure 7 describes confec- - Transparency - or interface parameters.
tion model sources. In artifi- (relative contribution between artificial and real
cial space, the model is either space components to create the environment)
dynamic, constructed, or
recorded. A dynamic model
describes the space changes
based on the participants action, while a constructed model is pilots heads-up display might show images and information on
a static model database. Recorded models capture a specific a transparent mask that overlays the pilots real vision. The
sequence of actions for later replay by the participant: as the fig- combination may also occur inadvertently as in a tactile feed-
ure shows, they apply to both artificial and real environments. back device that feels mostly like a tight and cumbersome glove.
Real environment models also include direct, sampled, and Figure 8 lists the parameters that allow the confection model
modified models. The transparency shown between the artifi- to change time and space properties of real or artificial envi-
cial and real model sources signifies that the sources can be ronments. An example of time alteration is a flight simulator
combined. So long as an effector delivering components of an that lets participants jump ahead to a desired point without
artificial model source does not fully mask the input to a per- flying the whole way there. Manipulation of the space param-
ceptual system, then the experienced model will be a combina- eter is illustrated by microteleoperation, where the participants
tion of the artificial and real environments. For example, a perspective and operations are reduced t o the size of molecules.

28 IEEE Computer Graphics & Applications

 A Conceptual Virtual Reality Model

Figure 8. Time and space

Time Space
parameters.
Direct Direct
1 :1 correlation between time (x, y) matching of the
in the environment and the participant space and the
participant involvement. environment
Multiple Distance
(nt)time modification (mz)distance scaling of the
The direct parameters reflect between participant space participant space and the
and the environment environment
the participants natural ex-
perience of space and time. Fixed Scaled
There is a natural hierarchy ( T ) fixed time between I(x, x 4 or (nx, my, 0.3
in managing and controlling participant space and the scaling of distance for the
environment spatial dimensions between
a V R system based on the pa- participant and the
rameters of the technical con- Remapped environment
fection model. First, the f(t) functional remapping of
interface mapping supports time between participant and Functional
definition of the participants the environment f(x, y, z) functional
remapping of distance for the
perception of the environ- spatial dimensions between
ment and actions on it. Next, participant and the
the model source defines the environment
static and dynamic aspects of
the environment. Finally,
space and time have equal im-
portance; they are independent of each other, but dependent on IO. J.M. Rolfe. and K.J. Staples. 4 s . . Flighr Simularion, Cambridge
the first two levels of the confection model. Univ. Press. New York. 1988.
1 1 . Taylor. M. M.. Tactual Perception of Texture. in Handbook of
Conclusion Perception. Vol. 3: Biology of Perceptual Systems, E.C. Carterette
The technical confection model has interesting implications and M.P. Friedman. eds.. Academic Press. New York, 1973, pp. 251-
for networked V R systems. We feel that exploring these impli- 272.
cations will benefit from the context of a conceptual VR model. 12. W. Robinett. Synthetic Experience: A Proposed Taxonomy, Pres-
The model proposed here includes a human model to show the FIICL:
Teleoperators and Virtual Environments. Vol. 1, No. 2, Spring
experiential side of V R systems and a technical confection 1992, pp. 229-247.
model for the creation of a participatory environment. Together 13. D. Zcltzer. Autonomy. Interaction. and Presence. Presence; Tele-
these models define a conceptual V R system from a systems operotor\- and Virrunl ErivironnientA, Vol. 1. No. 1. Winter 1992.
perspective that includes the human participant. u pp.127-132.

References
1. K.R. Boff and J.E. Lincoln, eds.. Engineering Data Cornpcnrliunz:
Human Perception and Performance. Vols. 1.2. and 3, Human Eng.
Div.. Harry G. Armstrong Aerospace Medical Research Lab.,
Wright-Patterson AFB. Ohio, 1988.
2. K. Boff. J.P. Thomas. and L. Kaufman. eds.. Handbook of Percrp-
tion & Human Performance,Vols. I and 11. John Wiley. New York.
1986.
3. E.G. Boring, Sensation and Perception in [he History of Experi-
mental Psychology, Irvington. New York. 1977.
4. J.J. Gibson, The Senses Considered as Perceprriul Systenis. Houghton
Mifflin, Boston, 1966.
5. J.J. Gibson. The Ecological Approach to Visirul Perccprion.
Lawrence Erlbaum Assoc., Hillsdale. N.J.. 1986.
6. T.W. Malone, What Makes Computer Games Fun? B-vre,Vol. 6,
No. 12, Dec.1981 pp. 258-277.
7. M. Csikszentmihalyi. Flow: The Psychology of Optimal Experience.
Harper Perennial, New York, 1990.
8. C. Heeter, BattleTech Masters: Emergence of the First U S . Vir-
tual Subculture, Mulrimedia Review.Vol. 3, No. 4. Winter 1992. pp.
65-70.
9. S.R. Bussolari et al., The Use of Vestibular Models for Design And
Evaluation of Flight Simulator Motion, A I A A Paper 89-3274.pre-
sented at the AI A A Flight Simulation Technologies Conference.
1989.
John N. Latta is president of 4th Wave, where he
is responsible for several state-of-the-art multi-
media projects, including a business television sys-
tem. H e is also actively involved in the analysis,
design. and integration of projects that inclade
personal computers, multimedia, and virtual re-
ality. Latta has a BES from Brigham Young Uni-
versity and an MS and PhD from the University of
Kansas, all in electrical engineering.
David J. Obergis a principal engineer with Global
Output\, where he is developing teleoperations
and telescience capabilities for a commercial space
platform His professional interests include space
systems development, GPS-based attitude deter-
mination and rendezvous and docking, and space
station propulsion systems Oberg received BS
and MS degrees in aeronautics and astronautics
from the Massachusetts Institute of Technology in 1985 and 1986, re-
spectively.
Readers can contact Latta at 4th Wave. P O Box 6547, Alexandria.
VA 22306: e-mail jnl@fourthwave.com.

January 1994 29