www.jntuworld.

com

Virtual Reality Technology

Entering the image

Need for better competitiveness, time-to-market, product development

and personalization
Reduction in hardware cost

More compelling, expressive and comprehensible than just viewing

Various levels, from Desktop VR to immersive high-end

The Applications of VR

N,000,000 is now N,000 , or even in smartphones

Wider range of applications
3D CAD, CAM systems have become more commonplace

SGN-5406 Virtual Reality

Autumn 2009
ismo.rakkolainen@tut.fi

Improved VR technology

Still many problems

VR applications from prototypes to industrial phase

3D rendering speed revolution!
Utilization and maintenance require highly skilled personnel. Development
costs may be high
Often custom software, incomplete documentation, conversion problems

Intuitive: becoming a general-purpose UI?

Applications of VR

Applications of VR

VR Needs Other Sciences

Signal

processing

Optics

Computer science

Ergonomics

Measurement tech.

Electronics

History

Psychology

Physiology

Signal

Process tech.

Computer science

Robotics

Ergonomics

Pedagogy

Measurement tech.

Sociology

Electronics

Philosophy

History

Art

Psychology

Etc. etc.

Physiology

Applications of VR

The Applications of VR

Applicable to enormous range of things

Do not try to force VR to everything
Imagination sets the limits
VR technology is getting more advanced and lower cost
High demand, enthusiasm
Useful and/or entertaining
Some applications (in Finnish):
Excellent courses on 3D, VR etc at annual SIGGRAPH
conferences, http://www.siggraph.org/s2009/sessions/courses/
See EUs Intuition research network reports
http://www.intuition-eunetwork.org/

Acoustics

Process tech.

Robotics

Pedagogy

Sociology

Philosophy
Art

Etc. etc.

Applications of VR

http://www.csc.fi/downloadPublication?uid=e252b504caf89b681bc95bc13e93e4ca

processing

Optics

Acoustics

Other Sciences Need VR

Some Potential Advantages of VR

Applications of VR

Cost savings, profit

Improved marketing, presentation
Conveying information, artistic expression, etc.
Exploring, familiarizing with data in a new way
Entertainment, games, escapism
Non-invasive experimentation
What if -scenarios
Safety, saving lives
Improved quality of life
Training hard and impossible tasks
Applications of VR

www.jntuworld.com
1

www.jntuworld.com

Some Fields of Applications

Training, Education: Simulators

Simulators, training, education

Perhaps the most important currently

Product-, virtual planning

Production, manufacturing

Visualization, data exploration

Presentations, marketing
Mission planning
Entertainment, games
ILM, PIXAR, etc.

Tele-presence
Augmented reality
web3D

Applications of VR

Why Simulator?
The

Emergency services college

Army
Police
Finnair
Flight controller training
Truck drivers
Cranes

Many more applications to come

Applications of VR

Military Applications

costs

Material, time, money

10

The first field of VR applications

NASA and US army have developed a lot
Future Combat Systems: $170B project!

Perils,

harmfulness
Rare or impossible situations
Fast modifications

Simulators, education
Augmented reality
HUD, head-up display, onto a windscreen
HMD, head-mounted display

E.g., full load to empty load

Tele-presence, tele-surgery, etc.

Samsung robot watchman
Honeywell Micro Air Vehicle (see image)
U.S. drones in Afghanistan, Pakistan

Controllability
Repeatability,

All kinds of simulators

One of the first VR application fields
Many simulators also in Finland:

real-time walk-through of complex systems

action record

Battle planning, troop training, etc.

Space technology
Repairing Hubble Space Telescope in 1999. The
role of VR used in astronaut training was useful

Applications of VR

Military Simulators

Finnish air forces, Pirkkala, Finland: Hornet

Flight, helicopter
Ship, submarine
Tanks, vehicles
Infantry, commando
Game-based learning, simulators

Networked battle training

Applications of VR

11

Military: Parachute simulator

12

Students wear the VR glasses while suspended in a parachute

harness, and learn to control their movements through a series
of computer-simulated scenarios
The computer receives signals from the student as they pull on
the risers that control the parachute
Saves lives, money, gas for the jets

http://adlcommunity.net/
Huge simulator centers
100s of LAN-connected mixed simulators
Goal as 100,000 objects

Applications of VR

Applications of VR

www.jntuworld.com
2

www.jntuworld.com

13

DARWARS Ambush! simulator

Military Application: recruiting

A set of PC simulators for military missions

multiplayer
http://www.darwars.net/

Virtual Vietnam, Virtual Iraq (PTSD) (CGW 8/2008)

Public military simulator (game)

A realistic training and battle

simulator

http://www.americasarmy.com/

DARPA project
Restricted to U.S. Military
User-authorable: soldiers can create new scenarios and training
Tactical language tutors

Not immersive VR, but a game on

a PC monitor

support.tacticallanguage.com

Applications of VR

Civil Simulators

15

The best are identical to the real thing

Often big hydraulic systems are used

United Airlines Flight Training Center

Finnair

Denver, CO, USA

The worlds biggest simulation center
Try it yourself! Boeing 777: $1350 / h

Applications of VR

Car simulators

17

18

Driving Simulators

Mobile truck training simulator

http://www.ek-aktiva.fi/
rekkasimulaattori-www/

Realistic rally simulator

Bus, truck training simulators
http://www.simrac.com/

Professional vehicles

Microsoft Flight Simulator < $50

Applications of VR

Car manufacturers
Driving schools
Drinking and driving in Haaga neurological institute

Many categories of simulators

DC-9 (8 KB (!) main memory, 112 KB full)

MD-11 etc.

16

Flight Simulators

Flight simulators

Other Simulators

Made by U.S. Army!

Made for positive publicity,
recruiting
Applications of VR

Since 1920s
Computerized in 1960s

14

Harvesters
Skylifts
Cranes
Trucks
Trains, ships

Virtual control desks for nuclear power plant

simulation, IEEE CGA 8/2008
Mock up = the physical copy of real cockpit
Applications of VR

Applications of VR

www.jntuworld.com
3

www.jntuworld.com

Military simus: Insta Group Oy

19

Walking Simulators

Treadport
OmniDirectional Treadmill (ODT)

Applications of VR

Some Fields of Applications

Applications of VR

21

Oil and Gas exploration

Espionage, military

Scientific Visualization

Exploratory holes 10M each, of which 2/3 dry

Seismic data is 3D
Visualization may raise success rate to 80%

imagery
video
3D terrain
geospatial data
simulators
mission planning
battlefield visualization
defense imaging
intelligence

Biology
Molecular structures
Astronomy

Applications of VR

23

Research Tool

Geological seismic interpretation

Physics
Drug design
Economy
Research
Education

24

Using various flavours of VR to gain knowledge of

humans
University of Tampere: human balance
Human cognition, perception
Behaviour, attitudes, etc.

Hayden Planetarium, NYC, www.amnh.org

22

3D objects
Complex phenomena
Can be more clear, expressive
and comprehensible
Touching the image
Immersion
Processes
Simulations
Flows etc.

Applications of VR

Applications of Scientific Visualiz.

20

Science, other phenomena

Applications of VR

Physics, flow dynamics

Astronomical macrostructures
Numerical tables to visualizations
Economics
Etc.

Applications of VR

www.jntuworld.com
4

www.jntuworld.com

25

Medicine

Psych. treatment: pain reduction

26

Medicine Meets VR-conference (MMVR)

Virtual Human project
Augmented reality
Surgery
Examining the fetus

Winning the fears

Fear of high places

Fear of flying
Fear of public speaking
Military trauma care
E.g., http://www.vrphobia.com/

Drug rehabilitation
VR pain control:
http://www.hitl.washington.edu/research/burn/
Applications of VR

Applications of VR

27

Medicine

Surgeon simulator case

Visualizations
Telemedicine, military medicine

Image analysis, which involves mostly automatic tissue segmentation and vessel
extraction
Segmentation refinement, where defects can be checked for, uncovered, and
corrected
Treatment planning, in which a surgical plan can be detailed based on data
inspection, spatial analysis, and resection simulation

Development of new drugs

Endoscopic examination simulators
Virtual surgery and visualisation

Small defects can be corrected in LiverPlanner through the use of direct

deformation tools, while template shape tools help users correct larger
defects
LiverPlanner employs a camera-equipped optical tracking system, a Tablet
PC, a custom-designed hybrid 2D-3D input device dubbed the Eye of Ra,
and a stereoscopic large-screen projection system
Liver Surgery Planning Using Virtual Reality
IEEE Computer Graphics and Applications (12/06)

IEEE CG&A Nov/Dec 2006

http://www.brainlab.com

Applications of VR

Applications of VR

29

Telepresence

LiverPlanner: liver surgery planning

Combines VR and high-level image analysis algorithms
Effectiveness and ease of use
Three main stages:

daVinci Surgical System

28

In VR the user is immersed in a computer generated world

In telepresence the user is immersed in a remote real world
Real and virtual data can be mixed
Robots are frequently used as part of a tele-operated system
Tele-operation in dangerous or impossible environments

Nano-, Macropresence

30

Nanopresence, macropresence
The scale is micro/macroscopic
Also timescale can be adjusted
www.romereborn.virginia.edu

Live connection possible

Chernobyl tele-operation with VR and robots

U.S. drones, UAVs in Pakistan, teleoperated from Las Vegas

Tight spots, physically demanding jobs, boring jobs

Unmanned vehicles are lighter, smaller
Battlefield surgery, telemedicine
Communication, tele-conferencing

Applications of VR

Applications of VR

www.jntuworld.com
5

www.jntuworld.com

31

Telepresence

CAD & Design

Dangerous

tasks
Impossible places
Multiuser systems

Traditional miniature models, prototypes

32

The material cost

The cost of making, the need for space
Need for miniature models reduced
Full scale clay car model: $1M
Fast alteration of plans and design
Small items can even be printed as 3D physical objects

Big, complex devices

Boeing 777
Submarines

Engineering
Automotive styling etc.
Most car manufacturers employ

Applications of VR

Applications of VR

33

Virtual Design

Entering the image, moving in it

Virtual models

Architecture

34

Buildings
Traffic networks
Virtual prototypes
Urban and rural environments
The search of potential problems
Visualization of plans
Marketing, presentation

Fast prototyping
Fast design
Flexibility
Alternations
Time-to-market
Life-like
Expressivity
Remote viewing

Assembly validation
Ergonomic analysis
Plant design, optimization
Training, marketing
Applications of VR

Applications of VR

35

Art
Illusion,

Animations, manuals
Collaboration over the net

experience
new tool and art form
Museums, installations
Reconstructions
Salvador Dali would have been delighted!

Virtual Orchestra

36

A training tool for the conductor

Applications of VR

Applications of VR

www.jntuworld.com
6

www.jntuworld.com

37

Simulations

Cool visualizations
Boosts marketing
Supports sales
More expressive, appealing
Clarifies complex systems
Small money for big companies

Assembly simul.

models

Logistics simulation, bottlenecks

Traffic

simulations

Finding of bottlenecks in rush

Prototyping layouts

Construction projects
Machines, devices

Security

Applications of VR

Applications of VR

39

Other Visualizations

Traffic, 3D city maps

Air traffic control
Virtual planetariums
Las Vegas casino, restaurant decorations
Historical reconstructions
3D Shoe sizer
Legal

Virtual Stage

Used in TV in daily basis

Weather reports

Combining several image sources

Chroma-key
Tracking of cameras
Virtual worlds + reality
Mixed reality

Applications of VR

Applications of VR

41

3D Movies

3D movies with colored glasses were popular in the 1950s

A new coming of 3D movies recently
3D versions of movies give better profit than standard movies,
but require investment
Thousands of movie theaters are being equipped
Stereoscopic filming, or 2D -> 3D transformation software
Avatar (Dec09): $237 million for production

40

Accidents, crimes
Crime scene 3D scanning

38

Ergonomy
Factory

Presentations

42

3D Movies
Hollywood plans to release
60 3D movies in 18 months
3D theater screens: ~6500
NA 4000, Eur 1200, As 1300

3D digital cinema is here, thousands of 3D theaters in USA

RealD: market leader (3300), XPAND (1100), Dolby (1000),
IMAX (300), MasterImage (300), Sony 4K (300)
Panasonic 3D HD camera, E&S ESLP 8K laser 3D projector

A host of consumer-oriented 3D products are likely in the next

few years. Web 3.0?
3D: the next big wave in consumer electronics?
Hard to pirate a 3D movie with a smuggled in-theater camera
Applications of VR

Applications of VR

www.jntuworld.com
7

www.jntuworld.com

43

Movies, TV

Not really VR, but 3D stereoscopic

TV is broadcast, but could become more interactive

Border between TV, PC and web may become unclear

Making 3D Movies
A major transition in Hollywood
The mistakes that can happen in 3D movie making
Left/right eye reversal, unintentional monoscopic frames, out of sequence
stereo frames, action within a limited volume, focus depth changes, etc.

3D content is necessary to enable the widespread deployment

and commercial success of 3D displays
Many standard movies Its hard & expensive to make a movie,
never mind 3D

All the normal tasks plus many 3D issues intertwined

Old movies can be converted to 3D

Semi-automatic, can sometimes be fair quality (or bad)
E.g., Philips, DDD, In-Three, Conversion Works, Mercury 3D,
JVC, make3D, imcube Media,
Quality conversion is a massive post-production task
For conversion of a 90-minute film, the post-production schedule
will cover about 10 months
Some new movies are already filmed as 3D just in case

The screen size has an effect

Recent examples: Beowulf, Chicken Little, Star Wars, U2 3D,
Nightmare Before Christmas, Polar Express, Avatar
The 3D industry needs to train a lot more people in the art and
science of 3D movie making
Better 3D workflow tools, particularly in 3D editing, are needed
(being developed in-house in major studios)
DIY: StereoMovie Maker, http://stereo.jpn.org/eng/stvmkr
http://nowin3d.com/, http://www.digdia.com/
A list of 3D movies, 3D ready HDTVs etc.: www.3dmovielist.com/

Applications of VR

Applications of VR

45

3DTV is coming

Several ideas, prototypes, visions

Blu-ray Disc Association: 3D spec in Dec09
Already here: Colorcode3d glasses in Super bowl 2009
Is it technically, commercially feasible (in recession)?
A need, a market for real people (not engineers)?
Commercial issues will dictate deployment
How about usability (for couch potatoes)?
3DTV emerging (glasses or autostereoscopic)
3D movies may help, but many obstacles
Few standards, infrastructure not ready
EUs 3DTV project, http://www.3dtv-research.org/
3D Consortium, http://www.3dc.gr.jp/english/
3D @ Home Consortium, www.3dathome.org

ddd.com: 3D Conversion in a Set Top Box

TDVision, http://www.tdvision.com/

Sensio Technologies Inc., www.sensio.tv

Develops full HD stereoscopic 3D real-time encoder

2D->3D real-time decoder ASIC chip

to be integrated into Hyundai's LCD-TVs and SpectronIQ "Full 3D" TV

47

Entertainment, Games

Game consoles at home!

iZ3D 22 3D gaming monitor for $999
Resident Evil 5 is designed to exploit S3D
Walls into displays

48

LBE, location-based entertainment

Namco Bandai Games
VR entertainment centers
DisneyQuest VR park, Orlando
Futuroscope (France)
Universal Studios, Disneyland

New kinds of interaction, tracking?

What is the most violent place of the world?

Our living rooms in front of TV sets!

VR has a lot to give for violence etc.

InThree: semiautomatic 2D->3D conversion, in-three.com/

YouTube is working on a 3D stereoscopic video player
Sky UK launches Europe's 1st 3DTV in 2010 (&Channel 4 UK?)
Sonic Solutions & Nvidia add 3D content to Roxio / CinemaNow
Internet movie service for 3D ready TVs and PC displays
Texas Instruments: 3D DLP technology for existing HDTVs
Sony becoming an active player (3D Bravia TV, Playstation etc)
http://www.informationdisplay.org/ July 2009: 3DTV special
Applications of VR
General info: http://www.stereoscopy.com/

Entertainment, Games

The office of the future, MIT KidsRoom

Home VR theatres?

46

3DTV

Real-time, special LCD TV, polarization glasses

Applications of VR

44

Youll know when S3D is a success when you dont talk about
it. When was the last time you went to a movie and came out
saying, WOW did you see the color in that movie?

Applications of VR

Many kinds of VR experiences

Virtual actors
The virtual actor is created in real time based on tracking of
a real face
Motion capture can be used to create more realistic games

Applications of VR

www.jntuworld.com
8

www.jntuworld.com

49

Virtual golf cafs in Korea

CyberSpace Mountain
Design your ultimate roller coaster
ride it on a 360 pitch / roll simulator

12,000 simulators, 300,000 players

Always nice weather indoors
Mat tilts downward to replicate a downhill slope
In Japan and Korea, golf is very expensive, but VR golf is cheap
Plenty of international golf courses to choose from
Cameras and sensors capture the club speed, the ball's launch
angle, and its spin. The computer creates its trajectory, audio
and visual effects
http://www.businessweek.com/globalbiz/content/oct2008/gb200
81027_293085.htm

Aladdin's Magic Carpet Ride

Applications of VR

Applications of VR

51

DisneyQuest

Virtual Jungle Cruise

50

LBE: DisneyQuest

DisneyQuest

52

Invasion

Applications of VR

Applications of VR

53

Augmented Reality (AR)

Augmented reality alters how we see the real world

Numerous flavors and applications

54

AR Visions

(in Finnish: listty todellisuus)

Applications of VR

Applications of VR

www.jntuworld.com
9

www.jntuworld.com

55

Mobile AR / VR

An emerging field: devices get smaller, display gets bigger,

processing faster
3D user interfaces?
3G phones with multi-sensors
iPhone and other smartphones
Location-based visualization

Google Maps 3D for Symbian

VR Desktop

GIS-based applications
Numerous consumer personal viewers emerging

56

Take head-mounted display

Add tracking =>
Virtual desktop, personal data space (mobile AR, iPhone)
Extends display
Immersion
Interaction

E.g., Icuiti iWear, MicroOptical myvu, i-O Displays iTheater,

Oriscape Cyberman GVD310A

The disappearing computer, ambient intelligence

"The-computer-as-we-know-it" will have no role in our future
everyday lives
http://www.arconsortium.org/
Mobilizy.com: Augmented Reality Mark-up Language (ARML)
Applications of VR

VR Desktop

Applications of VR

57

MobiVR

Microdisplay + finger pointing

Enables easy input on a hand-held device
Enables also VR apps on a mobile phone
http://www.cs.tut.fi/~ira/MobiVR.html

Tactile

Web3D Applications

58

The same application possibilities as with most of other VR

A growing branch of VR, not yet immersive
Typically displayed on a monitor

A new UI, aid for communication

Everyman VR
Turbulent technologies, browsers
Mobile phones becoming 3D!
Ads, product presentations
3D comics, games
Community-based services

Compresses well!

feedback ?

Sony PS3 Home, Project Wonderland, Active worlds, muse, worlds.com,

moove, Lively (Google, ended)

Applications of VR

Some Haptic Applications

Applications of VR

59

Medicine: surgical simulators for medical training; manipulating

micro and macro robots for minimally invasive surgery; remote
diagnosis for telemedicine; aids for the disabled such as haptic
interfaces for the blind
Entertainment: video games and simulators that enable the user
to feel and manipulate virtual solids, fluids, tools, and avatars
Education: giving students the feel of phenomena at nano,
macro, or astronomical scales; what if -scenarios for nonterrestrial physics; experiencing complex data sets
Industry: integration of haptics into CAD systems such that a
designer can freely manipulate the mechanical components of an
assembly in an immersive environment
Graphic Arts: virtual art exhibits, concerts, and museums in which
the user can login remotely to play the musical instruments, and
to touch and feel the haptic attributes of the displays; individual or
co-operative virtual sculpturing across the internet

Biggest Gaps for VR Technology

60

The investment (purchase, integrate, install, maintain)

Large corporations, space, military
Prototype and pilot nature expert personnel required
But smartphones, game consoles may make consumer products

CAD VR integration

Content: tools are more for engineers, not for designers

Lack of physical contact with 3D virtual objects

Limited interaction tools, modalities and user interfaces

Multi-user visualization systems

Precise tracking, expensive trackers
Collision detection handling of very large models
Simulation of complex physical phenomena like thermal diffusion or
electromagnetic interferences
Course recap in 60 seconds: Dont be a WIMP

Separate systems: incompatible. No direct support for VR

Often only 2D is used in design. 3D too advanced for SMEs

Only low-level, limited or specialized haptic devices

Interaction paradigm for shape and part design

Applications of VR

Applications of VR

www.jntuworld.com
10