Unit-4 Virtual Reality

1. Applications of multimedia 2. Intelligent multimedia system 3. Desktop virtual reality 4. VR operating system 5. Virtual environment displays and orientation making 6. Visually coupled system requirements 7. Intelligent VR software systems. 8. Applications of environment in various fields.

SANJAY SINGH (LECTURER)

Applications of multimedia
Multimedia finds its application in various areas including, but not limited to, advertisements, art, education, entertainment, engineering, medicine, mathematics, business, scientific research and spatial temporal applications, see Banerji & Ghosh (2010). Several examples are as follows: Creative industries Creative industries use multimedia for a variety of purposes ranging from fine arts, to entertainment, to commercial art, to journalism, to media and software services provided for any of the industries listed below. An individual multimedia designer may cover the spectrum throughout their career. Request for their skills range from technical, to analytical, to creative. Commercial Much of the electronic old and new media used by commercial artists is multimedia. Exciting presentations are used to grab and keep attention in advertising. Business to business and interoffice communications are often developed by creative services firms for advanced multimedia presentations beyond simple slide shows to sell ideas or liven-up training. Commercial multimedia developers may be hired to design for governmental services and nonprofit services applications as money. Entertainment and fine arts In addition, multimedia is heavily used in the entertainment industry, especially to develop special effects in movies and animations. Multimedia games are a popular pastime and are software programs available either as CD-ROMs or online. Some video games also use multimedia features. Multimedia applications that allow users to actively participate instead of just sitting by as passive recipients of information are called Interactive Multimedia. In the Arts there are multimedia artists, whose minds are able to blend techniques using different media that in some way incorporates interaction with the viewer. One of the most relevant could be Peter Greenaway who is melding Cinema with Opera and all sorts of digital media. Another approach entails the creation of multimedia that can be displayed in a traditional fine arts arena, such as an art gallery. Although multimedia display material may be volatile, the survivability of the content is as strong as any traditional media. Digital recording material may be just as durable and infinitely reproducible with perfect copies every time. Education In Education, multimedia is used to produce computer-based training courses (popularly called CBTs) and reference books like encyclopedia and almanacs. A CBT lets the user go through a series of presentations, text about a particular topic, and associated illustrations in various information formats. Edutainment is an informal term used to describe combining education with entertainment, especially multimedia entertainment. Learning theory in the past decade has expanded dramatically because of the introduction of multimedia. Several lines of research have evolved (e.g. Cognitive load, Multimedia learning, and the list goes on). The possibilities for learning and instruction are nearly endless. The idea of media convergence is also becoming a major factor in education, particularly higher education. Defined as separate technologies such as voice (and telephony features), data (and productivity applications) and video that now share resources and interact with each other, synergistically creating new efficiencies, media convergence is rapidly changing the curriculum in universities all over the world. Likewise, it is changing the availability, or lack thereof, of jobs requiring this savvy technological skill. Newspaper companies all over are also trying to embrace the new phenomenon by implementing its practices in their work. While some have been slow to come around, other major newspapers like The

SANJAY SINGH (LECTURER)

New York Times, USA Today and The Washington Post are setting the precedent for the positioning of the newspaper industry in a globalized world. Engineering Software engineers may use multimedia in Computer Simulations for anything from entertainment to training such as military or industrial training. Multimedia for software interfaces are often done as collaboration between creative professionals and software engineers. Industry In the Industrial sector, multimedia is used as a way to help present information to shareholders, superiors and coworkers. Multimedia is also helpful for providing employee training, advertising and selling products all over the world via virtually unlimited web-based technology Mathematical and scientific research In mathematical and scientific research, multimedia is mainly used for modeling and simulation. For example, a scientist can look at molecular of a particular substance and manipulate it to arrive at a new substance. Representative research can be found in journals such as the Journal of Multimedia. Medicine In Medicine, doctors can get trained by looking at a virtual surgery or they can simulate how the human body is affected by diseases spread by viruses and bacteria and then develop techniques to prevent it. Document Imaging Document Imaging is a technique that takes hard copy of an image/document and converts it into a digital format

VIRTUAL REALITY
Virtual reality is a computer generated environment that makes the user think that he/she is in the real world environment. For example Close your eyes and imagine that you are in the beach. You listen to the ocean waves. The shadow of the people walking projected over you. You can hear the laughs of the people while you also feel the sunlight over your skin. At the end you see the sun diving in to the ocean. You even built a sand castle. This is an idea of virtual reality: Making you feel like being there while you are here. Also called as Artificial reality, virtual world, virtual space, cyber space. The virtual world is hosted on a computer in the form of a databse.The database resided in the memory of the computer. The database generally consists of points in spaces as well as texture (images). Vertices may be connected to form planes, commonly referred to as polygon. Each polygon consists of at least three vertices. The polygon could have a specific color and the color could be shaded of the polygon could have a texture pasted on it. Virtual objects consist of polygons. Virtual object will have, a positions (X, Y, Z) and orientation (YAW, PITCH, ROLL) and attributes like gravity or elasticity.

SANJAY SINGH (LECTURER)

Virtual world is rendered with a computer rendering involves: the process of calculation the scene that must be displayed for a virtual corner view (on a flat plane), from a specific point, at a specific orientation & a specific field of view (FOV). In past: CPU of the computer was mainly used for rendering ( s/w Rendering). Later: CPU that render the virtual world to display screen (h/w rendering). These CPU are normally situated on graphics accelerated cards, but may also be situated on the motherboard of the computer. Hardware rendering much faster than the software rendering. For humans, VR is a means to visualize, manipulate & interact with computer & extremely complex data. Technically, it is a computer based simulation of environment with 3D-images requiring the use of periphery equipment such as Data gloves & Head Mounted Displays to allow the user interaction. It can be graphic based or text based .The real benefit of VR is the ability to touch, animate , pickup & reposition virtual objects & create totally new configurations. This virtual environment is created with the computer h/w or s/w presented to user in such a way that it appears & feel like a real environment. To enter the virtual environment user makes a special use of h/w or s/w. these devices receives their input from the computer systems. The h/w & s/w used for this environment are very expensive. Some of the standard input devices such as keyboard & mouse, multimodal devices such as wired glove. Boom arm is utilized by the user to interact with the virtual environment. The simulated environment can be simulated to a real world. For example simulation for pilot or VR games.

3 IS of VR
INTERACTIVITY IMMERSIVE IMAGINATIVE VR sometimes referred to as Synthetic VR: Desktop VR Immersive VR

Desktop VR
A computer screen is normally used as the display medium. The user views the virtual Environment on the computer screen. In order to experience the user must look at the computer screen.

SANJAY SINGH (LECTURER)

Immersive VR The user is immersed in or surrounded by the virtual environment. Immersive VR is examined by with multi-display system; the field of view (FOV) is extended by using several monitors or projectors. When using projectors the image may be front projected or backward projected onto the viewing screen. May simulation utilize three screen (front, left, right) to provided an extended field of view. The user where the configuration is surrounded by projection screens is refer to as CAVE environment. The image may also be projected on a DOME that may vary in shape & size. With multi-display system, the user may look around as if in real world. HMD (Head Mounted Display): It consist of two maintenance displays that are mounted in front of the users eyes. With the head means special optics enables the user to view the miniature screen. HMD are also contains 2 head phone, so that user may experience the virtual environment actually. HMD is normally fitted with a head tracked. The position (X, Y, Z) & orientation (Yaw, Pitch, Roll) of the tracker. As the user looks around the position & orientation information is continuous relayed to the nose environment. The computer calculates the approximate view that the user see in the virtual environment, and this is displayed on the miniature displays. For example lets assume that virtual environment is the inside of car and that the user is sitting behind the steering wheel. If the user looks find, the head tracker will measure this orientation, and relay it to the computer. The computer would then calculate the forward view, & the user will see the windscreen, wipers and bonnet of car (the user will obviously see the outside world or out of window view). If the user looks down the computer will present a view of the steering wheel. If the user further down, the acceleration pedal, clutch and brake pedal will be shown. The orientation information may also used to experience stereo and 3-D sound. If the user looks straight (he/she) will hear the engine noise of the car. The volume & phase will be equal for the right and left ear. If the user looks to the left, the sound of e engine noise will be higher in the right ear & lower in the left ear. Trackers that only track the orientation are only referred to as 3 degree of freedom or 3 DOF while the trackers that track the position are referred to as 6 DOF. DATA GLOVE: Objective in the virtual world may be manipulated by means of data glove. A data glove measures the bend of the users fingers. The user may grab a virtual object & put it at a different object. The user may also throw a object. The position and orientation of the user hands is measured with 6 DOF trackers. It is a force feedback glove, the user will also be able to deform the virtual object & feel the object. In order to navigate in the virtual world, a space controller is used. The space controller could be a normal joystick or a computer mouse. For example: when the mouse is move find; the user moves find in the direction in the virtual world/object, when it is moved to the left, the user moves to the left etc. Force feedback joystick or mice could provide hap tick uses

SANJAY SINGH (LECTURER)

to the user.Eg. When the user moves to the virtual world, normal joystick & computer mice are usually used in desktop VR system. In immersive VR system we normally use baseless joystick as space controllers. This enables the user to leave the desktop & to import with the virtual world while standing up. It is also possible for different users to share the same virtual world. This is normally achieved by connection the host computers to computer network. Each user host computer broadcast the position and orientation of the user in the virtual world. The users may therefore see each other in the virtual world.

DESKTOP VIRTUAL REALITY

The term VR is used by many difficult people & currently has many meanings. There are some people to whom VR is a specific collection of technologies that is HMD, along GIP devices & audio. Virtual reality is a map for humans to visualize, manipulate & interact with visualization extremely complex data. The visualization part refers to the computer generating visual, auditory or other sensual i/ps. The images are graphical renderings of a world within the computer. This world may be a CAD model, a scientific simulation or a view into a database. The user can interact with the world & directory manipulates objects within the world. Some worlds are animated by other processes. Types of VR-System (1) Window On World System (WOW):- One of the most common and accessible forms of VR is desktop VR or WOW VR systems. These systems do not rely on any specialist i/p or o/p devices in order to use them. Typically a normal computer mouse & monitor is sufficient. This works on the approach one must look at a display screen as a window through which one beholds a virtual world. The challenge is to make the picture in the window look real, sound real & the objects are real. (2) Video Mapping:- In this system the user monitors with the help of videos cameras & thus form an interactive environment. The computer identifies the user body & overlays it upon a computer generated scene. Thus user can match a monitor which shows the combined image. By gesturing & moving around in front of the camera the user can interact with the virtual environment. (3) NON-IMMERSIVE VR SYSTEM:- VR system can be divided into 3 groups:(a) Immersive (b) Non-Immersive (c) Hybrid Immersive system, Replace overview of the real world with computers generated images that react to the position & orientation of the users head.

SANJAY SINGH (LECTURER)

A Non-Immersive VR System: leaves the user visually aware of the real world but able to observe the virtual world through same display device such as graphics workstations,, the user navigates the virtual environment using a device such as space mouse. This device enables the user to translate & rotate the view point using ^ DOF. (4) Hybrid VR System:- They permits the user to view the real world with virtual images superimposed over this view- such system are also known as augmented reality system. A practical example is founding the HMDs used by the fighter pilots which allow the pilot to view their outside world simultaneously with over layer synthetic graphics.

APPLICATION OF VR:(1) Medical (2) Architectural (3) Entertainment (4) Education (5) Manufacturing (6) Science (7) Military VR will be used to model and explore familiar environments associated with kitchen, plane, offices, studios, submarines, cars and hospitals. It will also be used to explore unfamiliar environments found in molecules, atoms, galaxies, viruses, bacteria and crystals. However, we also employ spatial models to understand abstract problems found in mathematics, economics, simulation, artificial life and networks.

VR-HARDWARE (Visually Coupled System Requirement)

When interacting with the virtual environment various sensors and transducers are needed to monitor a users actions & feedback signals that reflect the status of something within the environment. For instance an interactive glove may be tracked in three dimensions; finger positions may be monitored and the glove may incorporate pressure pads that inflate when a virtual touch condition is detected. To monitor a gloves position, a sensor is needed to produce the X, Y and Z coordinates relative to some physical frame of reference, which is complemented by orientation angles, such as yaw, pitch and roll. Finally the pressure pads must give a tactile stimulation when activated. Designing a sensors to capture this data is not a fundamentally a difficult problem. What is difficult is making their devices reliable, repeatable, accurate, portable, fast, safe, non-invasion, robust and low cost.

SANJAY SINGH (LECTURER)

TRACKING SENSORS
A central requirement of any immersive VR system is the accurate tracking of the users head in terms of its position and orientation. Various methods have been employed for head tracking including: electronics, low frequency, electromagnetic fields, optical image, processing and the earths magnetic field. The users hand is also tracked using identical sensors technology. In the case of a glove , the sensors is attached to the back of the glove, while a 3D- mouse has the sensor encapsulated within the device. RANGE: Range is a measure of the active tracking distance for determining translation and orientation within the operational specification. Both range and linearity of A.C electromagnetic system are influenced by conductive metals such as stainless steel and aluminum in the viscosity of the tracker. This is due to the eddy current that is induced by the alternating electromagnetic fields, which is turn setup new field pattern that distort the original field. Ferrous metallic structures in the fabrication of floors, walls and ceiling are also a problem. An alternating approach to employ D.C magnetic fields which do not cause these continuous eddy currents. Trackers work reliably up to a radius of 2-3 m in a perfect environment, but his range is reduced by local electromagnetic noise and metallic objects.

ADL-1
The adl-1 from shooting star technology is a 6 DOF tracking system that employs mechanical coupling. The user means an adjustable headband which is connected to a fixed mounting via two jointed light height arms. Two potentiometers are used to measure joint angles in their arms which provide data to specify the position & orientation of the users head. Logitech Head Tracker: They are manufactured by Logitech, including measures absolute position and orientation using ultrasonic speakers and microphone. Three speakers are mounted in large fixed triangle, while 3-microphone are housed in a small mobile triangle is normally attached the users head. The technique of triangulation enables the position of two mobile triangular to be specified relative to the fixed triangle. Because ultrasonic sound is used, the sound must be within the line of height of the speakers. Space Ball-2003: it is manufactured by space ball technology. Including 8 is an i/p device for measuring simultaneous translations and relations about the X, Y and Z axes. When the ball is gripped by the user, the forces & torques are measured and I/p to a host computer. The applied force approx range- 0.5-20 N Torque- 15 600 N The ball radius is 60 mm

SANJAY SINGH (LECTURER)

Bio Muse Bio control system, including manufacture the Bio muse. A bio electric signal controller. This accepts i/p of brain electrical activity, music signals and eye movement, captured through a headband and small sensors. The o/p channel is used to activate events in a computer system. The muse received data from 4 main sources of electrical activity in the human body: Eye motion activity (ECG Signals) Heart (EKG Signals) Brainwaves (EEG Signals) The vertical and horizontal eye mounts are used by the Bio muse to create an eye controller joystick & it can be configured to measured eye convergence. FORCE FEEDBACK SENSORS A simple touch can be generated by low frequency electromagnetic transducers which are small and lightweight. EXOS have pioneered a range of products for sensing real time hand gestures and the feedback of force information from virtual environment. Where large torques are required careful design is required to minimize weight of the adulators and not interfere, with the natural movements of the user. Thus, the system must be light & freely back drivable, by either active or passive means. Touch-Master It is a non-reactive, tactile feedback device which provides a sense of contact when activated. The sensors consist of voice coil transducers driven at approximately 200Hz and they are attached to the user thumb & first finger. When interfaced via control unit to a host computer, an event derived by a virtual collision can activate the sensor creating a tactile sensation. SAFIRE: It measures the torques developed at users wrists, thumbs, index & middle fingers, which can then be used to control some virtual process. The thumb and index fingers is attached to 3 DOF & the middle finger with 2 DOF. It can be used in variety of simulation exercise where forces exerted by the user play an important part of manipulation. Force Arm Master: to engage the arm & shoulder & capable of possibly monitoring or active driving, the force arm, elbow & shoulder. Force Feed Back Master: In application such as surgical simulation or molecular modeling, there is a requirement of track & feedback forces relating to the orientation of handle. It works under angular torque of 140 degree of yow, 140 degree pitch & 240 of roll. Power Stick: it is a joystick to sense & create forces inn pitch & roll. The continuous force is approx 0.45 Kg in all direction & peak force is 2.36 Kg.

SANJAY SINGH (LECTURER)

GLOVES
They allow the participant to communicate with the virtual world with hand gestures. Fiber optics & LEDs are used to detect the bending of the finger while the position & orientation are detected by magnetic tracking device incorporated in the glove. Gloves can be used entertainment, medical training.

TREADMILS: Treadmills are a natural navigation device, device the mount of walking is a spontaneous action, Treadmills provide the optimal navigation system for architectural environments . HMDs (Head Mounted Displays):
An HMD indicates the user from the real world and substitute binocular views of the VR. One approach to HMD design includes two small flat panel liquid crystal displays, in front of which are mounted optics to provide a comfortable focal point. When they are positioned such that their optical centers match the inter ocular distance of the users eyes, the nearer should experience a stereoscopic scene. In the real world our eyes are constantly converging & focusing as our attention is drawn from one point of interest to another. HMDs have been developed for military pilots for integrating navigational & tactical data with out of the cockpit scenes & for the commercial market, there exist a wide range of HMDs

Military HMDs: The technical specification for military HMDs is a primary consideration
followed by the secondary one of the cost. A military pilot is unable to hear any headgear that could came high gravity force. Consequently any display system must have a minimal impact upon the weight of to pilots helmet. At the same time pilot requires high resolution information to be displayed upon an optically accurate & rugged display. Designers have explored the use of cockpit mounted display to the pilots helmet through flexible coherent fiber optics bundles. The displays are full of color with a horizontal resolution of at least 1000 lines. Computers generated image are then superimposed over the pilots view the real world using off axis optics. A typical specification provides a field of view of 60 degree horizontal, by 55 degree vertical with about 45 degree binocular overlap. The specification also addresses features such as minimum aberrations, peripheral vision, eye relief, optical transmission. Scene collimation, helmet chip page & optical mapping function. They are expensive & cannot be considered for non-military application.

General purpose HMD: It is a lightweight mono scope or stereoscopic, HMD that can fit to the
users head without interfering the normal vision. When gazing upwards into the HMD, the user sees a collimated view of the displays images. For including degree of immersion, the display is lowered in the normal gaze direction of the user. The displays are a dual matrix LCD with a resolution of 420 X 230 pixels per eye, with a FOV of approx 30 degree. It also incorporates stereo audio channels & optical systems.

SANJAY SINGH (LECTURER)

10

POSITION: Navigation, 3D-motion, tracking and positioning in real time within the world is
provided by the tracking devices. Used wit HMDs, displays, Gloves & X, Y, & Z position can be repeated back & translated to the internal environment to the virtual environment to provide movement direction.

TOUCH: Hap tic response or resistive feedback adds the additional sense of touch & feel to the
internal world, allowing the user to explore the physical properties.

SOUND: The ability to have the speech and sound. Positional within the virtual world enhances the
level of intelligibility of correctly designed & implemented audio communication systems.

Teleconferencing enables people to work in productive collaboration. NAVIGATION: The ability to navigate in six degree of freedom X, Y, Z, yaw, pitch, Roll is an
integral part of moving and viewing virtual world. Devices like space board and space mouse are used for this purpose.

GRASP: The ability to provide real time 3D interaction, which allows the users to reach in and
interact with graphically modeled objects within the virtual world. Conjunction of tracking devices, translation & rotation provide the user with complete grasp of the virtual world.

GRAPHIC CARDS: Graphic cards play an essential role in programming and flexibility of the VR
world.

MANIPULATION AND CONTROL DEVICES: One key element for interaction with a
virtual world is a means of tracking the position of real world objects such as head or hand.

MOUSE: The traditional mouse can be an affective interaction device for very simple VR
Application.

Virtual Environment Displays & Orientation Making: Visually Display System:

HMD BOOM CAVE 3D-GLASSES HAPTIC SYSTEM

SANJAY SINGH (LECTURER)

11

 AUDIO SYSTEM

BOOM Binocular Omni Orientation Monitor

Fake space including supply 3 BOOM devices: BOOM (is a monochrome device) BOOM-2C (uses a color space based on Red & Green) BOOM-3C ( Is a full color device) All three display devices are stereo using CRT technology. There are six joints in the supporting counter balanced arm, enabling the user to move the BOOM within a sphere of approximately 2m radius. The arm joints contain sensors that enable the position & orientation of the BOOM to be computed. There are 2 buttons mounted on the ends of the grip handles which can be programmed to perform any task in a VR application. A type of stereoscopic viewing device that has became popular alternate to HMDs. This device treats users head motion in 6 directions. This means that user can look up, down, left, & right as well as his/her head & the computer will display the appropriate picture.

CAVE
Cave is Automatic virtual environment. A cave enables one or more persons to experience the sensation of being completely surrounded by high resolution, 3d videos and audio. It is a room formed from three rear projections screen for walls and a down projection screen for the floor. High resolution video projector displays computer generated, stereo images at a field rate of 12o Hz, while computer controlled amplifies relay sampled sound through a n/w of speakers. Inside a CAVE user wears a pair of stereo shelters glasses to select left & right images. A silicon graphics onyx with 3 virtual realities. Engine is the source of the images, and the output determined by tracking equipment work on the users head and hand. The room is large enough to permit other viewers to share the same experience. CAVE has prone to be very useful in exploring visualization of precompiled datasets; a user can do very thing that can be achieved with a workstation, but with the obvious advantages of full immersion. When extra processing power is available for integrating simulation s/w the user can interactively explore new scenarios, and expressive a real visual response. Application of CAVE Interactive molecular modeling Scientific visualization Sound simulation

SANJAY SINGH (LECTURER)

12

 Architecture Weather simulation Medical modeling

3D- GLASSES
There are the popular devices for VR displays & are used in conjunction with a screen. The screen rapidly displays alternating images for the left & right eyes; while the glass block the left & right eyes, corresponding to the images presented on the screen. Thus gives the perception of Stereoscopic world in 3D.

HAPTIC SYSTEMS: They are present in synthetic environment. A range of application can benefit from
the devices which allow you to control objects using the scene of touch. With the appropriate systems, sensation of roughness, smoothness, viscosity friction can be experienced.

AUDIO SYSTEM: VR system usually provides monophonic sound, which can be complementary to the
visual rendering but does not provide a realistic simulation of one sound perception. A stereo system, where sounds come from two separate sources, is the next step. ORIENTATION TRACKING: Mechanical armature can be used such armature s may look like a desk lamp or they may be highly complex ex skeletons. The drawbacks of mechanical sensors are the exuberance of the devices & its restrictions on motions. Ultrasonic sensors can be used to track position and orientation. A set of emitter & receivers are used with a known relationship between the emitters &b/w the receivers. The emitters are pushed in sequence & the time tag to each receiver is measured. Triangulation give orientation drawbacks to ultrasonic are low resolution, long tag times & other noise in the environment. Inertial trackers have been developed that are small & accurate enough for VR. These devices only provide a rotational measure work. They are also not accurate for slow orientation changes.

Intelligent VR-Software Systems:-

SANJAY SINGH (LECTURER)

13