If you remember the virtual reality (VR) hype extravaganza in the early 1990s, you probably have a very specific idea of what virtual reality gear includes. Back then, you could see head-mounted displays and power gloves in magazines, on toy shelves and even in films -- everything looked futuristic, high tech and very bulky.
It's been more than a decade since the initial media frenzy, and while other technology has advanced by leaps and bounds, much of the equipment used in virtual reality applications seems to have stayed the same.
Despite first impressions, the field of virtual-reality technology continues to advance, albeit at a slower rate than technology in other disciplines. Advances are often the result of other industries, like military applications or even entertainment. Investors rarely consider the virtual reality field to be important enough to fund projects unless there are specific applications for the research related to other industries.
What sort of equipment does VR rely on? Depending on how loosely you define VR, it might only require a computer with a monitor and a keyboard or a mouse. Most researchers working in VR say that true virtual environments give the user a sense of immersion. Since it's easy to get distracted and lose your sense of immersion when looking at a basic computer screen, most VR systems rely on a more elaborate display system. Other basic devices, like a keyboard, mouse, joystick or controller wand, are often part of VR systems.
In this article, we'll look at the different types of VR gear and their advantages and disadvantages. We'll start with head-mounted displays.
A Head-mounted Display (HMD) is just what it sounds like -- a computer display you wear on your head. Most HMDs are mounted in a helmet or a set of goggles. Engineers designed head-mounted displays to ensure that no matter in what direction a user might look, a monitor would stay in front of his eyes. Most HMDs have a screen for each eye, which gives the user the sense that the images he's looking at have depth.
The monitors in an HMD are most often Liquid Cystal Displays (LCD), though you might come across older models that use Cathode Ray Tube (CRT) displays. LCD monitors are more compact, lightweight, efficient and inexpensive than CRT displays. The two major advantages CRT displays have over LCDs are screen resolution and brightness. Unfortunately, CRT displays are usually bulky and heavy. Almost every HMD using them is either uncomfortable to wear or requires a suspension mechanism to help offset the weight. Suspension mechanisms limit a user's movement, which in turn can impact his sense of immersion.
A few HMD models use other display technologies, though they are very rare. Other display technologies include:
There are many reasons engineers rarely use these display technologies in HMDs. Most of these technologies have limited resolution and brightness. Several are unable to produce anything other than a monochromatic image. Some, like the VRD and plasma display technologies, might work very well in an HMD but are prohibitively expensive.
Many head-mounted displays include speakers or headphones so that it can provide both video and audio output. Almost all sophisticated HMDs are tethered to the VR system's CPU by one or more cables -- wireless systems lack the response time necessary to avoid lag or latency issues. HMDs almost always include a tracking device so that the point of view displayed in the monitors changes as the user moves his head. (We'll examine tracking devices in a later section.)
Some systems use a special set of glasses or goggles in conjunction with other display hardware. In the next section, we'll look at such a system -- the CAVE display.
Virtual Reality and the CAVE
Students and researchers at the University of Illinois - Chicago developed what many VR specialists feel is the most immersive display system for VR environments. It's called the CAVE system, which stands for Cave Automatic Virtual Environment.
A CAVE is a small room or cubicle where at least three walls (and sometimes the floor and ceiling) act as giant monitors. The display gives the user a very wide field of view -- something that most head-mounted displays can't do. Users can also move around in a CAVE system without being tethered to a computer, though they still must wear a pair of funky goggles that are similar to 3-D glasses.
The active walls are actually rear-projection screens. A computer provides the images projected on each screen, creating a cohesive virtual environment. The projected images are in a stereoscopic format and are projected in a fast alternating pattern. The lenses in the user's goggles have shutters that open and shut in synchronization with the alternating images, providing the user with the illusion of depth.
Tracking devices attached to the glasses tell the computer how to adjust the projected images as you walk around the environment. Users normally carry a controller wand in order to interact with virtual objects or navigate through parts of the environment. More than one user can be in a CAVE at the same time, though only the user wearing the tracking device will be able to adjust the point of view -- all other users will be passive observers.
In the next section, we'll look at another kind of virtual reality display called the workbench.
Virtual Reality Workbenches
One display system that some VR researchers feel is only tangentially related to virtual environments is the workbench display. In the early '90s, the Office of Naval Research's Computer Science Liaison Scientist, Larry Rosenbaum, led a team of Navy VR engineers in the creation of a large display monitor that allowed multiple users to view it at the same time. Users can view the display vertically or tilted horizontally like a table or bench.
Users wear special goggles while looking at the workbench, just as they would in a CAVE system. Each user sees the same image projected from the workbench display. Because of the stereoscopic projection and the goggles' lens shutters, the objects displayed on the workbench appear to be three-dimensional.
The reason some VR researchers feel the workbench isn't a true representation of creating a virtual environment is due to immersion. Because the user is looking at a display that doesn't fill his field of vision, he remains aware that he is in the real world, although that world now includes virtual objects that he can manipulate. There is no virtual environment to explore -- if the user looks away from the display, he'll see an ordinary, physical room. However, this doesn't change the fact that workbench displays can be very useful.
One use for the workbench display is medical training. A surgeon can practice a procedure on a three-dimensional virtual patient while surrounded by a real medical staff. If the surgeon were to perform the same procedure while wearing an HMD, the people around him would either be characters under computer control or computer avatars representing other humans. With the workbench display, interaction with other people is natural and completely real.
Workbench displays are also helpful to military tacticians. Programmers can create realistic, three-dimensional representations of battlefields, giving military personnel an accurate view of battle situations. A good model can also reveal potential bottlenecks or hidden enemy encampments.
Other applications using workbench displays include visualization of scientific research or product research and development.
In the next section, we'll look at some of the devices that allow users to interact with virtual environments.
Virtual Reality Clothing
While creating impressive graphics and display systems remains an important part of the VR experience, many researchers feel that developing intuitive devices for user interaction is more crucial. Basic interactive devices like keyboards or joysticks are easy to use, but they also tend to hamper the sense of immersion. Ideally a user would become unaware of the interaction device completely.
Though progress has been slow, there are still some exciting developments in human-machine interfaces (HMI). While many industries help drive developments in graphics technology, not many have a vested interest in exploring new kinds of HMI. Typically, the industries involved in advancing HMI include the entertainment field, academic institutions and small VR firms. Still, there are several interesting HMI devices used in some VR systems. Particularly interesting are the devices designed to be worn by users. These include gloves and bodysuits.
Gloves have played a role in the VR craze from the very beginning, even though the original designers didn't necessarily intend for them to be used in VR systems. Using a wired glove, you can interact with virtual objects by making various hand gestures. Many people call the gloves DataGloves or Power Gloves, though both those terms specifically refer to particular models of gloves and are not generic terms. Not all gloves work the same way, though all share the same purpose: allowing the user to manipulate computer data in an intuitive way.
Some gloves measure finger extension through a series of fiber-optic cables. Light passes through the cables from an emitter to a sensor. The amount of light that makes it to the sensor changes depending on how the user holds his fingers -- if he curls his fingers into a fist, less light will make it to the sensor, which in turn sends this data to the VR system's CPU. In general, these sort of gloves need to be calibrated for each user in order to work properly. The official DataGlove is a fiber-optic glove.
Other gloves use strips of flexible material coated in an electrically conductive ink to measure a user's finger position. As the user bends or straightens his fingers, the electrical resistance along the strips changes. The CPU interprets the changes in resistance and responds accordingly. These gloves are less accurate than fiber-optic gloves, but they also tend to be much less expensive.
Of course, if you want a really accurate and responsive glove, you should use a dexterous hand master (DHM). The DHM uses sensors attached to each finger joint. You attach the sensors to your joints with mechanical links, which means the glove is like an exoskeleton. These gloves are more accurate than either fiber-optic gloves or those using electrically conductive material, but they are also cumbersome and clunky.
In the next section, we'll look at VR input devices.
Virtual Reality Input Devices
If you don't have a CAVE system or the cash to drop on a DataSuit, there are still a few options available to you to provide users a way of getting around a virtual environment without using a wand or joystick. Researchers believe devices that allow for more natural navigation within a VR environment also increase the user's sense of immersion. With this in mind, engineers and scientists developed a few different systems for user navigation.
One system is the treadmill. A treadmill is useful because the user remains stationary with respect to the real world, but feels as if he is actually walking through the virtual environment. Researchers have found it relatively simple to link a treadmill to a computer system so that a user's steps result in an appropriate adjustment in the system's graphics. An obvious limitation of normal treadmills is that you can only walk in two directions: backward or forward.
Some companies have developed omni-directional treadmills. These devices allow a user to step in any direction. Normal treadmills use a single motor, which exerts force either forward or backward relative to the user. Omni-directional treadmills use two motors -- from the user's perspective the treadmill can exert force forward, backward, left or right. With both motors working together, the treadmill can allow a user to walk in any direction he chooses on a walking surface wrapped around a complex system of belts and cables.
An alternative to a treadmill is a pressure mat. You may have seen pressure mats used with video games like "Dance Dance Revolution." There are many kinds of pressure sensors, though the most common are electromechanical pressure sensors. An electromechanical pressure sensor is a relay that activates when pressure is applied to the sensor. When the circuit closes, an electric current runs through it, signaling the CPU to make changes to the graphic output sent to the user.
The company VirtuSphere, Inc. offers a unique way for users to move around inside a virtual environment. It looks like a human-size hamster ball -- the user gets inside the sphere and walks around in it. The sphere rests on a stable platform that has several wheels resting against the sphere, allowing it to roll in any direction while staying in the same fixed position. Sensors in the wheels tell the CPU which way the user is walking, and the view within the user's HMD changes accordingly.
Within CAVE systems, some VR researchers are experimenting with a technique called passive haptics. "Haptics" refers to the sense of touch, so a haptic system is one that provides the user with physical feedback. A joystick with force-feedback technology is one example of a haptic interface device. Passive haptics are a little different in that they don't actively exert force against a user. Instead, passive haptics are objects that physically represent virtual elements in a VR environment. For instance, a real folding table might double as a virtual kitchen counter. Having something real to touch in a virtual environment enhances the user's sense of immersion and helps him navigate through the simulation.
In the next section, we'll look at the kinds of tracking systems you find in HMDs, DataGloves and other VR gear.
Virtual Reality Tracking Systems
Tracking devices are intrinsic components in any VR system. These devices communicate with the system's processing unit, telling it the orientation of a user's point of view. In systems that allow a user to move around within a physical space, trackers detect where the user is, the direction he is moving and his speed.
There are several different kinds of tracking systems used in VR systems, but all of them have a few things in common. They can detect six degrees of freedom (6-DOF) -- these are the object's position within the x, y and z coordinates of a space and the object's orientation. Orientation includes an object's yaw, pitch and roll.
From a user's perspective, this means that when you wear an HMD, the view shifts as you look up, down, left and right. It also changes if you tilt your head at an angle or move your head forward or backward without changing the angle of your gaze. The trackers on the HMD tell the CPU where you are looking, and the CPU sends the right images to your HMD's screens.
Every tracking system has a device that generates a signal, a sensor that detects the signal and a control unit that processes the signal and sends information to the CPU. Some systems require you to attach the sensor component to the user (or the user's equipment). In that kind of system, you place the signal emitters at fixed points in the environment. Some systems are the other way around, with the user wearing the emitters while surrounded by sensors attached to the environment.
The signals sent from emitters to sensors can take many forms, including electromagnetic signals, acoustic signals, optical signals and mechanical signals. Each technology has its own set of advantages and disadvantages.
- Electromagnetic tracking systems measure magnetic fields generated by running an electric current sequentially through three coiled wires arranged in a perpendicular orientation to one another. Each small coil becomes an electromagnet, and the system's sensors measure how its magnetic field affects the other coils. This measurement tells the system the direction and orientation of the emitter. A good electromagnetic tracking system is very responsive, with low levels of latency. One disadvantage of this system is that anything that can generate a magnetic field can interfere in the signals sent to the sensors.
- Acoustic tracking systems emit and sense ultrasonic sound waves to determine the position and orientation of a target. Most measure the time it takes for the ultrasonic sound to reach a sensor. Usually the sensors are stationary in the environment -- the user wears the ultrasonic emitters. The system calculates the position and orientation of the target based on the time it took for the sound to reach the sensors. Acoustic tracking systems have many disadvantages. Sound travels relatively slowly, so the rate of updates on a target's position is similarly slow. The environment can also adversely affect the system's efficiency because the speed of sound through air can change depending on the temperature, humidity or barometric pressure in the environment.
- Optical tracking devices use light to measure a target's position and orientation. The signal emitter in an optical device typically consists of a set of infrared LEDs. The sensors are cameras that can sense the emitted infrared light. The LEDs light up in sequential pulses. The cameras record the pulsed signals and send information to the system's processing unit. The unit can then extrapolate the data to determine the position and orientation of the target. Optical systems have a fast upload rate, meaning latency issues are minimized. The system's disadvantages are that the line of sight between a camera and an LED can be obscured, interfering with the tracking process. Ambient light or infrared radiation can also make a system less effective.
- Mechanical tracking systems rely on a physical connection between the target and a fixed reference point. A common example of a mechanical tracking system in the VR field is the BOOM display. A BOOM display is an HMD mounted on the end of a mechanical arm that has two points of articulation. The system detects the position and orientation through the arm. The update rate is very high with mechanical tracking systems, but the disadvantage is that they limit a user's range of motion.
To learn more about gear used in virtual reality systems, check out the links on the next page.
More Great Links
- "Leaping into the (virtual) future." The Fountain. University of North Carolina at Chapel Hill Graduate School. On-line Version, Spring 2004. http://gradschool.unc.edu/fountain/spr_05/sharif.html
- "The Virtual Reality Responsive Workbench." The Advanced Information Technology Branch of Information Technology Division at the Naval Research Laboratory. 2002. http://www.ait.nrl.navy.mil/3dvmel/projects/Workbench/Workbench.html
- "Virtual Reality." NCSA's Science for the Millenium. http://archive.ncsa.uiuc.edu/Cyberia/VETopLevels/VR.History.html
- Beier, K. "Virtual Reality: A Short Introduction." University of Michigan Virtual Reality Laboratory at the College of Engineering.
- Carlson, Wayne. "A Critical History of Computer Graphics and Animation." The Ohio State University. 2003. http://accad.osu.edu/~waynec/history/lesson17.html
- Hayward, Vincent, et al. "Tutorial: Haptic interfaces and devices." Sensor Review. Vol. 24, No. 1. 2004.
- Kohli, Luv, et al. "Combining Passive Haptics with Redirected Walking." ICAT 2005.
- Rash, Clarence E., Ledford, Melissa H. and Mora, John C. "Helmet Displays in Aviation - Images Sources." Visual Science Branch, Aircrew Health and Performance Division, U.S. Army Aeromedical Research Laboratory. http://www.usaarl.army.mil/hmd/cp_0002_contents.htm
- Robles-de-la-Torre, Gabriel. "The Importance of the Sense of Touch in Virtual and Real Environments." IEEE Multimedia. Vol. 13, No. 3, pp. 24-30. Jul-Sept, 2006. http://www.roblesdelatorre.com/gabriel/GR-IEEE-MM-2006.pdf
- Sutherland, Ivan E. "The Ultimate Display." Proceedings of IFIP Congress, pp. 506-508, 1965.
- The Encyclopedia of Virtual Environments http://www.hitl.washington.edu/scivw/EVE/index.html
- United States Patent # 5,742,263. Head tracking system for a head mounted display system.
- United States Patent # 6,152,854. Omni-directional treadmill.
- United States Patent # 6,916,273. Virtual reality system locomotion interface utilizing a pressure-sensing mat.