\documentstyle[12pt]{myart} %\documentstyle[12pt]{article} \pagestyle{empty} \textwidth=6.5in \textheight=8.625in \topmargin=-0.33in \evensidemargin=0in \oddsidemargin=0in \parindent=0pt \parskip=12pt \pagestyle{empty} \begin{document} \bibliographystyle{plain} %\baselineskip=12pt % Macros for PostScript Figures: \pagestyle{empty} \thispagestyle{empty} \input psfig \date{} \title{Surface Mine Truck Safety Training: A VR Approach to Pre-Operational Vehicle Inspection\thanks{This work was partially supported by Newmont Gold Co., Echo Bay Minerals, the Nevada Division of Industrial Relations, and the Mine Safety Health Administration.}} \author { Damien Ennis$^{1}$, %Lu Chun Liao$^{1}$, %Mohammad Islam$^{1}$, Benjamin Lucchesi$^{1}$, Nerissa Oberlander$^{1}$, Keith Wesolowski$^{1}$, \\ Frederick C. Harris, Jr.$^{1}$, and Pierre Mousset-Jones$^{2}$ \\ $^{1}$Department of Computer Science, University of Nevada, Reno, NV~~ 89557 \\{\tt fredh@cs.unr.edu} \\ $^{2}$Department of Mining Engineering, University of Nevada, Reno, NV~~ 89557 \\{\tt mousset@mines.unr.edu} } \maketitle \thispagestyle{empty} \vspace{1in} \section*{ABSTRACT} \vspace{-12pt} In the surface mining industry, the cost of workplace accidents is high. One aspect of safety training is the pre-operational vehicle inspection. Training operators to correctly inspect vehicles is costly and time consuming in both equipment availability and man-hours. This paper outlines the motivation for and the development of an alternative training method, a virtual reality (VR) based Mine Vehicle Inspection Simulator (MVIS), which can cut costs. Implementation issues are discussed, an attempt is made to draw conclusions on the success of this method, and future work is outlined. \section{INTRODUCTION} \vspace{-12pt} Workplace accidents in the mining industry reduce production, increase costs, and result in temporary and permanent disabilities or even death to mine workers. Accidents are a major concern in day-to-day mining operations, where they can be expensive in terms of both cost and employee morale. One of the most important tools for on-the-job accident prevention is worker training. However, the cost of accident prevention training is high, particularly when the method attempts to provide a realistic representation of the risks associated with mining vehicle operation and the proper techniques that avoid or manage those risks. Preparing video demonstrations, conducting safety training tours of work sites, and conducting on-site safety briefings are all effective training tools, but these methods can be disruptive to daily operations and expensive. As a general rule, the more realistic a training exercise, the more expensive it is. Virtual reality (VR) is a technology for developing training tools that offers an excellent approach to reducing both job accidents and the high cost of training. The Mine Vehicle Inspection Simulator (MVIS) developed at the University of Nevada is an example of such a VR training tool. It provides a cost-effective simulation of vehicle inspection without the need to take a vehicle out of production. This paper provides a brief overview of the challenges facing the surface mining industry with particular reference to the pre-operational vehicle inspections of off-highway haulage trucks that are typically used in this type of mining. VR technologies currently available and their applications are discussed, and the approach selected for VR-based vehicle inspection and how the implementation was tested are explained. Finally, the results of this testing along with some conclusions and ideas for future work are presented. \section{TRUCK INSPECTION} \vspace{-12pt} General attitudes regarding industrial safety, environmental concerns, and industrial design have advanced significantly in recent years, and the mining industry has not been an exception. The introduction of new safety and environmental legislation throughout the world has changed the emphasis of industrial law from prescriptive legislation to the adoption of more effective management systems. Many mining companies have responded to these new ideas by introducing modern management philosophies~\cite{staley92:cscatsim}. A range of new techniques has been applied to meet the new legislative and production requirements. Large mining organizations are now looking for ways to improve their performance, including the use of new technologies for training employees. These new technologies include VR, videos, simulators, and planning systems. Their use has resulted in a safer working environment for all employees. The pre-operational vehicle inspection of a haul truck, such as the Caterpillar Model 785, is one of the first things a new employee at a surface mine learns how to do. The proper operation and maintenance of the haul truck fleet at these mines is the lifeblood of their profitability, and employees may be rotated to and from these vehicles throughout their employment. Therefore, profitability and safety go hand in hand in a company's desire to see that the employees perform the pre-operational inspection well. Current methodologies require employees to read about the inspection, see pictures of the parts to inspect, and be told the proper order in which to inspect parts. After passing a short test, a haul truck is brought out of production and the new employee inspects this vehicle under the supervision of a safety trainer. This is an effective training method but can be detrimental to company profitability. In order to perform an in-person inspection, an operational truck must be taken out of production for most of a shift. Current ore prices are so close to the break-even point that the loss of a truck from production will impact a company's profitability. This method is also inconvenient for employees that need to refresh their skills either due to the time elapsed since their initial training or their rotation to a new position. \section*{VR TECHNOLOGIES} \vspace{-12pt} The value of virtual reality is that it can offer experiences that would otherwise be inaccessible to an individual, because such experiences might be too expensive, too dangerous, occur at the wrong time, or in the wrong location. VR can be used as an alternative method of training because it can give the individual experience beyond training methods currently in use. One of the most successful examples of a VR application is commercial aircraft flight simulation. This extremely expensive implementation of VR provides a very realistic setting for pilot training that includes all of the sights, sounds, control motions, and physical movement associated with flying a modern jet. Hardware and software used to implement VR applications have improved dramatically over the last few years. High-end PC-class workstations, with the addition of a 3D accelerated video card, have the potential to support VR applications which were once restricted to expensive hardware-specific applications. In addition, a number of peripheral hardware devices, such as VR goggles, gloves, 3D sound, and motion tracking systems are designed to heighten the user's sense of reality and increase training effectiveness. Furthermore, software vendors are marketing a range of packages for developing VR applications. As noted by Foley, van Dam, et al. in \cite{fvdfhp97:itcg}, ``Interactive graphics is a field whose time has come.'' Software Application Programming Interfaces (APIs) that draw geometric primitives for developing graphically intensive software are becoming standardized. One such API is OpenGL. The challenge of using a low-level graphics API for developing software is that the application developer must make a substantial effort to create even the simplest 3D object. To reduce the complexity of developing graphically intensive software, toolkits built on top of low level graphics APIs are available. These toolkits focus development on user interaction and scene management rather than on drawing objects. One method of implementing this type of high-level toolkit utilizes scene graph technology (SGT). Scene objects such as cameras, lights, and geometry allow the programmer to concentrate on scene composition rather than on drawing objects. Scene graph technology shifts the graphics programming paradigm with the introduction of scene objects. Examples of APIs that implement SGT include Open-Inventor, Java3D, and Fahrenheit. In addition, these packages are independent of windowing systems and platforms. \section*{VR APPLICATIONS} \vspace{-12pt} The growth in the field of virtual reality has been fueled by the need for better computer-user interfaces. For this reason, applications of VR have been applied to a broad range of industries. In general, applications of VR can be reduced to three main categories: visualization, simulation, and control. The idea behind data visualization is that large sets of data can be better understood if they are presented in a form that allows people to assimilate important features of the data quickly. An obvious example of this concept is a simple bar graph. By correlating the numerical representation of data to magnitudes of bars in a graph, a person can evaluate the importance of the data rapidly. Because people are heavily biased toward visual pattern recognition as opposed to analytical analysis of numerical data, they respond better to graphical representations of data. VR takes this idea one step further by fully immersing a person in a representation of the data. Guan et al. \cite{gskhnc:vbtnpitvw} describe an application designed to let neurosurgeons study brain pathology, blood vessels, skull, and the surrounding tissue using real-time volumetric renderings of patient data. With this information, the surgeon can plan the best approach for surgery. % ( example of data visualization - VR to view stock market data ) Closely related to data visualization, the field of simulation is probably the largest and best known of all VR applications. VR allows the simulation of real world (and fictitious) situations by giving users experiences that might otherwise be infeasible for economic or safety reasons. Probably the best known application of this category is commercial flight simulation. Good examples of this can be seen at Reflectone \cite{flightsim}. % ( example of simulation - example of flight sim ) Virtual reality applications in the category of control offer some of the best possibilities for industrial applications. By simulating the environment where a remotely controlled device is located, users can control that device as if they were actually there. Applications of this technology allow users to control devices in economically prohibitive, hazardous, or inaccessible locations. % %( examples such as defusing bombs, Mars pathfinder, %deep sea probes, etc. ) One of the most famous examples of this was the microrover (named Sojourner) that NASA sent to Mars in 1996. Sojourner was man's first attempt to operate a remotely controlled device on another planet \cite{sojourner}. \section*{APPROACH TO VR AND VEHICLE INSPECTION} \vspace{-12pt} The goal of this virtual reality application is to provide an efficient and cost effective platform for users to learn the skills necessary for a vehicle inspection. These skills include identifying problems with the vehicle and determining the steps needed to correct these problems. Using this objective as a baseline for the content of the simulation, a system was implemented that presents the user with a model of a vehicle allowing the user to maneuver around the scene with an interface for selecting individual parts of the vehicle. It was first necessary to resolve two fundamental concerns of implementation common to all VR systems: presentation and interaction. The first involves how information is presented to the user both in terms of display technology and visual representation of data. The second deals with how the user interacts with the data presented. The key reason for using VR is to enhance computer user interaction by utilizing the user's ``natural talents for analysis and pattern recognition''\cite{pt95:vrttlg}. By making information easier for the user to assimilate, the user becomes more involved in what is seen and determines what action to take rather than simply evaluating the information. For example, the vanishing point of a simple drawing gives the viewer a sense of position, direction, and depth. The user can be mentally positioned relative to objects in the drawing because they relate to natural perceptions. For this reason, in this implementation of VR, data is displayed in a perspectively correct 3D scene. The data presented in the vehicle inspection is sky, ground, horizon, and a vehicle for inspection. As the user moves, the display interactively updates to reflect the user's position and perspective relative to the objects in the scene. As a result, users have a sense of position, size, and orientation while in the VR scene. A flat-screen display is used to display a 3D scene. A flat display was chosen over more expensive technologies like head-mounted displays due to cost and availability considerations. However, the flexibility of the hardware and software technologies used to implement the system allows the addition of other hardware including stereo-scopic glasses to add additional realism. Stereo-scopic glasses, which are worn like regular glasses, consist of two lenses that can change their opacity at a high frequency. By alternating the left and right eye's opacity from transparent to opaque while at the same time alternating the scene being displayed on the flat screen from a left eye's perspective to a right eye's perspective, the optical illusion of three dimensionality and perspective is further enhanced. To facilitate natural movement within a 3D environment, the software utilizes a six degree-of-freedom (6DOF) input device, such as a Space Orb$^{\tt TM}$. Using this hand-held device, the user can translate and rotate their location within the 3D scene. Other input devices for moving in VR worlds, such as keyboard, mice, and most joysticks force the user to move in a single plane with no rotation. However, a 6DOF device allows a user to translate both position and viewing orientation simultaneously. This results in natural movement in 3D space using a single intuitive device (See Figure~\ref{fig:person}). %The next step in implementing the VR system is to incorporate %user interaction with the scene. In order to select parts of a vehicle for inspecting, a standard mouse is used. By positioning the mouse pointer over the part of interest and clicking the mouse button, parts can be selected. Once selected, a dialog box containing information about the part is displayed. This information includes the name of the part and a description of the part. An edit box is made available for the user to enter the problem description and the appropriate action to take. As an additional feature, the user can re-inspect any part up to a maximum number of inspection attempts. This number can be set by the designer of the inspection scenario. During an inspection of a vehicle, all interaction the user has with the scene objects are recorded and timed. At the end of the inspection, a full account of the inspector's performance can be produced with information such as the amount of time taken to inspect the vehicle, a list of parts inspected, a list of parts not inspected, and the action taken by the inspector on inspected parts. \newpage The following steps summarize a user's interaction with a VR world while performing a vehicle safety inspection. \newcounter{bean} \begin{list}{$\bullet$}{\usecounter{bean} \setlength{\rightmargin}{\leftmargin} \parsep=0pt\itemsep=0pt\topsep=0pt} \item First, the user locates a part on the vehicle. To do this the user moves around the 3D scene using the 6DOF input device. \item Second, the user selects a part to inspect with the mouse. \item Third, the user types a course of action to correct the problem. \end{list} \begin {figure}[h] \centerline{\psfig{figure=mvc-003f.ps,height=4in}} \caption{A Person Conducting an Inspection with a SpaceOrb by SpaceTec [6]} \label{fig:person} \end{figure} Traditionally limited to workstations, VR can be prohibitively expensive to implement. In addition to realism, another objective was to make the system cost effective. As a result, the cost of the target platform was made an important consideration. Due to high availability of inexpensive components, PC-class workstations running Microsoft Windows NT with high-end video hardware was the obvious answer. This platform choice allows implementation on existing desktop PCs with minor modifications while at the same time remaining compatible with the next generation of PCs. Furthermore, the decision to use a high-level graphics API built on top of OpenGL and a standardized programming language makes it possible to rapidly port to platforms other than the PC. \begin {figure}[h] \centerline{\psfig{figure=new_fig2-bw.ps,width=6.5in}} \caption{Screen Shots of MVIS.} \label{fig:inspection} \end{figure} \section*{TESTING AND RESULTS} \vspace{-12pt} The testing sequence, using approximately 15 individuals, was organized into two stages. The first stage was designed to determine how long it would take for the users to become familiarized with the interface and acclimated to using the SpaceOrb$^{{\tt TM}}$ for navigating in the VR world. The second stage was designed to test the accuracy of the graphics representation by analyzing the ability of the user to correctly interpret the size of the vehicle. In the first stage, the testers were presented with 3 simple objects (cube, sphere, cone) positioned in a VR world. The testers were told that two objects could be inspected and one could not. The task of the testers was to find the inspectable objects while exploring the scene. This simple exercise allowed the user to practice using the SpaceOrb$^{{\tt TM}}$ and the interface for inspecting objects. Testers became proficient with the SpaceOrb$^{{\tt TM}}$ and navigation in approximately 10 minutes on average. In the second stage, the testers were presented with a full scale model of a CAT 785 haul truck placed in a VR world (See Figure~\ref{fig:inspection}). The truck was the only object in the scene. In this stage, the task of the testers was to navigate the scene and to explore the truck presented to them. Although testers were encouraged to find as many inspectable parts on the truck as possible, the intention was for the user to see as much of the scene from as many different angles as possible. At the end of the simulation the testers were asked several questions one of which was the approximate height of the truck. Those who tested the system were initially told that their height was five feet six inches. Sixty-seven percent answered within two feet and eighty-nine percent answered within four feet of the actual truck height (18 feet). Other than their height, the testers were given no visual clues. \section*{CONCLUSIONS AND FUTURE WORK} \vspace{-12pt} % \begin{itemize} % \item Overall results \\ % itemize benefits of VR training. \\ % brief summary of results % \item Future work \\ % stereo-scopic glasses.\\ % scenario setup \\ % vehicle tear-apart. % \end{itemize} This paper describes a VR-based Mine Vehicle Inspection Simulator (MVIS). MVIS provides a method of training that is superior in several ways to traditional training methods including book work, live vehicle inspections, and training videos. First, this method allows the user to learn through interaction rather than viewing static material. Second, it is cost effective because a vehicle does not need to be taken out of production for practicing inspections as with live vehicle inspections. Third, our system automatically records the user's progress as he or she interacts with the program providing a metric for performance. Finally, the vehicle used in the program is fully configurable so that different scenarios can be created and tested. In summary, our implementation of VR for training is more flexible and cost effective than other available training methods. The current product is being enhanced with the ability to separate an equipment model into parts that can be modified for the inspection application. This will make it possible to do inspection training for any vehicle for which there is a 3D model. VR has considerable potential for other applications in the areas of mining and mine safety. A vehicle operation training program is currently in development. This VR application involves creating a driving simulation for a haul truck along with the associated scenario set-up applications. This will allow trainers to place static and dynamic hazards into a particular open-pit mine model and have the trainees drive a vehicle in the mine while spotting and dealing with the hazards that have been arranged in different locations of the mine. Other applications being investigated are modeling of environmental reclamation of mines and underground mine evacuation training. \parskip=0pt \itemsep=0pt \bibliography{/staff/fredh/papers/bibdir/vr} \end{document}