A VR Approach to Surface Mine Truck Safety Training

Benjamin Lucchesi Nerissa Oberlander

Department of Computer Science

Frederick C. Harris, Jr.  Pierre Mousset-Jones
Department of Computer Science Department of Mining Engineering
fredh@cs.unr.edu  mousset@mines.unr.edu

University of Nevada Reno, NV 89557

Abstract

In the surface mining industry, the cost of workplace accidents is high. One of the most important tools for on-the-job accident prevention is safety training. Two areas of safety training are the pre-operational vehicle inspections and driver training. Training operators to correctly inspect and operate vehicles is costly and time consuming in both equipment availability and man-hours. This paper outlines the motivation for and the development of alternative training methods using virtual reality (VR). Two training tools developed at the University of Nevada are the Mine Vehicle Inspection Simulator (MVIS) and the Mine Vehicle Driving Simulator (MVDS). This paper provides a description of these tools as well as future product enhancements.

1 INTRODUCTION

Workplace accidents in the mining industry reduce production, increase costs, and result in debilitating injuries or death to mine workers. Accidents are a major concern in day-to-day mining operations and are expensive in terms of cost and employee morale.

One of the most important tools for on-the-job accident prevention is worker training. However, the cost of accident prevention training is high, particularly when the training method attempts to provide a realistic representation of risks associated with mining vehicle operation and the proper techniques that avoid or manage those risks. Preparing video demonstrations, conducting safety training tours of work sites, and conducting on-site safety briefings are all effective training tools. But these methods can be expensive and disruptive to daily operations.

Virtual reality (VR) is a technology for developing training tools that offer an excellent approach to reducing both job accidents and the high cost of training. Two such tools developed at the University of Nevada are the Mine Vehicle Inspection Simulator (MVIS) and the Mine Vehicle Driving Simulator (MVDS). These tools address the training issues of pre-operational vehicle inspection and driver training.

These products have been developed using PC-based real-time 3D graphics and cutting edge VR input devices. These products, designed for the Microsoft Windows NT platform, offer mining companies a cost effective alternative to traditional training methods.

In Section 2 we discuss the pre-operational vehicle inspection and the development of the Mine Vehicle Inspection Simulator as well as its features and future improvements that we are planning. In Section 3 we present the Mine Vehicle Driving Simulator and discuss how it can assist in operator training. This is followed by conclusions in Section 4 and acknowledgments.

Figure 1: Screen Shot of MVIS.

2 PRE-OPERATIONAL VEHICLE INSPECTION

Mining companies around the world agree that pre-operational vehicle inspections are effective in decreasing haulage-related accidents. Performed prior to taking a vehicle into the field, inspections by the vehicle operator identify hazardous damages that occur from normal use. The cost benefits of these inspections are twofold. First, they make the workplace safer by keeping faulty equipment out of the field and helping to reduce haulage related accidents. Second, pre-operational inspections extend vehicle longevity as an additional component to vehicle maintenance routines. However, training employees to correctly perform pre-operational inspections is expensive. Traditional training methods such as written material, videos, seminars, and live inspections suffer from one of two problems. Either the training method poorly conveys critical information (written and spoken) or is extremely expensive to conduct (live vehicle inspections). The need for a cost-effective training method appropriate for teaching pre-operational inspections has led to the development of the Mine Vehicle Inspection Simulator (MVIS). By using state of the art PC and VR technologies, MVIS combines the learning experience gained from live vehicle inspections with the cost effectiveness of written material and video training aids.

MVIS is a VR simulation of the pre-operational vehicle inspection designed for mid to high-end PC-class computers running the Microsoft Windows NT operating system. A screen shot of MVIS is shown in Figure 1. As a feature-rich software package, MVIS brings together PC-based computing and VR technology to offer mining companies a cost-effective alternative to traditional training techniques in terms of time, initial investment, and training effectiveness.

The key to MVIS’s effectiveness as a training tool is the virtual reality environment in which the trainee works. MVIS successfully implements virtual reality using 3D graphics and input devices suitable for movement in 3D space.

2.1 The Inspection Process

When the simulation is started, trainees are presented with a 3D model of a vehicle. Using the input device provided with the software, the trainee walks around the vehicle and performs a visual inspection. The trainee looks for parts that appear damaged or require a detailed inspection regardless of condition. To perform the inspection, the user identifies an item of interest by selecting it with the mouse. This action causes a window (dialog box) to be displayed allowing the trainee to type in inspection information. This information would be a description of the process used to inspect that part. This information can then be used to determine if the trainee

• knows how to correctly identify parts of the vehicle that require inspection

• recognizes damaged parts

• knows the proper course of action to take in the case of a damaged part

Results are recorded by the simulator, and a report is produced upon completion of the inspection. These results can then be reviewed by a qualified trainer for evaluation purposes. Additional information recorded by the simulator includes the total time taken to perform an inspection, position in the scene where the trainee inspects an object and the number of attempts made to inspect each part.

2.2 Value-Added Features

During development of MVIS, several training professionals from the mining industry contributed ideas and suggestions for improvement. We have taken many of these suggestions and have either integrated them into our current implementation of MVIS or are planning to incorporate them in future versions.

2.2.1 VR interaction

The key to effective virtual reality is the ability to draw users into the simulation. By making the simulation as real as possible, users will get more out of their experiences in VR. One major area of VR that affects its effectiveness is the interface used for movement in 3D space. MVIS utilizes an input device more suitable for movement in 3D than traditional computer input devices. Traditional computer input devices (mouse, track balls, touch pads, and keyboard) are ineffective for moving in 3D space because these devices limit movement to two dimensions in a single action. For example, the arrow keys on a keyboard can only be used to move in a single direction. This is fine for walking, but how does the user look down or up while walking forward? In order to change the viewing direction, the user must either use a different key or a combination of keys. The same applies when using a mouse-type device (mouse, track ball, joy stick). These devices create a complicated user interface for 3D movement which can be confusing and discouraging. To alleviate this problem, we decided to take a different approach and use a 6-Degree-of-Freedom (6DOF) input device. As name implies, a 6DOF device allows for six types of movement: movement along the three axis (X, Y, Z) and rotation around these axis. Using this device, users can easily perform complex movements such as moving forward while scanning for obstacles on the ground or for those hanging from a ceiling. Moreover, all actions are performed using a single, intuitive, hand-held unit [6]. An example of a person using such a device with MVIS is shown in Figure 2.

2.2.2 Inspection dialog box

The inspection dialog box in MVIS (shown in Figure 3) is a window that allows trainees to enter inspection information for a vehicle part. In the first implementation of MVIS, this dialog box presented the user with a list of actions they could perform on a part being inspected. Mining professionals suggested that the dialog box be changed to query trainees for a description of how they would inspect a part rather than prompting them for a decision based on a list of options. This change was made based on the following reasons:

1. Providing a list of options to choose from is equivalent to a taking a multiple-choice test. Therefore, an individual with good reasoning skills could deduce the correct action. This does not give the trainer an accurate measure of the trainee’s knowledge.

2. Inspecting an individual part often involves a process rather than a single action. Multiple-choice tests are insufficient in these situations.

Therefore, we re-implemented the inspection dialog to contain a text box in which trainees can type a description of how they would inspect the part selected. This could include either the process needed to inspect a particular part or what to look for when inspecting that part. Furthermore, this modification allows MVIS to support inspection processes specific to each mine site.

Figure 2: An Inspection with a 6DOF Device (SpaceOrb [6]). Figure 3: MVIS with Inspection Box Open.

Figure 4: MVIS with the Position Locator View Open.

2.2.3 Lost in space

During preliminary tests of MVIS, the scene in which a user was placed contained no visual clues (such as sky, ground, and horizon) that could be used to maintain orientation. If a user got lost, it was difficult to regain orientation. Without a horizon, the user had no idea which way was up. With the addition of scenic elements, navigation in the VR world was much easier. However, we felt the addition of a second visual perspective would be useful to help users determine their position in complex scenes where they might be required to navigate between several landmarks. For these reasons, the Position Locator view was added to the user interface of MVIS. The Position Locator is a separate window that views the scene from a third person perspective, as shown in Figure 4. An icon is used to represent the position and viewpoint direction in the scene. The Position Locator can toggle between two default viewing modes, top-down and front view, for convenience and ease of use, or the user can freely rotate the entire scene in the viewer to create their own view from the best possible vantage point. The Position Locator can be opened and closed throughout an inspection at the user’s discretion.

2.2.4 Inspected part list

In real inspections, inspectors are often required to maintain an inspection log. However, because MVIS is computer based, the simulation records details of an inspection using the Inspection Dialog box. As an additional tool to aid trainees using MVIS, an Inspected Part List is provided. The Inspected Part List is a separate window that floats on top of the application and maintains a list of parts the trainee has inspected. This window also allows easy access to inspection details on a part-by-part basis so trainees can review their work before completing the inspection. The Inspected Part List can be opened or closed throughout an inspection. An example of the Inspected Part List can be seen in Figure 1.

2.2.5 Inspection administration

Although MVIS is primarily a training tool, it is equally effective for vehicle operator evaluation. For this reason, it is important that trainees using the software are unable to tamper with scenario configurations after an inspection has begun in an attempt to gain an advantage on their performance evaluation. Several safeguards are built into MVIS to ensure that, once an inspection has begun, all user interface elements (menus and buttons) that gain access to scenario configurations are locked and cannot be accessed until the inspection is complete.

2.2.6 Vehicle and vehicle configuration flexibility

Another benefit of MVIS is its configuration flexibility. MVIS uses a standard graphics file format to represent vehicles in VR. With this feature, a mine company can provide inspection models for every type of vehicle employees would encounter at a particular job site. Another feature of the file format used in MVIS is part configurations for individual vehicle components. In order to use this feature, defective part versions of inspectable parts must be defined in the inspection model. An example of a defective part might be a leaky or cracked strut. Each part in a vehicle can have an unlimited number of defective part versions. At runtime MVIS uses vehicle configuration information to determine which defective part version will be used when the model is displayed. Data that defines a vehicle configuration is called a scenario and is stored in a file that accompanies a model. This file can store any number of scenarios for the vehicle it accompanies. To select a scenario at runtime, a menu option is available prior to starting an inspection that allows the MVIS administrator to select a scenario. As a second method for defining a vehicle scenario, a scenario configuration dialog is available. This dialog box displays a list of available parts as well as a list of defective part versions available for each part. The MVIS administrator can select individual defective parts to be included in a vehicle configuration.

2.3 Future Enhancements

MVIS is an ongoing project that is continually being enhanced and modified to reflect the needs of our customers and the latest trends in the mine safety industry. Enhancement ideas that we have received from mining professionals that are slated for addition to MVIS include improved inspection evaluation, proximity selection, inspection ordering, model and scenario customization, and an enhanced inspection dialog box.

2.3.1 Improved inspection evaluation

At this time, inspection results are produced in the form of a report. This form of performance evaluation is suitable when the number of inspection points is small but becomes cumbersome as the number of inspection points grows. To make the process of evaluating an inspection easier, an inspection performance evaluation program is in development. This program will allow evaluators to gain access to valuable information as quickly as possible using an intuitive graphical user interface. As an added feature, this software will also give the evaluator access to information that was previously inaccessible. For example, the package will allow the reviewer to play back an animation of the path the trainee took while inspecting a vehicle. As a training aid, this information is useful in determining if the trainee is taking the most efficient path around a vehicle during an inspection.

2.3.2 Proximity selection

One of the major challenges facing mine safety trainers is conveying the importance of performing the actual inspection to new employees. All to often, vehicle operators perform cursory inspections and overlook important details that can be identified only by close examination. To help better train vehicle inspectors, proximity selection is being added to MVIS. In the current version of MVIS, objects can be inspected from any distance in the VR world. A bed pin on haul truck could conceivably be inspected from three miles away (provided the user can choose the part with the mouse). Proximity selection is a way to enforce maximum distances between the user and the part before a part can be selected. In the bed pin example, the MVIS administrator could enforce the policy that inspectors must be within three feet of a bed pin before it can be inspected. The distance from which a part can be selected will be defined in an inspection scenario definition on a part-by-part basis.

2.3.3 Inspection order

Most mining companies design their own process for performing the pre-operational vehicle inspections. It is the goal of MVIS to accommodate as well as complement the various ways to perform inspections and to aid trainers in teaching the process of inspections. A soon-to-be-added feature of MVIS is the ability to enforce the order in which groups of parts are inspected. For example, a mine might require that the front left wheel assembly (strut, steering linkage, wheel, tire, etc.) be inspected before all other parts. However, according to mining professionals, it is inconsequential in what order the parts within a group are inspected. The inspection order allows a training administrator to define groups of parts, as well as an order in which the groups are inspected. The unique aspect of this feature is that parts within a group can be inspected in any order, while group inspection order is enforced. Inspection of a group is not complete until all parts in that group have been inspected. This feature will be controlled from within the inspection scenario definition.

2.3.4 Model and scenario customization

Although model data files used by MVIS are standardized, the composition of the model data within the file is highly proprietary to MVIS. Therefore, we are currently designing a software tool that will allow mine companies to convert existing CAD/CAM models of vehicles and equipment (including defective parts) to the format supported by MVIS. This package will allow a scene designer to decompose a vehicle into inspectable parts and non-inspectable parts for use with MVIS and to identify where defective parts will be included in the model. This package will also be capable of designing and storing the scenario definition file for a specific model.

2.3.5 Enhanced inspection dialog box

One of the major challenges of providing a realistic inspection experience in VR is the representation of highly accurate models of vehicles. It is difficult to obtain computer models of vehicles that have the kind of detail needed for performing a thorough inspection. These models are usually guarded secrets of the vehicle manufacturer. Furthermore, even if one of these models was available, the sheer complexity of the model would be beyond the capabilities of modern PC hardware to render in real time for VR. For these reasons, we are going to add a detail view to the inspection dialog box. The detail view is a subsection of the inspection dialog box that displays an actual picture of the part being inspected. This feature will help eliminate the need to have extremely detailed models of vehicles while still demonstrating to the trainee exactly what should be examined.

3 DRIVING SIMULATOR

The second aspect of mine safety training being investigated at the University of Nevada is driver training. The United States Occupational Safety and Health Administration (OSHA) stated in 1995 that industrial trucks accidents are the second leading cause of fatalities in the private sector [5]. On average, there are 107 fatalities involving industrial trucks per year. The Mine Safety and Health Administration (MSHA) reports that over the last few years accidents involving operators of off-highway haulage vehicles in surface ground mining operations have resulted in between 15 and 20 fatalities per year and between 350 and 450 near fatalities per year [4].

Our team has developed a prototype simulator to assist in driver training appropriate to the surface mining industry. Our prototype package, the Mine Vehicle Driving Simulator (MVDS), is an attempt to determine the requirements of such a simulator with respect to technological challenges of software development and customer requirements. The prototype development of MVDS focused on two key areas: driving simulation and pit hazard simulation. MVDS is designed to train drivers on the skills necessary to safely operate power haulage in an open pit surface mine. MVDS accomplishes this by simulating vehicle operation in an open pit mine.

3.1 Product Overview

When a person begins their driver training with MVDS, they are seated in front of a computer screen on a desk with a steering device. MVDS uses a consumer grade steering device with a steering wheel, simple gear shift, and gas and brake pedals. On the screen, the trainee is presented with a 3D model of what they would see out of the front windshield of their vehicle when driving in a mine, as shown in Figure 5. MVDS can convert files containing mine layout data produced by mine software to a format it can use. This allows the trainee to view the pit in which they will be driving.

The trainee is to drive around the mine (possibly fulfilling some task) while observing the “rules of the road.” These rules for mine vehicle operators differ considerably from Department of Motor Vehicle regulations for highway vehicles. For example, in most mines, operators are required to drive on the left rather than the right side of the

Figure 5: Screen Shot of MVDS Prototype.

road. A simple representation of a dashboard is displayed at the bottom of the screen. The dashboard shows the trainee several gauges including speedometer, gearing indicator, and tachometer. The dashboard in combination with the steering device give the trainee a feel for where some of the gauges are located.

As the trainee drives, he/she is presented with hazards to avoid. MVDS supports two types of hazards, static and dynamic. Each presents the trainee with different types of situations to deal with. Static hazards, such as parked pickups, can be hard for trainees to see. Dynamic hazards, such as moving people, present trainees with the problem of trying to figure out where the hazard will be moving and whether a change course is needed to avoid the hazard.

3.2 Product Customization

Once the first prototype of MVDS was completed, the trainers that used it asked whether training scenarios could be added. In order to allow on-site trainers to create new scenarios, we have developed the MVDS setup program, shown in Figure 6. This program will allow on-site trainers to make new scenarios for trainees to navigate as the mine and operating regulations change. It can also be used to familiarize current vehicle operators with pits with which they are not familiar. The trainers will be able to change the mine by moving shovels, loaders, etc. and by closing routes the trainee may have used in the past.

When designing a scenario, the trainer can add static and dynamic hazards. To place a hazard, the trainer selects a location in the mine and chooses a hazard to go in that location. If the trainer is placing a dynamic hazard, the location is a path (or series of locations) the hazard will follow at a pre-defined speed. Figure 7 shows the placement of a dynamic path in a mine scenario. The bubbles show the path being selected by the trainer for a dynamic hazard. These can be turned on and off in the setup program. Any object that is modeled in an Open Inventor-compatible file format can be added as a static or dynamic hazard. We currently provide cones, people, pickups, loaders, and other vehicles.

Once a scenario is fully configured, this information can be saved to a file which is used by MVDS. Scenarios can be modified to reflect changes in the mine and MSHA regulations over time. This gives trainers the flexibility to build a library of scenarios used to accurately test the skills of vehicle operators.

Figure 6: A Completed Scenario Created in MVDS Setup. Figure 7: Creation of a Dynamic Hazard Path.

3.3 Future Enhancement

MVDS is an ongoing project that is continually being enhanced and modified to assist our customers in safety training. We have begun the design of enhancements including an improved physics model, improved simulation interaction, an expanded feature set, and a distributed architecture that will allow multiple drivers in the same pit.

3.3.1 Improved physics model

One of the key components of our next implementation of MVDS receiving the most attention is the physics model. The physics model of the simulation is responsible for interfacing user input with a simulated physical world in VR. The physics model determines vehicle simulation and environmental simulation. Vehicle simulation simulates the mechanical capabilities of the vehicle itself including turning radius, acceleration, braking capabilities, most of which change depending on whether the vehicle is loaded or unloaded. Environmental simulation is responsible for managing the physics of environmental conditions present in a mine. Environmental conditions we are considering adding include rain, snow, and high wind. These poor weather conditions greatly affect a driver’s ability to safely operate a vehicle. For example, rainy weather not only reduces a driver’s visibility, which will be selectable by the trainer, but also degenerates haulage roads causing vehicles to respond much differently than in dry weather conditions. In our new physics model, these examples of simulated physics and more will be implemented to give drivers a more accurate image of not only how a vehicle handles, but also how to safely operate a vehicle in various weather conditions.

3.3.2 Improved interaction

The effectiveness of any VR simulation is directly related to the level of realism of its user interaction. We can improve the user interaction of MVDS with the addition of force feedback steering wheels. These steering wheels have a motor in them to allow the wheel to be forced in a certain direction. If a mine vehicle driving down a haul road hits a berm on the side, the steering wheel will be forced very hard in the opposite direction. With a force feedback steering wheel, we can cause the wheel to do the same thing in the simulation.

3.3.3 Expanded hazard options

During initial evaluation, training professionals began to ask for more hazard selections. We are considering adding proximity-triggered hazards and time-triggered hazards. A proximity-triggered hazard is one that would appear when training vehicle moved close to it. For example, when the vehicle approached a surveyor’s truck, the surveyor may walk out in front of the vehicle from behind his truck. A time-triggered hazard is one that occurs at a certain time in the simulation. For example, a truck may begin moving from a parked location, or a rock slide may occur.

3.3.4 Distributed architecture

To further enhance the realism of the simulation, we are considering creating a distributed version of MVDS which will allow multiple trainees to be in the same simulation at the same time. The trainees will have to interact with each other as well as with other hazards that may already be in the scenario. Trainees will have to deal with unpredictable behavior of other trainees, completely eliminating the possibility of the trainee memorizing the situation being presented.

A distributed version of MVDS will also allow trainers to watch the trainees from a separate room, where trainees will not be aware of the close monitoring. The trainer will have multiple views of the simulation: one of the entire mine, with all of the trainees’ positions, and one shadowing a specific trainee. When shadowing a trainee, the trainer will be able to get detailed information on what the trainee is doing. The trainer will be able to see the distance between the trainee’s vehicle and other vehicles in the area, read all of the gauges on the trainee’s dash, and have a small window showing the 3D model of the mine that the trainee can currently see out of the windshield. This will allow the trainer to monitor those trainees that are just beginning more carefully than the experienced ones, without hanging over their shoulder.

Another advantage of a distributed version of MVDS will be faster reaction to user input. By taking the computationally intensive portions of MVDS and dedicating a machine to those processes, the graphics will be displayed on the trainee’s machine at a much faster rate. This will also allow the computationally intensive portions to run faster as well, since they will not be competing with the graphics for the machine the trainee is using. By separating the graphics and other portions, we will be able to improve the performance and capabilities of the entire program.

4 CONCLUSIONS AND OTHER PROJECTS

Initial responses to MVIS [1] and MVDS [2] have been extremely positive. The fact that these tools have been developed for the PC platform has been very appealing to the mining industry, since most companies use this platform.

There remains a great deal of work that can be done with VR in the areas of mining and mine safety. Besides the enhancements to the products we have outlined in this paper, we are also investigating applications for underground mines. Our investigations would complement the research already under way in the development of the Mine Emergency Response Interactive Training Simulation (MERITS) [3] being done at the NIOSH Pittsburgh Research Lab.

Mine regulations require extensive emergency training for underground mine personnel. Current training programs take considerable time and effort on the part of miners, supervisors, and trainers and do not adequately prepare mine personnel to handle complicating factors such as blocked routes, failure of lighting, and reduced visibility due to smoke. A VR training system would improve the convenience and quality of training, allowing a mine to better equip its personnel to handle emergencies. VR training would also provide rescue teams better training for emergencies in many mines without having to visit those mines for training.

Although virtual reality simulators are not new, employing such simulators to reduce training costs and accident rates is relatively new to the mining industry. VR training brings together the cost effectiveness of written and video-based training materials with the training effectiveness of practical experience.

ACKNOWLEDGMENTS

This work was partially supported by Newmont Gold Co., Echo Bay Minerals, the Nevada Division of Industrial Relations, and the Mine Safety Health Administration.

References

[1] Damien Ennis, Benjamin Lucchesi, Nerissa Oberlander, Keith Wesolowski, Frederick C. Harris, Jr., and Pierre Mousset-Jones. Surface mine truck safety training: A VR approach to pre-operational vehicle inspection. In Proceedings of APCOM’99 Computer Applications in the Mineral Industries 28th International Symposium, Colorado School of Mines, Golden, Colorado, October 20-22 1999.

[2] Benjamin Lucchesi, Nerissa Oberlander, Frederick C. Harris, Jr., and Pierre Mousset-Jones. Surface mine truck safety training: Scenario setup for a VR driving smulator. Submitted.

[3] Mine Emergency Response Interactive Training Simulation. http://www.cdc.gov/niosh/pit/mer merits.html. NIOSH, Pittsburgh Research Lab, Mining Health and Safety Research.

[4] Mine Safety Health Administration (MSHA). Injury tables (1996-1998). MSHA, Denver CO.

[5] Occupational Safety Health Administration (OSHA). Powered industrial truck operator training. Internet

   WWW page, at URL: http: //www.osha-slc.gov/ FedReg_osha_data/ FED19950314.html, 1995.

[6] Spacetec IMC, Inc. http://www.spacetec.com.