This project developed a prototype system for georeferenced augmented reality visualization of buried utility geospatial data and real-time monitoring of an excavator’s proximity to buried utilities in its vicinity. A graphics algorithm to place virtual 3D buried utility models in an augmented scene was successfully designed and validated. Next, a 3D machine control system for monitoring an excavator’s pose in real time during excavation operations was designed and developed. The research methodology involved three stages: Tracking, Representing in 3D, and Analysis. The technical feasibility of a fiducial marker-based end effector pose estimation system was experimentally evaluated for excavators, and its performance demonstrated to consistently estimate bucket tooth position within 2.5 cm (1 in.) of absolute error. Potential pitfalls and solutions to practical field implementation problems were identified that also included lighting and occlusion. The accuracy of the pose estimation was primarily dependent on camera calibration, marker corner detection, and marker rig precision. A multi-plane 3D camera calibration method using an affine transformation model was designed. A stereo marker rig was also designed along with a method to compare the camera pose estimation with the ground truth. Finally, algorithmic optimizations were designed to overcome some of the factors that contribute to the uncertainty of corner detection in fiducial markers. It was also found that in the outdoor environment the bright sunlight illuminance complicates the marker detection, but its negative effect can be cancelled by automatic exposure. These improvements lead to significant performance improvements in stereo fiducial marker detection at large scales. A working prototype was tested on several active construction sites with positive feedback from excavator operators confirming the solution’s effectiveness.

The final report is available.