Shimizu, Carnegie Mellon University Jointly Develop High-Precision 3D Model System of Infrastructure that Reflect Damages from Drone Images

Shimizu Corporation (President: Kazuyuki Inoue) and Carnegie Mellon University (CMU) have jointly developed a system of creating high-precision 3D models. The system is to enhance the deterioration prediction technology for reinforced concrete infrastructure that also reflect fine cracks and damages from imaging data of reinforced concrete infrastructure by drone measurements. In the future, this high-precision 3D modeling will be used to pursue the development of technology for simulation analysis of the resilience and remaining lifespan of structures. The goal is to establish a system for predicting deterioration of infrastructure capable of integrated support of the entire process from data measurement to evaluation and diagnosis.

As the aging of infrastructure built during high economic growth, mid-50s to early 70s, extending the lifespan of structures and mitigating the risk of accidents and disasters through proper maintenance and management are societal problems. Meanwhile, the number of skilled workers who perform such maintenance and management is declining. To resolve these problems, it is essential to develop ICT measurement technology to replace visual inspection and analytical technology capable of accurately and precisely diagnosing and predicting structural deterioration.

In March 2018, Shimizu and CMU therefore embarked on joint R&D on a system for predicting infrastructure deterioration. The results from this represent the first phase of joint R&D. We succeeded in creating a highly detailed 3D model that creates the positioning and shape within an error range of a few millimeters, a feat which could not be achieved with conventional technology. This is aimed at the automated creation of structural analysis model that accurately reflect the state of damage essential for improving the precision of simulation analysis.

The system just developed is a 3D measurement system installed on a UAV (drone) that uses both stereo image measurement and laser measurement (LiDAR). It was built by incorporating 3D modeling techniques using proprietary algorithms and image analysis AI technology that identifies structural deformation from images.

We used a measuring device that incorporates a stereo camera consisting of two high-resolution cameras and LiDAR to obtain 3D measures of a structure. A highly precise 3D shape is created from the photo images obtained. Specifically, the target structure is measured by the measuring device installed on the drone, and the positioning data from LiDAR is used to correct and reinforce the positioning accuracy of the 3D point cloud data obtained from the stereo camera, thereby creating a highly detailed 3D model.

Image analysis AI is then used to superimpose the crack shapes detected from the high resolution images taken by the stereo camera on the 3D model, completing a high-precision 3D model that includes information on damage. It takes several days to create a 3D model of a typical reinforced concrete bridge that is around 200 m long, including the measurements.

During first phase of this joint R&D, Shimizu established the development categories and measured reinforced concrete structures, while CMU took charge of drone control, 3D modeling techniques, AI-based crack detection, and development of other technologies. In the next phase, we will aim to achieve autonomous drone flight control and increase the sophistication of the measuring and evaluation technology. Shimizu will also be responsible for R&D on structural analysis and diagnosis, and will aim to complete the infrastructure deterioration prediction system. We think this will contribute to more efficient maintenance and management of infrastructure, and formulation of the best repair plans.