Maintain fenders with a combination of drones and AI

With the advent of the fourth industrial revolution, the need to maintain port facilities using drones is increasing globally. Furthermore, ensuring proactive maintenance of port facilities to ensure their sustainable safety and serviceability becomes even more necessary because of the number of aging port facilities in Korea, whose lifespan is over 30 years in 2030, is expected to increase by about 50%.

In particular, it is very important in terms of port activities to ensure the safe arrival of ships in port for loading and unloading purposes. Fenders perform an important role in these situations. The fender is installed on the sea side of the superstructure of the berth wall to avoid damage to the hull and structure caused by the ship’s mooring force and frictional force. However, since most fenders are not directly accessible by road, inspectors are generally advised to approach using a floating boat and visually inspect the condition of the fenders. It is very dangerous, time consuming and difficult to obtain detailed information on damage from waves and other risks.

The Korea Institute of Construction and Civil Engineering (KICT) has announced a new test method for automatic detection of fenders that combine an AI model and a vision sensor on a vehicle. Unmanned aircraft. It specifically uses a deep learning network with a densely connected decoder-encoder format. It is one of the widely used networks for object detection at the pixel level, inspired by the eccentric function of human vision.

The AI ​​algorithm, developed by KICT’s Structural Engineering Research Department, a research team led by Dr. Min, Jiyoung, is named “densely connected receiving field pyramid (DRFP)” or “small version of DRFP (DRFPt)”. It aims to accurately and quickly extract pixel-level fenders from multiple UAV images.

To search efficiently over a wide area at once and to reduce computational complexity, the standard convolution and the expanded convolution are densely connected in a pyramidal pattern. And a dataset of fenders was collected using UAVs on various port facilities. The detection performance of the proposed model is compared with other deep learning models in the literature.

The results showed that the proposed model reliably detected fenders in images taken from various angles, with IoU and F1 scores exceeding 88%, despite the color changes. or the shape caused by the tide. Here, IoU (Intersection on Union) means the ratio of the overlapping area to the combined area of ​​the estimate and the underlying truth. The F1 score is a statistical measure of the accuracy of a test. 100% means perfect overlap and precision.

There are many risk factors in every nook and cranny of port facilities that can pose potential threats to inspectors. As a result, many port authorities are actively trying to adopt new remote inspection technologies such as UAVs (unmanned aerial vehicles) and USVs (unmanned floating vehicles), both to ensure safety. for inspectors both to facilitate their detailed and quantitative inspection of structural members. difficult to reach by road. These unmanned vehicles are often equipped with vision sensors, through which they continuously record video footage or single photos as they continue to move around the structure.

Considering the large scale of gate structures spanning kilometers, the native data size of high-resolution video recordings is often too large for conventional computers to manage. For example, about 4,000 aerial photo occupying 50GB of storage was collected in a 1.25 km covered concrete stretch and main submersible well structure at the Port of Incheon in South Korea, imaged with a 4k camera with a 50% overlap carried on unmanned aircraft. Therefore, to ensure efficient management of the huge aerial photo data over time, it is important to quickly extract only the target objects that require maintenance from the photo or video, and store and manage them. necessary quantitative information about the status of the target audience.

Lead researcher Dr. Min, Jiyoung said: “We are planning to upgrade this model to a fender condition check system. It will allow us to quantitatively detect the fenders. Damage such as missing parts or cracks from just UAV images This UAV-AI hybrid technology automatically assesses the future serviceability of the fender, ensuring safety for inspectors and reduce time costs in the field.”

Provided by the National Science and Technology Research Council