Edge Image Processing: Development of a Robust Sensor Platform for Bore-Hole Detection on Autonomous Ground Vehicles in Open-Pit Mines

TTP were recently challenged to develop a sensor system to detect the presence and centre of boreholes. This system was to be attached to an AGV ensuring it could reliably deploy its sensors into a borehole.

Open-pit mining is like the beating heart of the modern economy, pumping out the raw materials we rely on daily, from metals in your smartphone to materials used in construction. Extracting these recourses is no mean feat, bore-holes packed full of explosives are used to break down the ores into smaller pieces that can be transported out for processing. Once drilled, boreholes are precisely inspected to check their alignment, structural integrity and measure the presence of debris or water. An inaccuracy in any of these could be catastrophic, from inefficient blasts to major safety risks, it’s important to get it right. For years this job has been left to people exposing them to air pollution, unstable ground, falling debris and heavy machinery. The deployment of autonomous ground vehicles (AGVs) promises to change this, taking people out of harm’s way. TTP were recently challenged to develop a sensor system to detect the presence and centre of boreholes. This system was to be attached to an AGV ensuring it could reliably deploy its sensors into a borehole.

The obvious answer is computer vision. These algorithms essentially ‘see’ the borehole by identifying dark patches (caused by shadows) in an image of the mine surface. The theory is simple: find a dark circle in the image, compute its centroid, and you’ve found the borehole. A bit of pre-processing then filtering these by size and circularity helps gets rid of any spurious detections. So that’s what we did.

This works well, with a little bit of tuning and image preprocessing, you can get pretty good results, without breaking the bank on hardware costs. What an elegant solution! In this image our algorithm worked well, the reality of reliable bore-hole detection in open pit mines is a whole different beast. Harsh shadows confuse the sensors into thinking they’re seeing multiple boreholes, soil can be as dark at the boreholes themselves and the uneven surface means no hole is truly round. That is if the camera can see through the clouds of dust that are all too common in mines.

So how can we make this system robust? You could try to control the environment. For example adding curtains around the AGV to block out the sun and some artificial lighting to make the scene repeatable. AI is another option, but that would require thousands of labelled training images taken in wide range of operating conditions, a time consuming and costly process. Our solution was to take a more active approach and use structured light. Rather than relying on passive illumination, we project a laser pattern onto the ground, which naturally distorts as it interacts with the undulating surface. By measuring these distortions, we can accurately determine the boreholes position and shape, irrespective of shadows and soil colour. The hardware is only slightly more complicated, we still use a camera but with an additional filter that prioritises the wavelength of the laser. This robust algorithmic approach is much better suited to the harsh and ever-changing environmental conditions in mines.

A robust algorithm is no good if the camera is constantly covered in dust and the hardware can’t survive the high temperatures or being knocked around. Creating a system that not only detects boreholes but also thrives in the harshest environments requires a holistic approach. It is all about combining different technologies in the right way. Structured light for reliable detection, a compute efficient algorithm that can run on the edge, a robust mechanical enclosure with adequate thermal performance and a layer of sensor cleaning. No individual piece can do it all.

By integrating these robust technologies into AGVs, we’re not only making mining far safer, but more efficient too. Human workers no longer need to put themselves at risk inspecting boreholes in unstable conditions, and the repeatability of these autonomous systems ensures that every borehole is drilled, inspected, and detonated exactly as planned.

With minerals being the backbone of modern infrastructure, from smartphones to skyscrapers, having a reliable automated process for borehole detection is a small but critical step towards securing the flow of resources that power the world.