Safety @ Sea
Person Overboard Technology
Beating the Human Eye at Person Overboard Detection
Spending long hours looking for a person in the water through binoculars has many human-element pitfalls. Human eyes get tired, distracted and can miss things. AI is the not-so-simple alternative.
By Wendy Laursen
Of the 308 POB incidents from ships, fishing vessels and recreational boats reported to MAIB between 2015 and 2023, 40% lost their lives.
Image courtesy SEA.AIDevelopers at SICK Sensor Intelligence started their AI-based person overboard (POB) detection development by throwing boxes out of their office window. Then, nine water-loving colleagues jumped from heights of 10, five and three meters at the local swimming pool to provide imagery for training their AI algorithms.
The challenge in using sensor data, such as the company’s 3D-LiDAR, for detecting someone falling from a vessel is that they can fall through the measuring planes of the sensor in a fraction of a second. For a real-time response, AI algorithms need to detect them in the moment.
To achieve the required speed, they have to ignore irrelevant details in the images: data overload handicaps real-time analysis. And to be successful at sea, they also have to perform in adverse weather and ignore non-human objects – causing SICK to also throw deck chairs overboard and use a large dummy bird on a fishing rod at sea.


Once a person has fallen overboard, it doesn’t take long for the search area to cover thousands of square miles. Sandra Wienbeck, Product Manager, Identification & Measuring at SICK, says that due to sea drift and the speed of the vessel, it is usually quite difficult to define the exact location of a POB, and it takes a lot of time to stop and turn a cruise ship. Valuable time can be lost if an onboard search is required before action is taken – that’s the point of installing AI fall detection systems, they are constantly on watch.
80% of deaths from drowning occur in the first 30 minutes, and the UK Marine Accident Investigation Branch estimates that it takes under 11 minutes for a person in cold water to become unresponsive. Of the 308 POB incidents from ships, fishing vessels and recreational boats reported to MAIB between 2015 and 2023, 40% lost their lives.
Still uptake of POB technology, even by cruise operators, has been slower than some anticipated. Part of the reason is that the detection systems can produce too many false alarms by confusing the movement of things such as birds or hats with POBs. ISO standard 21195 was published in 2020 to help overcome the problem. It sets a minimum performance rate of 95% detection probability, and systems should not generate more than one false alarm a day. That way crews are more likely to respond immediately to alarms.

Once the person is in the water, there are a different set of challenges for humans and AI to spot and them track them. “Coast Guards assume 20% probability of detection from when they're doing a search pass during a rescue operation,” says Andy Telling, Business Development Director, at POB equipment company Zelim. “So if you've got somebody on watch looking out to sea, the chances of them spotting a person are pretty low, but you also have the potential for false detection as the human eye can play tricks on you and doesn't work well at night.”
That leaves a lot of room for AI to improve success rates, but, like humans, they rely on good visual input. The Singapore government’s HTX Sense-making and Surveillance Centre of Expertise team has tackled poor image quality caused by fluctuating light and wave conditions and the motion-induced blur that can affect cameras mounted on ships by enhancing color and contrast to achieve dim (low signal to clutter ratio) and small (typically <10 x 10 pixels) target detection. They have also superimposed clips of human bodies on sonar images to generate training data for their AI model for finding submerged people.
SEA.AI’s Sentry includes visual cameras as well as thermal cameras that can detect temperature differences as small as 0.05°C. The resulting AI detection technology can identify people in the water up to 700 meters away, day or night, and can track both individuals requiring rescue and the SAR personnel deployed to save them.

Current AI object detection and tracking methods can be thought of as either one and two-stage detectors. Two-stage algorithms typically identify candidate regions within an image and then proceed to classify and precisely locate the target. One-stage detectors use a faster, global regression-based classification approach instead.
As a study published by researchers from Hensoldt Optronics recently pointed out, the task differs depending on whether the images are coming from surface level or in the air. Ships offer a two-dimensional view from the side. In contrast, drones can view the ocean from above and offer a more stable platform for cameras. For use on drones, though, AI technology developers have to find a balance between lightweight hardware design and real-time processing capability.
Researchers from Dalian Maritime University have produced a “lightweight” version of the one-stage, open-source “You Only Look Once” object detection algorithm specifically for POB drones with limited memory and computational resources.
For tracking, AI must be able to maintain object identification across multiple image frames. They have therefore combined it with a tracking algorithm and have achieved a tracking speed of 82.7 frames per second, which is suitable for maritime search and rescue operations.
Others, including SICK and Zelim, are integrating their own systems into drone operations. SICK has partnered with INTSA and its partners Techseed, Robotic Aviation and Hatteland Technology in the development of a system that automatically sends out a drone when a POB alert is triggered. The drone can deploy a life preserver and track the person until a rescue craft arrives.
Zelim’s AI-based detection and tracking system ZOE is built on a dataset of over 4.9 million images and can track a POB to the horizon. It has been installed on vessels and an offshore platform. It has also been trialed onboard drones by Canada’s Civil Air Search and Rescue Association. The drones were flown over a section of coastline that had mannequins floating on the sea surface, and ZOE correctly identified them whilst ignoring the buoys, lobster ports and light reflections that human observers incorrectly identified as POBs.
ZOE is also incorporated into the navigation system of Zelim’s remote-controlled rescue craft that has capacity for 11 survivors who can be assisted onboard even if they are unresponsive by a conveyor-belt like recovery system.
In a POB situation, both sea and aerial crews take on a multifaceted role that includes maintaining navigational safety whilst undertaking search and rescue operations. AI is evolving to assist them.
