To count how many seabirds were in a colony, David Johnston and his team at Duke University used to walk among the black-browed albatrosses and southern rockhopper penguins with a clicker, as though they were bouncers at a bar. When one section was finished, it would be multiplied up to estimate the population of the total area.
Happily, expert marine biologists no longer have to spend their time on such repetitive tasks . With the arrival of drones for data collection and artificial intelligence for analysis, much of the work can be automated, making it easier for scientists to keep a careful watch on our oceans.
Researchers at the University of Exeter have similarly used overhead drones to observe orcas, gaining more understanding about how they choose their friends and interact socially. In Australia, drones are used to spot sharks at beaches, helping to keep swimmers and surfers safe while avoiding destructive practices such as culling.
And off South America, in the South Sandwich Islands, researchers from Oxford and the British Antarctic Survey are using drones to capture images of seabirds and seals, letting more be counted without disturbing the animals. Even underwater footage is being stitched together and analysed by AI, in projects by Arizona State University aimed at monitoring at-risk coral reefs.
From seabirds to whales
Johnston has always been interested in clever new uses of technology. Seeing how precision drones were used in farms and vineyards inspired him to wonder how the same ideas could be applied to his own work. “The potential was dramatic,” he said. “It was very obvious after we tried our first few missions that the sky was the limit on what we could do.”
Traditional animal counting methods are expensive, inaccurate and difficult — and can potentially harm the animals. For example, to measure blue whales, scientists previously had to hunt and kill the massive beasts in order to size them up. Attempts to take photographs from aeroplanes had a low success rate, as whales don’t surface on demand.
Now, Johnston says, it’s possible to go out on a small boat, seek out the whales, and within minutes send a drone up for measurements – without any need for destructive intervention. On shore, drones can similarly measure marsh grass without human researchers trampling it. “It reduces the effect of humans on sensitive habitats,” he said.
Drones also have the advantage of covering a wide area quickly. Measuring ocean temperatures is a core part of modern marine science, but satellites can only study areas larger than 5km wide. Drones can cheaply get more detail, and can be sent out as needed to investigate sudden changes. For example, surveying a beach previously required a laser scanner, but the work can now be done in a tenth of the time with a suitably equipped drone. If a hurricane is incoming, a team can collect data on the shape of the beach before the waves and wind hit, and do the same immediately after the storm passes. “There’s an amazing quality of immediacy,” said Johnston.
Training bird AI
Interpreting data is as important as collecting it, and that’s where artificial intelligence comes in. Seabird colonies are massive, making them difficult to count, and in this case the task is further complicated by the way the birds build their nests. “These albatross, they create these really cool nests on little pillars, with the egg on top where they sit,” Johnston explained. “The penguins stagger around underneath that.”
This means researchers have to count a double layer of birds. Doing so quickly and accurately is crucial for tracking population fluctuations caused by climate change or overfishing. “Our biggest challenge is trying to detect those changes before it’s too late,” he said. AI makes it possible to get good estimates as frequently as they’re needed. Previously, counting a seal colony on the ground would have taken a few days, while seabird colonies would take at least a week. “With this model, you could run it in an hour or an hour and a half,” said Johnston, depending on what hardware you’re using.”
Humans aren’t left out of the loop. The computer algorithms still need to be trained on relevant data, which at Duke has been largely done by team technician Maddie Hayes. Photos of bird colonies were labelled by humans, to show the AI what to look for, and the model was then run against that data set to teach it the difference between an albatross and a penguin.
To check accuracy, the results are regularly compared to traditional methods – but the outcomes are encouraging. “In almost all cases, we find that the neural network does a better job than people,” Johnston revealed. And if the system makes mistakes, it makes them consistently, making it possible to identify and correct them. “We know what its bias is, so we can apply it very efficiently every time… reducing the uncertainty associated with the values we produce.”
Listen up
While Johnston’s work focuses on collecting and analysing data, Ben Milner of UEA's School of Computing Sciences has been tackling a different problem: cleaning up the data we have. And rather than photographing the ocean, beaches and animals, Milner works with sound, which is the main way whales are tracked underwater.
Today there are only 350 North Atlantic right whales in the world, and they’re under threat from all types of human activity, such as getting caught in fishing equipment or being hit by ships. When a pack traverses an area, researchers try to steer the whales a different way, while warning industrial users of that area to be careful of the animals, or halt operations to let them pass safely.
To do so, researchers need to know the location of the whales. Historically this has been done using observers on ships, but that’s expensive and unreliable. “They’d be looking to see the whales break the surface, or for spray patterns,” Milner said. “Obviously you can’t do that at night or in fog or low visibility.”
Underwater microphones, called hydrophones, provide a partial solution. These can be placed permanently on buoys, or temporarily run underwater by drones, to listen for the distinctive calls of the right whales. The only problem is that their vocalisations can be masked by other sounds, including the very ships or offshore rigs that researchers are hoping to keep away from the animals. It remained hard to protect the whales without triggering expensive and disruptive false alarms.
Milner’s solution was to create a deep-learning algorithm to detect and remove unwanted noises. “If you can remove the background noise, then it leaves a cleaner signal for you to detect the right whale without missing it, and avoiding a false trigger,” he explained.
To train the AI, thousands of sound clips of whales, along with background noises such as shipping and drilling, were turned into images called spectrograms. Those were classified and labelled, letting the deep-learning system understand what each sound represented. Thus trained, the system can calculate the probability that a given audio clip contains the sound of a right whale, and researchers can set a threshold for raising the alarm, erring on the side of either higher or lower certainty. “You might say that anything above 0.3 is a right whale, so you’ll get anything that’s similar to a whale,” Milner said. “Or if you want to reduce false alarms, you might set a probability above 0.8 — you’re going to miss some, but you’ll be more certain.”
The whole system helps increase the amount of data that can be examined, while also boosting accuracy. “It will reduce the number of missed whales, and the number of false alarms,” said Milner.
Reuse and recycle
The beauty of AI is that the right-whale detection system can be reused for other animals simply by feeding in a new data set. “We could train our classifier so it knows what a blue whale sounds like – we can actually train it so it knows the difference between a blue whale and a right whale,” Milner said. “It can then detect either of them.”
And with the right data sets, this system could have uses well beyond whales. One of Milner’s students is adapting the system to track small mammals near Chernobyl using passive sound monitoring devices, to monitor their activity and recovery.
That reuse of scientific techniques mirrors how researchers are re-using existing technologies to help monitor the animal world. While Milner’s team had to develop these AI models themselves, the core ideas come from human speech processing and recognition, which is Milner’s research background. “The technologies apply just as well to right whales, or small mammals, or bats, because they’re just acoustic signals that we represent as images in spectrograms,” he said. In short, the same sort of systems that let you ask Alexa or Siri a question are being used to save endangered whales in the North Atlantic.
That reuse of existing innovations is how Johnston’s projects came about, too. The idea was sparked by the use of drones to capture agricultural data, and the equipment used is often off-the-shelf, consumer products: much of the work done by Johnston would be possible with a drone picked up at Best Buy, he says.
Even the AI is pre-made: the algorithm used by Johnston and his team was released as an open-source project, meaning it could be downloaded for easy use. “If someone before us hadn’t been committed to open science and had their code up… we would not have been able to do this as efficiently,” Johnston explained.
The future of innovation in marine research isn’t necessarily invention, but creative remixing. “So many of the tools for scientists to excel are out there and available,” said Johnston. “It’s just assembling them in unique ways.”
