Automation of unmanned counting for spawners of chum and coho salmon with methods of artificial intelligence

Published methods of unmanned counting of pacific salmon allow to obtain high-quality photographic materials, but their processing is a rather hard labor. Practice of using the artificial intelligence methods, in particular deep learning models (otherwise — artificial neural networks), for this purpose is still insufficient, preliminary and concerns the counts of post-spawn chum salmon only. In this study, a deep learning model was created capable for effective automatic species-specific detection of chum and coho salmon spawners and post-spawn chum salmon in multi-species aggregations using enumerated data of unmanned surveys. This neural network is based on wide training materials and is able to process simultaneously images with several classes of objects and identify all of them. Detailed and reproducible technical description of the preparation, quality control, and operation with the deep learning model for pacific salmon is presented for a desktop GIS environment. Various factors affecting quality of the neural network are described, and recommendations for improving its work are given. Quality indicators of the model should be considered in the context of visual characteristics of target objects in processed photographic materials, which can vary significantly. Quality of the neural network output can be improved by using functionality of geoinformation software. Recommendations are given for optimizing the model development using the tools of photogrammetric and geoinformation software. Scheme for recalculating the neural network output based on processed photographic materials is proposed to obtain a corrected estimate of the total number of objects in the surveyed water area.

1. Bizikov, V.A., Peterfeld, V.A., Chernook, V.I., Kuznetsov, N.V., Petrov, E.A., Bobkov, A.I., Tkachev, V.V., Sidorov, L.K., and Boltnev, E.A., Metodicheskiye rekomendatsii po provedeniyu ucheta priploda baykal’skoy nerpy (Pusa sibirica) s bespilotnykh letatel’nykh apparatov v Baykal’skom rybokhozyaystvennom basseyne (Methodological recommendations for the accounting of the offspring of the Baikal seal (Pusa sibirica) from unmanned aerial vehicles in the Baikal fishery basin), Moscow: VNIRO, 2021.

2. Sviridov, V.V., Kotsyuk, D.V., and Podorozhnyuk, E.V., Photogrammetric counts of pacific salmon by means of unmanned aerial vehicles of consumer grade, Izv. Tikhookean. Nauchno-Issled. Inst. Rybn. Khoz. Okeanogr., 2022, vol. 202, no. 2, pp. 429–449. doi 10.26428/1606-9919-2022-202-429-449. EDN: CNQHAF.

3. Sviridov, V.V., Podorozhnyuk, E.V., Nikitin, V.D., and Skorik, A.V., Modifications of unmanned aerial survey on spawners of pacific salmons in the rivers of Sakhalin and Khabarovsk Regions, Izv. Tikhookean. Nauchno-Issled. Inst. Rybn. Khoz. Okeanogr., 2022, vol. 202, no. 4, pp. 1015–1031. doi 10.26428/1606-9919-2022-202-1015-1031. EDN: JULFRV.

4. Sviridov, V.V. and Povarov, A.Yu., Application of artificial intelligence methods to automate post-spawn chum salmon individuals enumeration by means of unmanned aerial vehicles in Khabarovsk Territory, Vopr. rybolov., 2024, vol. 25, no. 4, pp. 113–124. doi 10.36038/0234-2774-2024-25-4-113-124

5. Attard, M.R.G., Phillips, R.A., Bowler E., Clarke, P.J., Cubaynes, H., Johnston, D.W., and Fretwell, P.T., Review of Satellite Remote Sensing and Unoccupied Aircraft Systems for Counting Wildlife on Land, Remote Sensing, 2024, vol. 16, no. 4, pp. 1–23. doi 10.3390/rs16040627

6. Borowicz, A., Le, H., Humphries, G., Nehls, G., Höschle, C., Kosarev, V., and Lynch, H.J., Aerial-trained deep learning networks for surveying cetaceans from satellite imagery, PLoS ONE, 2019. vol. 14, no. 10, art. ID e0212532. doi 10.1371/journal.pone.0212532

7. Boulent, J., Charry, B., Kennedy,M., Tissier, E., Fan, R., Marcoux, M., Watt, C., Gagné-Turcotte, A., Scaling whale monitoring using deep learning: A human-in-the-loop solution for analyzing aerial datasets, Frontiers in Marine Science, 2023, vol. 10, pp. 1–13. doi 10.3389/fmars.2023.1099479

8. Butcher, P., Colefax, A., Gorkin, I. Kajiura, S., López, N. , Mourier, J., Purcell, C. , Skomal, G., Tucker, Ja., Walsh, A., Williamson, Ja., and Raoult, V., The drone revolution of shark science: A review, Drones, 2021, vol. 5, no 1, pp. 1–8. doi 10.3390/drones5010008

9. Desgarnier, L., Mouillot, D., Vigliola, L., Chaumont, M., and Mannocci, L., Putting eagle rays on the map by coupling aerial video-surveys and deep learning, Biological Conservation, 2022, vol. 267, pp. 109494. doi 10.1016/j.biocon.2022.109494

10. Dujon, A., Ierodiaconou, D., Geeson, J., Arnould, J., Allan, B., Katselidis, K., and Schofield, G., Machine learning to detect marine animals in UAV imagery: effect of morphology, spacing, behaviour and habitat, Remote Sensing in Ecology and Conservation, 2021, vol. 7, no. 3, pp. 341–354. doi 10.1002/rse2.205

11. Gorkin, R., Adams, K., Berryman, M.J., Aubin, S., Li, W., and Davis, A.R., Sharkeye: Real-Time Autonomous Personal Shark Alerting via Aerial Surveillance, Drones, 2020, vol. 4, no. 2, pp. 1–18. doi 10.3390/drones4020018

12. Gray, P., Bierlich, K., Mantell, S., Mantell, S., Friedlaender, A., Goldbogen, J., and Johnston, D., Drones and convolutional neural networks facilitate automated and accurate cetacean species identification and photogrammetry, Methods in Ecology and Evolution, 2019, vol. 10, no. 9, pp. 1490–1500. doi 10.1111/2041-210X.13246

13. Guirado, E., Tabik, S., Rivas, M., Alcaraz-Segura, D., and Herrera, F., Whale counting in satellite and aerial images with deep learning, Scientific reports, 2019, vol. 9, no. 1, pp. 14259. doi 10.1038/s41598-019-50795-9

14. Infantes, E., Carroll, D., Silva ,W., Härkönen, T., Edwards, S.V., and Harding, K.C., An Automated Work-Flow For Pinniped Surveys: A New Tool For Monitoring Population Dynamics, Frontiers In Ecology And Evolution, 2022, vol. 10, pp. 1–17. doi 10.3389/Fevo.2022.905309

15. Li, J.Y., Duce, S., Joyce, K.E., and Xiang, W., SeeCucumbers: Using Deep Learning and Drone Imagery to Detect Sea Cucumbers on Coral Reef Flats, Drones, 2021, vol. 5, no. 2, pp. 1–28. doi 10.3390/drones5020028

16. Rodofili, E., Lecours, V., and LaRue, M., Remote sensing techniques for automated marine mammals detection: review of methods and current challenges, Peer J., 2022, vol. 10, pp. 1–22. doi 10.7717/peerj.13540