“NorFisk: fish image dataset from Norwegian fish farms for species recognition using deep neural networks”
Authors: Alberto Maximiliano Crescitelli, Lars Christian Gansel and Houxiang Zhang,Affiliation: NTNU Aalesund
Reference: 2021, Vol 42, No 1, pp. 1-16.
Keywords: Annotated image dataset, Deep neural networks, Fish detection, Fish species recognition, Marine aquaculture applications
Abstract: Long-term autonomous monitoring of wild fish populations surrounding fish farms can contribute to a better understanding of interactions between wild and farmed fish, which can have wide-ranging implications for disease transmission, stress in farmed fish, wild fish behavior and nutritional status, etc. The ability to monitor the presence of wild fish and its variability with time and space will improve our understanding of the dynamics of such interactions and the implications that follow. Automatic fish detection from video streams at farm sites using neural networks may be a suitable tool. However there are not many image datasets publicly available to train these neural networks, and even fewer that include species that are relevant for the aquaculture sector. This paper introduces the first version of our dataset, NorFisk, which can be found publicly available at doi.org/10.18710/H5G3K5 It contains 3027 annotated images of saithe and 9487 of salmonids and it is expected to grow in the near future to include more species. Annotated image datasets are typically built manually and it is a highly time-consuming task. This paper also presents an approach to automate part of the process when generating these types of datasets with fish underwater. It combines techniques of image processing with deep neural networks to extract, label, and annotate images from video sources. The latter was used to produce NorFisk dataset by processing video footage taken in several fish farms in Norway.
PDF (13401 Kb)
DOI: 10.4173/mic.2021.1.1DOI forward links to this article:
| [1] Jennifer L. Bell, Randy Mandel, Andrew S. Brainard, Jon Altschuld and Richard J. Wenning (2022), doi:10.1002/ieam.4622 |
| [2] K Banno, H Kaland, AM Crescitelli, SA Tuene, GH Aas and LC Gansel (2022), doi:10.3354/aei00432 |
| [3] Ricardo J. M. Veiga, Inigo E. Ochoa, Adela Belackova, Luis Bentes, Joao P. Silva, Jorge Semiao and Joao M. F. Rodrigues (2022), doi:10.3390/app12125910 |
| [4] Alexa Sugpatan Abangan, Dorothee Kopp and Robin Faillettaz (2023), doi:10.3389/fmars.2023.1010761 |
| [5] M Paiano, S Martina, C Giannelli and F Caruso (2024), doi:10.1088/2632-2153/ad65b5 |
[1] The Nature Conservancy. (1964). Nature, 202(4930):337--337. https://www.kaggle.com/c/the-nature-conservancy-fisheries-monitoring/data, doi:10.1038/202337d0
[2] The OpenCV Reference Manual. (2020). 2020, https://docs.opencv.org/4.4.0/.
[3] Bjelland, H.V., Fore, M., Lader, P., Kristiansen, D., Holmen, I.M., Fredheim, A., Grotli, E.I., Fathi, D.E., Oppedal, F., Utne, I.B., and Schjolberg, I. (2015). Exposed Aquaculture in Norway, In OCEANS 2015 - MTS/IEEE Washington. IEEE, pages 1--10. doi:10.23919/OCEANS.2015.7404486
[4] Crescitelli, A.M. (2020). NorFisk Dataset, 2020. doi:10.18710/H5G3K5
[5] Cutter, G., Stierhoff, K., and Zeng, J. (2015). Automated Detection of Rockfish in Unconstrained Underwater Videos Using Haar Cascades and a New Image Dataset: Labeled Fishes in the Wild, In 2015 IEEE Winter Applications and Computer Vision Workshops. IEEE, pages 57--62. doi:10.1109/WACVW.2015.11
[6] Fernandez-Jover, D., Sanchez-Jerez, P., Bayle-Sempere, J.T., Valle, C., and Dempster, T. (2008). Seasonal patterns and diets of wild fish assemblages associated with Mediterranean coastal fish farms, ICES Journal of Marine Science. 65(7):1153--1160. doi:10.1093/icesjms/fsn091
[7] Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pages 580--587. doi:10.1109/CVPR.2014.81
[8] Grotli, E., Vagia, M., Fjerdingen, S., Bjerkeng, M., Transeth, A., Svendsen, E., and Rundtop, P. (2015). Autonomous Job Analysis: a method for design of autonomous marine operations, In OCEANS 2015 - MTS/IEEE Washington. IEEE, pages 1--7. doi:10.23919/OCEANS.2015.7401888
[9] Hossain, E., Alam, S. M.S., Ali, A.A., and Amin, M.A. (2016). Fish activity tracking and species identification in underwater video, In 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV). IEEE, pages 62--66. doi:10.1109/ICIEV.2016.7760189
[10] Joly, A., Goeau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.-P., Planque, R., Rauber, A., Palazzo, S., Fisher, B., and Muller, H. (2015). LifeCLEF 2015: Multimedia Life Species Identification Challenges, In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 9283, pages 462--483. 2015. doi:10.1007/978-3-319-24027-5_46
[11] Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollar, P. (2014). Microsoft COCO: Common Objects in Context, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 8693 LNCS(PART 5):740--755. http://arxiv.org/abs/1405.0312.
[12] Liu, S., Li, X., Gao, M., Cai, Y., Nian, R., Li, P., Yan, T., and Lendasse, A. (2018). Embedded Online Fish Detection and Tracking System via YOLOv3 and Parallel Correlation Filter, In OCEANS 2018 MTS/IEEE Charleston. IEEE, pages 1--6. doi:10.1109/OCEANS.2018.8604658
[13] Mandal, R., Connolly, R.M., Schlacher, T.A., and Stantic, B. (2018). Assessing fish abundance from underwater video using deep neural networks, In 2018 International Joint Conference on Neural Networks (IJCNN), volume 2018-July. IEEE, pages 1--6. doi:10.1109/IJCNN.2018.8489482
[14] Pelletier, S., Montacir, A., Zakari, H., and Akhloufi, M. (2018). Deep Learning for Marine Resources Classification in Non-Structured Scenarios: Training vs, Transfer Learning. In 2018 IEEE Canadian Conference on Electrical & Computer Engineering (CCECE), volume 2018-May. IEEE, pages 1--4. doi:10.1109/CCECE.2018.8447682
[15] Rathi, D., Jain, S., and Indu, D.S. (2018). Underwater Fish Species Classification using Convolutional Neural Network and Deep Learning, 2017 Ninth International Conference on Advances in Pattern Recognition (ICAPR). pages 1--6. doi:10.1109/ICAPR.2017.8593044
[16] Rauf, H.T., Lali, M. I.U., Zahoor, S., Shah, S. Z.H., Rehman, A.U., and Bukhari, S. A.C. (2019). Visual features based automated identification of fish species using deep convolutional neural networks, Computers and Electronics in Agriculture. 167(September):105075. doi:10.1016/j.compag.2019.105075
[17] Raza, K. and Hong, S. (2020). Fast and Accurate Fish Detection Design with Improved YOLO-v3 Model and Transfer Learning, International Journal of Advanced Computer Science and Applications. 11(2):7--16. doi:10.14569/IJACSA.2020.0110202
[18] Redmon, J. (2016). Darknet: Open Source Neural Networks in C, 2016. http://pjreddie.com/darknet/.
[19] Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection, In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 2016-Decem. IEEE, pages 779--788. doi:10.1109/CVPR.2016.91
[20] Redmon, J. and Farhadi, A. (2016). YOLO9000: Better, Faster, Stronger, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017-Janua:6517--6525. doi:10.1109/CVPR.2017.690
[21] Redmon, J. and Farhadi, A. (2018). YOLOv3: An Incremental Improvement, Tech report. http://arxiv.org/abs/1804.02767.
[22] Ren, S., He, K., Girshick, R., and Sun, J. (2017). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence. 39(6):1137--1149. doi:10.1109/TPAMI.2016.2577031
[23] Salman, A., Jalal, A., Shafait, F., Mian, A., Shortis, M., Seager, J., and Harvey, E. (2016). Fish species classification in unconstrained underwater environments based on deep learning, Limnology and Oceanography: Methods. 14(9):570--585. doi:10.1002/lom3.10113
[24] Salman, A., Siddiqui, S.A., Shafait, F., Mian, A., Shortis, M.R., Khurshid, K., Ulges, A., and Schwanecke, U. (2019). Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system, ICES Journal of Marine Science. (February). doi:10.1093/icesjms/fsz025
[25] Shah, S. Z.H., Rauf, H.T., IkramUllah, M., Khalid, M.S., Farooq, M., Fatima, M., and Bukhari, S. A.C. (2019). Fish-Pak: Fish species dataset from Pakistan for visual features based classification, Data in Brief. 27. doi:10.1016/j.dib.2019.104565
[26] Siddiqui, S.A., Salman, A., Malik, M.I., Shafait, F., Mian, A., Shortis, M.R., and Harvey, E.S. (2018). Automatic fish species classification in underwater videos: Exploiting pre-trained deep neural network models to compensate for limited labelled data, ICES Journal of Marine Science. 75(1):374--389. doi:10.1093/icesjms/fsx109
[27] Tamou, A.B., Benzinou, A., Nasreddine, K., and Ballihi, L. (2018). Transfer Learning with deep Convolutional Neural Network for Underwater Live Fish Recognition, In 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS). IEEE, pages 204--209. doi:10.1109/IPAS.2018.8708871
[28] Tuya, F., Sanchez-Jerez, P., Dempster, T., Boyra, A., and Haroun, R.J. (2006). Changes in demersal wild fish aggregations beneath a sea-cage fish farm after the cessation of farming, Journal of Fish Biology. 69(3):682--697. doi:10.1111/j.1095-8649.2006.01139.x
[29] Utne, I., Schjolberg, I., and Holmen, I. (2015). Reducing risk in aquaculture by implementing autonomous systems and integrated operations, In Safety and Reliability of Complex Engineered Systems, pages 3661--3669. CRC Press. doi:10.1201/b19094-481
[30] Villon, S., Mouillot, D., Chaumont, M., Darling, E.S., Subsol, G., Claverie, T., and Villeger, S. (2018). A Deep learning method for accurate and fast identification of coral reef fishes in underwater images, Ecological Informatics. 48(September):238--244. doi:10.1016/j.ecoinf.2018.09.007
[31] Viola, P. and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features, In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, volume1. IEEE Comput. Soc, pages I--511--I--518. doi:10.1109/CVPR.2001.990517
[32] Wang, X., Ouyang, J., Li, D., and Zhang, G. (2019). Underwater Object Recognition Based on Deep Encoding-Decoding Network, Journal of Ocean University of China. 18(2):376--382. doi:10.1007/s11802-019-3858-x
[33] Zhang, L., Li, W., Liu, C., Zhou, X., and Duan, Q. (2020). Automatic fish counting method using image density grading and local regression, Computers and Electronics in Agriculture. 179(October):105844. doi:10.1016/j.compag.2020.105844
BibTeX:
@article{MIC-2021-1-1,
title={{NorFisk: fish image dataset from Norwegian fish farms for species recognition using deep neural networks}},
author={Crescitelli, Alberto Maximiliano and Gansel, Lars Christian and Zhang, Houxiang},
journal={Modeling, Identification and Control},
volume={42},
number={1},
pages={1--16},
year={2021},
doi={10.4173/mic.2021.1.1},
publisher={Norwegian Society of Automatic Control}
};