Plankton Taxation with Open-Source Computer Vision tools
Dr Achim Gädke, Data Scientist
The Alaska Fisheries Science Centre (part of National Oceanic and Atmospheric Administration, Seattle, USA) and Lynker-Analytics (Wellington, NZ) have teamed up to monitor zooplankton populations in the Bering Sea more efficiently.
Zooplankton are the base of the ocean food chain, and are made up of free-floating microscopic animals like copepods and early developmental stages of larger animals, such as fish eggs and larval crab. Changing environmental factors can dramatically alter the species mix, size, and abundance of these populations. Likewise, monitoring the abundance and species composition of fish eggs can provide an early indication of changing stocks, especially for fish of commercial value, such as walleye pollock (Gadus chalcogrammus).
Previous methods of classifying, measuring, and counting plankton and fish eggs are labour intensive, slow and expensive. This initial project was to test the use of computer vision tools to dramatically improve the process.
Quantitative measurement is key to developing accurate models of the prey for the fishery species. The number of data points available to modelling is determined by the time required for counting and classification. The classification tasks vary and require a library of Computer Vision models: Mission-specific models alongside “standard” models.
Image augmentation (YoloV7 Training)
Computer Vision Object Detection Results (FiftyOne)
A vast image library of plankton species (> 11,000 images) was annotated and used to train various YoloV7 models. The image processing (augmentation) was chosen to match realistic conditions in the field: Mix of species at different magnification, imagery resolution and varying light conditions.
The ONNX ML format enables flexible deployment of model libraries. We achieved a sufficient throughput of images even on modest hardware (the model performed well on non-GPU (graphical processing unit) hardware, e.g a business laptop) and is expected to manage video feeds.
The accuracy of 96% for detecting and discriminating 50 different species exceeded our expectations. The image dataset did not only determine taxa (Taxa fine: 64 classes), but also sex and developmental stage. NOAA was provided a training and evaluation pipeline with open-source tools like CVAT and FiftyOne, so mission specific models can be developed and refined.
Confusion Matrix for YoloV7 detection > 50 species of plankton
The accuracy of the resulting models suggest that computer vision can be successfully applied to existing imagery of plankton and fish egg samples. The results of the project could also guide the best ways to capture images. There is now scope to expand their use into mixed samples, including images captured while in the field.
