Automated detection and classification of scallops, flounder, and habitat
Project leads: Liese Siemann and Tasha O'Hara
Collaborator: Matt Dawkins, Kitware, Inc.
Funded by: NOAA Sea Scallop Research Set-Aside
CFF began collaborating with Kitware in 2019. The objectives of our first project were to develop automated detectors for different classes of scallops (live vs. swimming scallops vs. clappers) and flounder using the stereo image pairs taken during our HabCam surveys. Our current project is focused on automating scallop size classification and developing detectors to classify benthic habitats. Data collected during this project will be used to study habitat preferences of juvenile scallops. Our projects contribute to a broader NOAA effort to support development of Video and Image Analytics for Marine Environments (VIAME), an open-source system for analysis of underwater imagery that is being developed by Kitware. Our partners in this effort are early supporters of VIAME, including scallop assessment scientists at the Northeast Fisheries Science Center.
Human annotation of images is the most time consuming part of generating reliable biomass estimates from optical surveys, and humans can be imprecise, particularly when classifying substrate types. and substrate classification. To develop accurate automated detectors, CFF has provided Kitware with annotated imagery from our 2017-2020 surveys. We are currently developing a classification scheme based on the Coastal and Marine Ecological Classification Standard to accurately describe habitat types.
To accurately measure animals that could be resting or swimming off bottom, determining their height off the seafloor is critical. Accomplishing this starts with generating depth maps from stereo images. These depth maps can also be used to create 3D renderings of the sea floor.
Automated detectors can find scallops missed by human annotators.
