330,386 Sightings
|
Flukebook is a free, online research platform that assists the conservation of whales and dolphins by automating individual identification and providing a common data model for collaboration, saving significant time and money in the research process. Flukebook uses artificial intelligence and computer vision to help researchers rapidly curate cetacean sightings and photo-identification data across their catalogs, and it provides a portal for citizen scientists to contribute to marine conservation and follow individual animals they have met. One platform. Many species. Many researchers.
|
Flukebook Partners
Algorithms
MIEW-ID
MIEW-ID (µID) is a modern deep learning algorithm used to identify individuals. MIEW-ID can learn what makes images similar or dissimilar (or what differentiates one animal from another) across a wide range of species. Distinguishing individuals through their unique body markings is a key concept for wildlife conservation. MIEW-ID learns embeddings for images from the database. Embeddings are the unique markings that represent individuals. When new images are analyzed, their embeddings are matched against clusters of those in the database. As an added benefit, MIEW-ID is able generate visualizations of matched features, providing import inspectability inside its neural network. Learn more about MIEW-ID.
PIE v2
Pose Invariant Embeddings (or PIE) is a deep learning approach to individual ID. PIE is trained to learn embeddings that are useful for distinguishing among individuals in a wildlife population. Unlike HotSpotter, which is a “static” pattern matcher (i.e. a fixed algorithm not trained for each separate species), PIE can be trained on a per-species basis. Wild Me has generated separate PIE models optimized for manta rays, humpback whales, orcas, right whales, and so on. And unlike fixed-catalog classifiers like Deepsense or Kaggle7, PIE can gracefully add new individuals to its catalog without being retrained: it learns the general task of mapping images into embeddings that represent individuals, rather than the specific task of sorting images into a fixed number of IDs. PIE strikes a lovely balance between a flexible general-purpose identifier and one that can be trained and refined on a given problem. In summary, PIE is a very powerful, reusable machine learning technique that can be trained to identify individuals across many species. Learn more about PIE.
HOTSPOTTER
Hotspotter is a SIFT-based computer vision algorithm. It analyzes the textures in an image to find distinct patterning, or "hot spots", and then compares those against other images in the database. Unlike machine learning-based approaches, HotSpotter can help build new catalogs for new species that need to match individuals but don’t have the training data yet for machine learning-based approaches. Hotspotter can also match new individuals without the need for network retraining. Hotspotter produces a ranked list of potential matches, increasing match scores with increasing similarity. Learn more about Hotspotter.
CURVRANK V2
CurvRank v2 is a machine learning-based approach to matching a curvature or an “edge”, such as the trailing edge of a cetacean fluke or a dorsal fin. Specifically, machine learning is employed to weight sections of a fin or fluke that contain individually identifying information and to represent the curvature in a comparable format less subject to deformation from changes in the pose of the animal in the image. Learn more about Curvrank v2.
DEEPSENSE
Deepsense is a deep learning approach to identify right whales based on aerial photos of the callosity patterns on their heads. Deepsense is the winning entry from a Kaggle competition. It is fast and highly accurate, however it must be re-trained when new whales are added to a catalog. Like many deep learning competition winners, this algorithm is optimized for its specific task and cannot be cross-applied to other wildlife challenges. It is also highly subjective to the quality of the data in a right whales catalog, with performance degrading significantly (due to learned errors) if the catalog is not already very well curated. Learn more about Deepsense.
DTW
OC/WDTW (or DTW for short) is originally based on a sound wave matching technique called Dynamic Time Warping. DTW uses the A* algorithm to extract the trailing edge of a fluke and then look for matches to other edges by measuring the difference between them. This computer vision technique does not use ML and is optimized only for the trailing edge of humpback flukes, but has good accuracy and speed. Learn more about DTW.
KAGGLE7
Kaggle7 is a deep learning approach to identifying individual humpback whale flukes based on one of the winning entries from a Kaggle competition. Kaggle7 is fast and highly accurate at identifying individual humpback whales by their natural fluke patterning, however it must be re-trained when new whales are added to a catalog. Like many deep learning competition winners, this algorithm is highly optimized for its task and cannot be cross-applied to other wildlife challenges. It is also highly subjective to the quality of the data in a humpback catalog, with performance degrading significantly (due to learned errors) if the catalog is not already very well curated. Learn more about Kaggle7.
FINFINDR
finFindR is a machine learning-based approach to matching a curvature or an “edge”, such as the leading and trailing edges of a dorsal fin. More specifically, machine learning is employed to identify cetacean bodies and dorsal fins in an image, extract a fin edge, create a feature matrix representing that edge, and then compare that edge to others in the catalog. Learn more about finFindR.
MIEW-ID (µID) is a modern deep learning algorithm used to identify individuals. MIEW-ID can learn what makes images similar or dissimilar (or what differentiates one animal from another) across a wide range of species. Distinguishing individuals through their unique body markings is a key concept for wildlife conservation. MIEW-ID learns embeddings for images from the database. Embeddings are the unique markings that represent individuals. When new images are analyzed, their embeddings are matched against clusters of those in the database. As an added benefit, MIEW-ID is able generate visualizations of matched features, providing import inspectability inside its neural network. Learn more about MIEW-ID.
PIE v2
Pose Invariant Embeddings (or PIE) is a deep learning approach to individual ID. PIE is trained to learn embeddings that are useful for distinguishing among individuals in a wildlife population. Unlike HotSpotter, which is a “static” pattern matcher (i.e. a fixed algorithm not trained for each separate species), PIE can be trained on a per-species basis. Wild Me has generated separate PIE models optimized for manta rays, humpback whales, orcas, right whales, and so on. And unlike fixed-catalog classifiers like Deepsense or Kaggle7, PIE can gracefully add new individuals to its catalog without being retrained: it learns the general task of mapping images into embeddings that represent individuals, rather than the specific task of sorting images into a fixed number of IDs. PIE strikes a lovely balance between a flexible general-purpose identifier and one that can be trained and refined on a given problem. In summary, PIE is a very powerful, reusable machine learning technique that can be trained to identify individuals across many species. Learn more about PIE.
HOTSPOTTER
Hotspotter is a SIFT-based computer vision algorithm. It analyzes the textures in an image to find distinct patterning, or "hot spots", and then compares those against other images in the database. Unlike machine learning-based approaches, HotSpotter can help build new catalogs for new species that need to match individuals but don’t have the training data yet for machine learning-based approaches. Hotspotter can also match new individuals without the need for network retraining. Hotspotter produces a ranked list of potential matches, increasing match scores with increasing similarity. Learn more about Hotspotter.
CURVRANK V2
CurvRank v2 is a machine learning-based approach to matching a curvature or an “edge”, such as the trailing edge of a cetacean fluke or a dorsal fin. Specifically, machine learning is employed to weight sections of a fin or fluke that contain individually identifying information and to represent the curvature in a comparable format less subject to deformation from changes in the pose of the animal in the image. Learn more about Curvrank v2.
DEEPSENSE
Deepsense is a deep learning approach to identify right whales based on aerial photos of the callosity patterns on their heads. Deepsense is the winning entry from a Kaggle competition. It is fast and highly accurate, however it must be re-trained when new whales are added to a catalog. Like many deep learning competition winners, this algorithm is optimized for its specific task and cannot be cross-applied to other wildlife challenges. It is also highly subjective to the quality of the data in a right whales catalog, with performance degrading significantly (due to learned errors) if the catalog is not already very well curated. Learn more about Deepsense.
DTW
OC/WDTW (or DTW for short) is originally based on a sound wave matching technique called Dynamic Time Warping. DTW uses the A* algorithm to extract the trailing edge of a fluke and then look for matches to other edges by measuring the difference between them. This computer vision technique does not use ML and is optimized only for the trailing edge of humpback flukes, but has good accuracy and speed. Learn more about DTW.
KAGGLE7
Kaggle7 is a deep learning approach to identifying individual humpback whale flukes based on one of the winning entries from a Kaggle competition. Kaggle7 is fast and highly accurate at identifying individual humpback whales by their natural fluke patterning, however it must be re-trained when new whales are added to a catalog. Like many deep learning competition winners, this algorithm is highly optimized for its task and cannot be cross-applied to other wildlife challenges. It is also highly subjective to the quality of the data in a humpback catalog, with performance degrading significantly (due to learned errors) if the catalog is not already very well curated. Learn more about Kaggle7.
FINFINDR
finFindR is a machine learning-based approach to matching a curvature or an “edge”, such as the leading and trailing edges of a dorsal fin. More specifically, machine learning is employed to identify cetacean bodies and dorsal fins in an image, extract a fin edge, create a feature matrix representing that edge, and then compare that edge to others in the catalog. Learn more about finFindR.
