Wildbook blends structured wildlife research with artificial intelligence, citizen science, and computer vision to speed population analysis and develop new insights to help fight extinction. Here is our vision in one minute.

Or if you have a bit more time, check out this video about our work with NVIDIA:

2019-01-11

Where's the Wildbook team? If you take a look at our master branch on Github.com and separately in our Image Analysis repo, you'll see commits flying fast and furious. Wildbook 7 is nearing completion, and many of our platforms have already been upgraded to stay current with master. However, our recent support from Microsoft has given us time to rip off old band-aids and remove old hacks. We're re-implementing all existing Wildbooks with a common, new computer vision pipeline that can pivot and behave differently for each species. This has been a huge lift and far more work than we expected, but we're very happy to have the time and funding to finally do it. It's absolutely critical to drive complexity and cost down, and sometimes only experience can show how that should be done. Here is a list of what you can expect in the upcoming release of Wildbook 7.

• Consolidated computer vision pipeline for multiple algorithms and species, especially HotSpotter and CurvRank
• Intelligent Agent data collection from YouTube, combining multiple forms of machine learning to capture more data from the public. On whaleshark.org, our Intelligent Agent is significantly outperforming human researchers in raw data collection and curation yet finding data generally otherwise missed by the research community.
• Improved Image Analysis performance and stability
• CurvRank algorithm integration, support, and stability and speed improvements
• Improved multi-species support for data management and computer vision
• For spotted species, faster spot pattern matching and more stable matching clients
• Better in-page editing features
• Improved User management, providing a better foundation for growth in citizen science and engagement
• Multiple

Past news

According to a July 2017 study in the Proceedings of the National Academy of Sciences, a “sixth mass extinction” is underway, a trend signalled by widespread vertebrate losses that “will have negative cascading consequences on ecosystem functioning and services vital to sustaining civilization.” This meta-study is based on multiple, independent analyses and represents a growing awareness in the wildlife research community that more rapid assessment, response, and review are needed to understand and counter this decline.

Unfortunately, wildlife research efforts are frequently underfunded and small scale. The collection and management of wildlife data remains a largely ad hoc and academic exercise focused on moving small data sets (often in Excel and Access) into local, custom population studies for “one-off” analyses without long-term data curation or collaboration across borders and regions. Arriving at a critical mass of data for population analysis can take years (especially for rare or endangered species). Long required observation periods and manual data processing (e.g., matching photos “by eye”) can create multi-year lags between study initialization and scientific results, as well as create conclusions too coarse or slow for effective and optimizable conservation action. This limits the scope, scale, repeatability, continuity, and ROI of the studies as they face the limits of their home-grown tools and IT capabilities.

Wildlife researchers lack a common yet customizable platform for collaboration and often don’t have the technical experience or budget to take advantage of advanced computing tools (e.g., computer vision, artificial intelligence). These tools allows projects to obtain, curate, and analyze “Big Data”, such as the potential of citizen scientists to collect and contribute large volumes of wildlife data through tourism and volunteerism.

Wildbook ® is an open source software framework to support collaborative mark-recapture, molecular ecology, and social ecology studies, especially where citizen science and artificial intelligence can help scale up projects. It is developed by the non-profit Wild Me (PI Jason Holmberg) and research partners at the University of Illinois-Chicago (PI Tanya Berger-Wolf), Rensselaer Polytechnic Institute (PI Charles V. Stewart), and Princeton University (PI Daniel Rubenstein).

Wildbook provides a technical foundation (database, APIs, computer vision, etc.) for wildlife research projects that are:

• tracking individual animals in a wildlife population using natural markings , genetic identifiers, or vocalizations
• collecting biological samples from a wildlife population and performing genetic and/or chemical analyses (e.g., stable isotope measurements, haplotype determination, etc.)
• engaging citizen scientists and\or using social media to collect sighting information
• looking to build a collaborative, distributed research network for a migratory and/or global species
• looking to develop a new animal biometrics solution (e.g., pattern matching from photos) for one or more species
• collecting behavioral and/or social data for a wildlife study population

The biological and statistical communities already support a number of excellent tools, such as Program MARK,GenAlEx, and SOCPROG for use in analyzing wildlife data. Wildbook is a complementary software application that:

• provides a scalable and collaborative platform for intelligent wildlife data storage and management, including advanced, consolidated searching
• provides an easy-to-use software suite of functionality that can be extended to meet the needs of wildlife projects, especially where individual identification is used
• provides an API to support the easy export of data to cross-disciplinary analysis applications (e.g., GenePop ) and other software (e.g., Google Earth)
• provides a platform that supports the exposure of data in biodiversity databases (e.g., GBIF and OBIS)
• provides a platform for animal biometrics that supports easy data access and facilitates matching application deployment for multiple species

Images have become the most abundant, available and cheap source of data. The explosive growth in the use of digital cameras, together with rapid innovations in storage technology and automatic image analysis software, makes this vision possible particularly for large animals with distinctive striped, spotted, wrinkled or notched markings, such as elephants, giraffes and zebras. This large number of collected images must be analyzed automatically to produce a database that records who the animals are, where they are, and when they were photographed. Combining this with geographic, environmental, behavioral and climate data would enable the determination of what the animals are doing, and why they are doing it.

Wildbook evolved out of multi-disciplinary, collaborative research conducted under National Science Foundation support (see ibeis.org). Wildbook employees computer vision and A.I. components to detect features in submitted images and detect and then identify individual animals. Wildbook brings massive-scale computer vision to wildlife research for the first time.

Wildbook integrates the data management software of Wild Me with the computer vision and A.I. research of RPI. Wildbook includes a two-part, multi-species computer vision pipeline to find and identify individual animals in photos collected under real-world conditions, especially with citizen science contribution.

Our detection pipeline is a cascade of deep convolutional neural networks (DCNNs) that applies a fully-connected classifier on extracted features. Three separate networks produce: (1) whole-scene classifications looking for specific species of animals in the photograph, (2) object annotation bounding box localizations, and (3) the viewpoint, quality, and final species classifications for the candidate bounding boxes.

In Wildbook, A.I.-powered detection finds and labels wildlife in photos.

The second major computer vision step is identification, which assigns a name label to each annotation from detection. To do this, SIFT descriptors are first extracted and then compared at keypoint locations. Scores from the query that match the same individual are accumulated to produce a single potential score for each animal. The animals in the database are then ranked by their accumulated scores. A post-processing step spatially verifies the descriptor matches and then re-scores and re-ranks the database individuals.

Example correct identifications. The upper left annotation in each frame is the annotation to be identified. The other frames are the other annotations for the same animal. The bottom left annotation is the primary matching frame. The colored line segments show connections between corresponding features of the same animal.

The results of computer vision are returned to Wildbook’s data management software, which supports rapid curation, export, and analysis. Data can be rapidly viewed in tables, maps, charts, calendars, and as thumbnails. Data can also be searched, filtered, and used in R, Mark, ArcGIS, Google Earth, and other applications.

Wildbook is used in public and private installations.

• Flukebook for multiple species of cetaceans, featuring computer vision for humpback whales, sperm whales, bottlenose dolphins, right whales, and other species upcoming
• Giraffespotter for collaborative giraffe research with the Giraffe Conservation Foundation, San Diego Zoo, and others
• Wildbook for Whale Sharks for whale sharks (Rhincodon typus)
• MantaMatcher for manta rays (Manta birostris and Manta alfredi)
• Internet of Turtles for hawksbill and green sea turtles
• Wildbook for Leopard Sharks

• Wildbook for Jaguars
• WWF Spain Wildbook for Iberian Lynx
• SpotAShark and N.C. Aquariums for ragged tooth nurse sharks (sand tigers)
• WWF Norppa Galleria for Saimaa Ringed Seals
• Spotting Giant Sea Bass for GSB along the California coast
• Oregon State University Marine Mammal Institute for
  • humpback whale genetics
  • right whales
  • sperm whales
  • Hector's dolphins
• Strandings of Oceania for cetacean strandings in the South Pacific
• Ocean Sanctuary's Seven Gill Shark Project
• SPLASH Catalog for humpback whales (Megaptera novaeangliae)
• University of Manitoba Polar Bear Library
• Scottish Natural Heritage for Flapper Skates
• Princeton University for zebras and giraffes
• …and others…

Learn how to get Wildbook.

Wildbook has an R package available at: https://cran.r-project.org/web/packages/RWildbook/index.html

The following support options can help you use Wildbook.

• User Manual
• Configuration Guide
• REST Guide
• Security Guide
• Wild Me Professional Services

Wildbook® is always free and open source. We are a community of IT professionals and wildlife researchers maintaining and improving a 21st century platform. However, sometimes you may need extra help on a deadline. Our non-profit Wild Me offers professional hosting and customization to fit your project's requirements. This helps us fund our non-profit projects. Contact us if you need help!

Wildbook is a long-term, multi-disciplinary, multi-institution project combining skilled people in computer science, data science, ecology, and software engineering..

Professor Tanya Berger-Wolf provides computer science, data science, and overall project leadership to Wildbook.

Jason Holmberg is the Director of Engineering for Wildbook. Jon Van Oast, Drew Blount, and Colin Kingen are the primary software developers. Jason Parham is our machine learning specialist. Together we bring a wealth of professional software engineering experience to Wildbook.

Professor Charles Stewart and his students provide artificial intelligence and computer vision research and technology to Wildbook.

Hendrik Weideman is completing his PhD on CurvRank, the matching technology that can identify individual animals based on natural edges, such as fin shapes.

Professor Dan Rubenstein provides ecology and biology guidance to Wildbook as well as field testing Kenya.

Dr. Jonathan Crall is one of the original developers of Wildbook's computer vision components under the IBEIS project and specializes in individual animal identification challenges in computer vision.

Dr. Scott Baker of Oregon State University designed the DNA-related components within the software and remains an active adviser on the project.

Ongoing support for Wildbook is funded by Microsoft's A.I. for Earth Program, Pineapple Fund, Amazon Web Services, collaborative co-investment by users, and donations to Wild Me.

Past development work for Wildbook has been supported by:

• National Science Foundation
• Oregon State University Marine Mammal Institute with the Office of Naval Research (ONR)
• Cascadia Research Collective
• Royal Caribbean Ocean Fund
• Wild Me
• British Ecological Society
• Georgia Aquarium
• Qatar Whale Shark Research Project
• Siren Fleet
• William C. Bannerman Foundation
• Prince Bernhard Nature Fund
• Waitt Foundation

Check out some screenshots of Wildbook in action.

You can help move Wildbook forward by making a donation! Your donation is tax deductible in the United States.

Can you donate BitCoin or Ethereum?

BitCoin wallet ID: 15KCE1xCGjhcDojjEpMhUA8L6JMdiNGTHy

Ethereum wallet ID: 0x5d41f2e86FeCD1205717B099a8546c5cF6F97e57

The following publications have resulted or been supported by Wildbook-related work:

• Paxton, A. B., Blair, E., Blawas, C., Fatzinger, M. H., Marens, M., Holmberg, J., Kingen, C., Houppermans, T., Keusenkothen, M., McCord, J., Silliman, B. R., and Penfold, L. M.. 2019. Citizen science reveals female sand tiger sharks (Carcharias taurus) exhibit signs of site fidelity on shipwrecks. Ecology 00( 00):e02687. 10.1002/ecy.2687

• Arauz R , Chávez E, Hoyos-Padilla EM, and Marshall AD. First record of the reef manta ray, Mobula alfredi, from the eastern Pacific. Marine Biodiversity Records 2019 12:3 https://doi.org/10.1186/s41200-019-0162-9

• Parham, Jason & Stewart, Charles & Crall, J.P. & Rubenstein, Daniel & Holmberg, Jason & Berger-Wolf, Tanya. (2018). An Animal Detection Pipeline for Identification. 1075-1083. 10.1109/WACV.2018.00123.
• Blount D, Holmberg J, and Minton G. (2018) Flukebook – A tool for cetacean photo identification, data archiving and automated fluke matching. International Whaling Commission report SC/67B/PH/03.

• McCoy E, Burce R, David D, Aca EQ, Hardy J, Labaja J, Snow SJ, Ponzo A and Araujo G (2018) Long-Term Photo-Identification Reveals the Population Dynamics and Strong Site Fidelity of Adult Whale Sharks to the Coastal Waters of Donsol, Philippines. Front. Mar. Sci. 5:271. doi: 10.3389/fmars.2018.00271

• Diamant S. et al. (2018) Movements and habitat use of satellite-tagged whale sharks off western Madagascar. Endangered Species Research, (36) 49-58.

• Rohner CA, Richardson AJ, Jaine FRA, Bennett MB, Weeks SJ, Cliff G, Robinson DP, Reeve-Arnold KE, Pierce SJ. (2018) Satellite tagging highlights the importance of productive Mozambican coastal waters to the ecology and conservation of whale sharks. PeerJ 6:e4161

• B. Norman, J. Holmberg, Z. Arzoumanian, et al. ; //Undersea Constellations: The Global Biology of an Endangered Marine Megavertebrate Further Informed through Citizen Science//, BioScience, , bix127, https://doi.org/10.1093/biosci/bix127
• Reynolds SD, Norman BM, Beger M, Franklin CE, Dwyer RG. Movement, distribution and marine reserve use by an endangered migratory giant. Divers Distrib. 2017;00:1–12.

• J. Parham, J. Crall, C. Stewart, T. Berger-Wolf and D. Rubenstein, "Animal Population Censusing at Scale with Citizen Science and Photographic Identification", AAAI 2017 Spring Symposium.
• Weideman, H.J, Z.M. Jablons, J. Holmberg, K. Flynn, J. Calambokidis, R. B. Tyson, J.B. Allen, R.S. Wells, K. Hupman, K. Urian, C.V. Stewart. 2017. Integral Curvature Representation and Matching Algorithms for Identification of Dolphins and Whales. doi: 10.1002/ar.23650

• S. Menon, T. Y. Berger-Wolf , E. Kiciman, L. Joppa, C. V. Stewart, J. Parham, J. Crall, J. Holmberg, J. Van Oast, “Animal Population Estimation Using Flickr Images”, 2nd International Workshop on the Social Web for Environmental and Ecological Monitoring (SWEEM 2017) .

• McKinney JA, Hoffmayer ER, Holmberg J, Graham RT, Driggers WB III, de la Parra-Venegas R, et al. (2017) Long-term assessment of whale shark population demography and connectivity using photo-identification in the Western Atlantic Ocean. PLoS ONE 12(8): e0180495. https://doi.org/10.1371/journal.pone.0180495

• 2016 IUCN Red List Assessment (Endangered) for Rhincodon typus (whale shark)

• Macena BCL, Hazin FHV (2016) Whale Shark (Rhincodon typus) Seasonal Occurrence, Abundance and Demographic Structure in the Mid-Equatorial Atlantic Ocean. PLoSONE 11(10):e0164440.doi:10.1371/journal.pone.0164440

• Norman B. and Morgan D. (2016) The return of “Stumpy” the whale shark: two decades and counting. Front Ecol Environ 2016; 14(8):449–450, doi:10.1002/fee.1418

• Z. Jablons. Identifying humpback whale flukes by sequence matching of trailing edge curvature. Master’s thesis, Rensselaer Polytechnic Institute, 2016.

• Araujo, G., Snow, S., So, C. L., Labaja, J., Murray, R., Colucci, A., and Ponzo, A. (2016) Population structure, residency patterns and movements of whale sharks in Southern Leyte, Philippines: results from dedicated photo-ID and citizen science. Aquatic Conserv: Mar. Freshw. Ecosyst., doi: 10.1002/aqc.2636.

• Norman B, Reynolds S and Morgan D. (2016) Does the whale shark aggregate along the Western Australian coastline beyond Ningaloo Reef? Pacific Conservation Biology 22(1) 72-80 Submitted. 1 April 2016

• Araujo G, Lucey A, Labaja J, So CL, Snow S, Ponzo A. (2014) Population structure and residency patterns of whale sharks, Rhincodon typus, at a provisioning site in Cebu, Philippines. PeerJ 2:e543

• Rohner CA, Pierce SJ, Marshall AD, Weeks SJ, Bennett MB, Richardson AJ (2013) Trends in sightings and environmental influences on a coastal aggregation of manta rays and whale sharks. Mar Ecol Prog Ser 482: 153–168, 2013.

• McKinney J, Hoffmayer ER, Holmberg J, Graham R, de la Parra R et al. (2013) Regional connectivity of whale sharks demonstrated using photo-identification - Western Atlantic, 1999 - 2013. PeerJ PrePrints 1:e98v1

• Bonner SJ & Holmberg, J (2013), Mark-Recapture with Multiple, Non-Invasive Marks. Biometrics. doi: 10.1111/biom.12045

• J. P. Crall, C. V. Stewart, T. Y. Berger-Wolf, D. I. Rubenstein and S. R. Sundaresan, "HotSpotter — Patterned species instance recognition," 2013 IEEE Workshop on Applications of Computer Vision (WACV 2013), pp. 230-237.

• Hueter RE, Tyminski JP, de la Parra R (2013) Horizontal Movements, Migration Patterns, and Population Structure of Whale Sharks in the Gulf of Mexico and Northwestern Caribbean Sea. PLoS ONE 8(8): e71883. doi:10.1371/journal.pone.0071883

• Fox S, Foisy I, De La Parra Venegas R, Galvan Pastoriza BE, Graham RT, Hoffmayer ER, Holmberg J, Pierce SJ. (2013) Population structure and residency of whale sharks Rhincodon typus at Utila, Bay Islands, Honduras. Journal of Fish Biology Volume 83, Issue 3, pages 574-587, September 2013

• Robinson DP, Jaidah MY, Jabado RW, Lee-Brooks K, Nour El-Din NM, et al. (2013) Whale Sharks, Rhincodon typus, Aggregate around Offshore Platforms in Qatari Waters of the Arabian Gulf to Feed on Fish Spawn. PLoS ONE 8(3): e58255. doi:10.1371/journal.pone.0058255

• Davies, Tim K., Stevens, Guy, Meekan, Mark G., Struve, Juliane, and Rowcliffe, J. Marcus (2012) Can citizen science monitor whale-shark aggregations? Investigating bias in mark-recapture modelling using identification photographs sourced from the public. Wildlife Research 39, 696-704.

• Marshall AD & SJ Pierce (2012) The use and abuse of photographic identification in sharks and rays. Journal of Fish Biology 80: 1361-1379

• Dick, D et al. (2014) geneGIS: Geoanalytical Tools and Arc Marine Customization for Individual-Based Genetic Records. Transactions in GIS, Volume 18, Issue 3, pages 324–350, June 2014.
• Holmberg J, Norman B & Arzoumanian Z (2009) Estimating population size, structure, and residency time for whale sharks Rhincodon typus through collaborative photo-identification. Endangered Species Research, (7) 39-53.
• Holmberg J, Norman B & Arzoumanian Z (2008) Robust, comparable population metrics through collaborative photo-monitoring of whale sharks Rhincodon typus. Ecological Applications 18(1): 222-223.
• Arzoumanian Z, Holmberg J & Norman B (2005) An astronomical pattern-matching algorithm for computer-aided identification of whale sharks Rhincodon typus . Journal of Applied Ecology 42, 999-1011.

The following publications have influenced our design and development of Wildbook:

• Hochachka WM, Fink D, Hutchinson RA, Sheldon D, Wong W-K, Kelling S. Data-intensive science applied to broad-scale citizen science. Trends Ecol Evol. 2012 Feb. 1;27(2):130. 137.
• Madon, B., Gimenez, O., McArdle, B., Scott Baker, C., & Garrigue, C. (2011). A new method for estimating animal abundance with two sources of data in capture-recapture studies. Methods in Ecology and Evolution, 2(4), 390-400.
• Michener WK, Jones MB. Ecoinformatics: supporting ecology as a data-intensive science. Trends Ecol Evol. 2012 Jan.;27(2).
• Schwarz, C. J. (2009). Migration and movement – the next stage. Pages 325-350 in Modeling Demographic Processes in Marked Populations Series: Environmental and Ecological Statistics , Vol. 3. Thomson, David L.;Cooch, Evan G.; Conroy, Michael J. (Eds.). Springer, New York.
• Williams BK, Nichols JD, Conroy MJ (2002) Analysis and management of animal populations. Academic Press, San Diego, CA.
• White GC, Burnham KP (1999) Program MARK: Survival estimation from populations of marked animals. Bird Study 46:120. 138.
• Whitehead, H. Analyzing Animal Societies: Quantitative Methods for Vertebrate Social Analysis (University of Chicago Press, 2008).
• Wieczorek J, Bloom D, Guralnick R, Blum S, Doring M, et al. (2012) Darwin Core: An Evolving Community-Developed Biodiversity Data Standard. PLoS ONE 7(1): e29715. doi:10.1371/journal.pone.0029715

Wildbook is a registered trademark of Wild Me.

Wildbook is distributed under the GPL v2 open source license.