App combines computer vision and crowdsourcing to explore Earth’s biodiversity, one photo at a time

Taxonomy goes online in the 21^st century

Charles Darwin’s theory of evolution by natural selection was developed in large part from the observations and collections of plants, animals and fossils that he made in the Galapagos islands and other stops during his voyage aboard the HMS Beagle. However, Darwin was an amateur naturalist unable to identify many of the species he recorded while on the islands. He sought the help of expert taxonomists in England, who were able to identify the organisms he collected and shipped to them.

Perhaps most famous are Darwin’s finches, which the ornithologist John Gould classified into 13 distinct species. Darwin initially paid very little attention to these small birds, which he mis-identified as a collection of finches, blackbirds and “gross-beaks.”  It wasn’t until he returned to England over a year later and met with Gould that he learned of their actual identity – a realization that strongly influenced his conclusions about natural selection.

For today’s amateur naturalists, the iNaturalist online platform has greatly reduced the effort and time needed to obtain an accurate taxonomic identification. Rather than capturing, preserving and shipping a specimen to an expert, iNaturalist users just need to snap a photo with their camera or smartphone and upload it. From there, computer vision technology takes over to suggest an identification within a few seconds.

iNaturalist was launched in 2008 as a Master’s project by students from the University of California, Berkeley School of Information and has since grown into a community of nearly 500,000 users who, collectively, have captured more than 6.5 million observations of over 120,000 species of plants, animals, insects and fungi.

Until recently, photo observations that users uploaded to the iNaturalist platform were identified solely through crowdsourcing. Users can choose to provide an Identity (or ID, a common name or scientific name) for their observation, which is then either confirmed, improved or questioned by other members of the community. In this way, iNaturalist provides a platform for collaboration and discussion among people who share common interests and want to learn more about their local environment.

The platform also connects users with experts who provide identifications in their area of interest. For example, Greg Lasley, an expert birder, has provided a bird ID for over 100,000 observations uploaded by other users. The rapidly growing number of observations has begun to overwhelm this community of users who provide IDs. It takes an average of 18 days for an observation to be identified by the community, though the wait time is less than two days for half of all observations, mainly those of common species or in areas with many users, such as the West Coast of the U.S.

Tech for training a computer as a taxonomist

To help minimize the number of organisms going unidentified, iNaturalist recently collaborated with Visipedia to develop artificial intelligence software that provides an immediate ID for an observation. In an e-mail to Mongabay-Wildtech, Dr. Scott Loarie, Co-Director of iNaturalist, explained, “We’ve recently worked with the Visipedia team to use the iNaturalist database of over 5.5 million observations to train up a computer vision neural network to automatically identify species.”

What exactly is ‘computer vision’?  It’s the technology that social media websites like Facebook use in their facial recognition software and that some cars use to detect pedestrians in their path. However, identifying a human-shaped object is much simpler than differentiating butterfly species. In an e-mail to Mongabay-Wildtech, Alex Shepard, iNaturalist’s Mobile Applications Developer, emphasized the need for a large database of quality images to effectively teach the software the distinctive visual features of different species.

 

“Our image identification software (the feature in our app that we call “suggestions”) only works because of the photos and identifications provided by our community,” said Shepard. ”The more and better photos we have of a species, the better the software is at recognizing it. Conversely, without any photos to “train” on, the image identification software won’t work at all.“

Once a species has 20 ”research grade” observations (photos for which the ID has been confirmed by at least two iNaturalist users), it is added to the computer vision network. Currently, the software is able to identify over 10,000 species, with a new species reaching that threshold and being added to the network every 1.7 hours. The software suggests identifications with varying degrees of certainty, ranging from species or genus to “we’re not confident enough to make a recommendation.” When the software is unable to suggest a species, users can then tap into the collective knowledge of the iNaturalist community, who are generally able to provide a more specific ID.

User-designed projects

iNaturalist provides a platform for user-created Projects, which the website defines as “collections of observations with a common purpose.” One of the most common types of projects is a bioblitz – an event during which citizen scientists come together to document as many taxa as possible in a specific area. The United Nations recently sponsored a worldwide bioblitz for World Environment Day. Over 13 days, nearly 10,000 iNaturalist users from around the world made over 100,000 observations of 19,627 species. Another popular project documents roadkill sightings, while others help to track the spread of invasive species. Any iNaturalist user can create a Project with his or her own specific guidelines, such as limiting observations to specific locations or taxa.

Users are also able to create Guides that document and provide more extensive information on the taxa present in a specific area. Guides can cover an area as large as continent or provide detailed information on the organisms that frequent a local park.

Contributions to global biodiversity assessments

Loarie described the initial vision of iNaturalist as providing a platform “to connect people to nature through technology.” However, said Loarie, “as iNaturalist has grown, we’ve also realized that it’s an increasingly important tool for scaling scientific research. With iNaturalist, we strive to simultaneously engage and educate the public while also generating scalable streams of research quality data for science and conservation.”

iNaturalist has proven to be an effective tool for documenting Earth’s biodiversity. Loarie reported that in 2016, “over 90% of all North American, non-bird records uploaded to the Global Biodiversity Information Facility were generated from iNaturalist.” In addition, multiple species have been discovered or re-discovered using the iNaturalist app. For example, in 2014 a snail that had not been observed since the voyages of Captain Cook in the 1700s was rediscovered after a photo was uploaded by an iNaturalist user.

A recent update to the app allows users to add annotations for Life Stage, Plant Phenology and Sex to their observations. For some species, this has already resulted in robust Seasonality graphs that show clear patterns in the timing and locations that stages or sexes of individual species are seen.

Globally, biodiversity data are skewed geographically, with the bulk of observations representing North America and Europe, and across taxa — some species are simply easier to observe and found in locations that are easier to access. The data generated by the iNaturalist community are no exception and, as a result, this bias is also present in the new computer vision technology. According to Shepard, “one of the limitations of this approach is that the software isn’t great at recognizing species that our community hasn’t photographed and identified. If we want the image identification software to be a globally successful biodiversity monitoring tool, we need more photographs and more identifications of more species from around the world.”

Loarie added, “iNaturalist can only be used to collect occurrence or presence-only data but not presence-absence or abundance data. We’re currently working to better understand the implications of sampling biases in estimates of distribution, phenology, and relative abundance from these data. eBird is a great example of a citizen science platform that has managed to collect not just occurrences but also presence-absence and abundance data for birds.”

The importance of a growing iNaturalist community

Darwin’s finches are notoriously difficult to identify, with subtle variations in beak size and shape the only differences among most of the species. In addition, the birds have been shown to evolve rapidly in response to environmental changes. Still, for a visitor to the Galapagos islands equipped with a smartphone or camera, iNaturalist could provide some help– the small, medium and large ground finches have all crossed the magic 20 photo threshold and are in the computer vision network. For the other 10 finches, it’s likely that the iNaturalist community would be standing by to provide suggestions. Another option – consulting iNaturalist’s crowd-sourced place guide for the Galapagos.

The iNaturalist app’s new automatic ID feature is a useful tool for anyone interested in identifying and learning more about the organisms they see in their local environment. As more people consult the app while exploring their surroundings, its usefulness will continue to grow. In the words of Grant van Horn, one of iNaturalist’s collaborators at Visipedia, “The model that is shipped with the iNaturalist app is only as good as the data that the iNaturalist community generates. While not an explicit limitation, this relationship means that improved automated classifiers will only come about with an actively engaged community that continues to collect data and explore the world.”