Backyard Birder

I live in an area that is visited by many a variety of birds (and squirrels). I thought it wold be interesting to explore the abilities of Amazon's DeepLens to be able to identify bird species by both visual and song characteristics as well as keep track of how many squirrels disrupt the feeders each day.

I had a difficult time finding good seed image libraries for bird species and wanted to get my hands dirty with this new hardware. I thought I could leverage the ease of getting a sample project running on DeepLens to do a course collection of bird images that I could further develop into my species identification model.

The AWS DeepLens hardware allows locally running image processing to occur using models trained using a variety of machine learning methodologies. The graphics processing hardware allows for low latency video stream processing at the edge (i.e. disconnected from the cloud).

My application, using an included sample MXNet trained model (deeplens-object-detection), identifies the visitors to my bird feeders in my yard. The sample model is able to identify birds with reasonable accuracy. There is no classification for squirrels in this model but they score relatively likely against the "cat" and "dog" classifier (this is a hackathon, isn't it?). Upon a high identification probability score of recognizing a bird or a squirrel, a message is published to a message topic (MQTT) where a lister process will tally the counts and store them to a database where a daily squirrel vs. bird scorecard can be accessed via the web.

Created By: Paul Langdon

Initially I utilized a pre-trained model, but this is the first step in a broader project that is a "train the trainer" application. Using existing coarse models as the source for generating new seed images to train a new model using AWS SageMaker, in this case, to specifically identify bird species.

Check out this video of the detection: youtube

This project will give you some information about the DeepLens framework, hardware and a references to a guide for setting up your own device and customizing a model to do your own detection.

I encourage you to use it as a starting point to exploring this platform. The code to this project can be found on github at https://github.com/plangdon/deeplens_birds.

The architecture of this solution can be adapted to support your trained model and notification/action pipeline.

Solution Architecture
In designing this architecture, I had the following goals:

• Push as much processing and resource utilization to "the edge"
  • video bandwidth is large, sending video stream to cloud for processing would be expensive and have high latency
  • notifications or counts are orders of magnitude smaller than video and easy sent to cloud for storage and action
  • constant video processing in the cloud would incur lots of CPU costs
• Support image capture locally so device could be deployed to remote location without 100% connectivity or on low-bandwidth networks (cellular)
  
  • images captured of birds would be incorporated into future training for specific species identification
  • locally stored images can be off-loaded from device on demand when network is enabled or physically removed via removable media (sneaker-net)
    
• Support notifications to the cloud using secure, messaging protocol