Sensor Aware Machine Learning for Edge Devices

Open Access
Kim, Dong Hyun
Area of Honors:
Computer Engineering
Bachelor of Science
Document Type:
Thesis Supervisors:
  • Vijaykrishnan Narayanan, Thesis Supervisor
  • Vijaykrishnan Narayanan, Honors Advisor
  • Kyusun Choi, Faculty Reader
  • Convolutional Neural Networks
  • Unity 3D
  • Synthetic Data
  • MobileNet
Neural networks have produced breakthroughs in numerous domains, and many owe their success to the availability of large, labeled datasets. These datasets help us solve the problems for which they were designed, but are ineffective at yielding solutions to problems that differ drastically in context (e.g. daytime versus nighttime images), even if the underlying task (e.g. species recognition) is very similar. Even for existing tasks, deployment conditions can vary so much that more labeled samples are needed. For instance, even if prior efforts were sufficiently forward-looking to expect that determining the species of an animal likely requires images of the animal at different times of the day, they may not have also considered indoor versus outdoor conditions in the training set that would impact zoo versus domestic versus in-nature classification rates. Despite an increasing number of public datasets, there are always more to be desired, namely more labeled datasets on which to learn for new tasks that are not yet popular enough to have justified the manual efforts required in current labeling approaches. In this thesis, we explore training on datasets particular to environmental factors, specifically different lighting conditions, in a way that minimizes human effort in labeling. We explore synthesizing the dataset as a mean to circumvent limitations of manually labeling and collecting samples under a larger range of potential environments. We consider how context aware datasets might produce models for achieving best classification accuracy under different deployment conditions, and how the correct model to use can be predicted by an endpoint device. By comparing models on datasets that differ only in lighting conditions, we conclude that lighting conditions contribute to the model accuracy rate and that further exploration of sensor aware learning is warranted.