Visual Scan Path Feature Based Image Emotion Analysis

Open Access
Costa, Michael Jerrick
Area of Honors:
Computer Science
Bachelor of Science
Document Type:
Thesis Supervisors:
  • James Z Wang, Thesis Supervisor
  • John Joseph Hannan, Honors Advisor
  • images
  • emotion analysis
  • scan path
  • machine learning
  • computer vision
This research project proposes a novel set of features for use in emotional analysis of images. These features are based on visual scan paths generated by converting an image to a saliency map using a saliency algorithm and processing that saliency map through an attentive shift model. Given a set of points that make up a scan path, one can produce features that characterize the nature of the path such as length, angle, circular variance, and complexity. It is expected that these features will be able to encode higher level spatial structure than traditional spatial features. This thesis details the construction of a dataset for analyzing the effects of visual stimuli on emotional response that was used to test these scan path features. This research project also investigates the effects of using varying types of saliency models and attentive shift models as well as the number of points used to compute features.