Nonlinear Neuron Discriminant Functions for Alternate Deep Learning Training Algorithms
Open Access
Author:
Petrone, Steven
Area of Honors:
Computer Engineering
Degree:
Bachelor of Science
Document Type:
Thesis
Thesis Supervisors:
Christopher H Griffin, Thesis Supervisor Vijaykrishnan Narayanan, Thesis Honors Advisor
Keywords:
deep learning neural network posynomial geometric programming c++ signomial
Abstract:
In this work we present a novel neuron discriminant function that allows for alternate training
algorithms for deep learning. The new neuron type, which we call a posynomial neuron, can be
combined with linear neurons to represent functions that are exponentials when inferencing new
data, but are only polynomials of the network weights. We show that the properties of these net-
works can be resistant to the vanishing gradient problem. We also formulate training the network as
a geometric programming problem and discuss the interesting benefits this can have over training
a network with gradient descent, such as data set analysis and network interpretability. We provide
a C++ library that implements both posynomial and sigmoidal networks but provides flexibility for
additional novel layer types. We also provide a tensor library that has applications beyond deep
learning.
