Leveraging Graph Neural Networks for Efficient Word Representations
Open Access
Author:
Himes, Ryan
Area of Honors:
Computer Science
Degree:
Bachelor of Science
Document Type:
Thesis
Thesis Supervisors:
Truong Xuan Tran, Thesis Supervisor Ting He, Thesis Honors Advisor
Keywords:
Graph Neural Networks NLP Machine Learning Word Embeddings Text Classification
Abstract:
Sentence structure can consist of a complex structure of tokens and relationships between tokens which can be hard to represent with only a sequential representation. To capture more information about sentence structure we propose representing a sentence as a graph of tokens and relationships between tokens to learn dynamic word embeddings. The graph structure is used as input for a graph neural network (GNN) to learn syntactic and semantic information about the sentence by learning a next word prediction task to learn the embeddings. We also experiment with using the graph structure as input for different natural language classification tasks. Results show that using the GNN ARMAConv on natural language classification tasks can increase accuracy over a sequential representation.
Also, using the newly trained dynamic word embeddings achieves a higher validation accuracy compared to all other word embeddings tested on the tweet disaster classification data set.
