Neural Network Dependency Parser
Completion: October 2023
Training:
For input, the model is trained on a series of input pairs (x,y), where x is a parser state (consisting of a stack and a buffer), and y is the transition the parser should make (e.g. left-arc, right-arc, shift). These input pairs will be made using one hot representations for words based on its Part of Speech (POS) tag within each state. The training program also contains outputs pairs of (transitions, label), where a label is a dependency label. These outputs are also represented as one hot vectors to be fed into the neural network.
The Neural Network is created using Keras, an API for Tensorflow.
Network Composition:
The Neural Network consists of:
* One Embedding layer, the input_dimension being the number possible words, input_length is the number of words using this same embedding layer.
* The output_dim of the embedding layer is 32, which is then flattened.
* A Dense hidden layer of 100 units using relu activation.
* A Dense hidden layer of 10 units using relu activation.
* An output layer using softmax activation.
Parser Creation:
The standard transition-based algorithm is used to create a parser, using the trained and stored model to predict transitions. As long as the buffer is not empty, the feature extractor obtains a representation of the current state and the call model.predict(features) and retrieve a softmax actived vector of possible actions.
Tags: Natural Language Processing Python Neural Network TensorFlow Keras Columbia