Natural Language Processing

• Stanza is a Python NLP package developed by the Stanford NLP Group
• Since many of you have interests in Russian literature, here is a set of interesting resources
• Speech and Language Processing is a book by Dan Jurafsky and James Martin with a free online version
• The CS224n: Natural Language Processing with Deep Learning Stanford University course has many resources available on its site

Click below to open the presentation.
Once in the presentation, navigate the slides with the left and right arrows and refresh any page to show the link that will take you back here.

Let's go through an example of NLP using fastai. I am using here one of their demo examples.

Below is the code that I will run. Because we only have one GPU and because it would take an extremely long time to run this code on CPUs, unfortunately, you will have to watch and you won't be able to run this on your JupyterLab.

# First, we load the necessary functions from fastai
# Note that import * is appropriate with fastai
# but not a very good practice with other packages
from fastai.text.all import *

# Then we build our data loaders
# This is an object of the fastai class DataLoaders
# It contains a validation data loader and a training data loader
dls = TextDataLoaders.from_folder(
    '/project/def-sponsor00/dhsi/.fastai/data/imdb',
    valid='test',
    bs=16
)

# Let's have a look at the data
dls.show_batch()

# Now, we define our learner
# AWD_LSTM is the architecture (https://arxiv.org/abs/1708.02182)
learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, metrics=accuracy)

# Finally, we fine-tune the pretrained model with our data
# over 2 epochs with a learning rate of 1e-2
learn.fine_tune(2, 1e-2)

# Then we can use our model to predict whether sentences are positive or negative
learn.predict("This is amazing.")
learn.predict("I love it.")
learn.predict("This is terrible.")

If you want to really understand what neural networks are and how they work, I really recommend you to watch the 4 videos below from 3Blue1Brown by Grant Sanderson . They are easy to watch, fun, and do an excellent job at introducing the functioning of a simple neural network.