Reference books

• There is no official textbook for the course, but if you would like to read further about NLP, here are some good reference books on NLP:
  • Speech and Language Processing by Dan Jurafsky and James Martin.
  • Natural Language Processing by Jacob Eisenstein
  • A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg (see also Neural Network methods for Natural Language Processing).
• To learn more about deep learning
  • Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
  • Dive into deep learning by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola

Debugging Neural Networks

• General tips from Andrej Karpathy for training neural networks
• Tips from Graham Neubig (CMU CS11-747) for debugging NLP models

Podcasts

• NLP highlights with Matt Gardner, Pradeep Dasigi, and Waleed Ammar

Tools

• Here are some useful NLP tools
  • AllenNLP [demo] Modern NLP tools using Pytorch from AI2
  • Spacy.io [demo] Popular python based NLP pipeline
  • Stanza [demo] Python NLP library from Stanford
  • Stanford CoreNLP [demo] Java NLP pipeline from Stanford
  • NLTK [demo] [book] Useful basic Python tools for NLP
• Deep learning for NLP
  • TorchText
• Sequence models
  • FairSeq
• Static Word Embeddings
  • Word2Vec
  • GloVe
  • FastText
• Pretrained language models
  • HuggingFace course on transformers
    • HuggingFace transformers Docummentation for various transformer based models from HuggingFace
  • OpenAI API
  • Llama
• Machine Translation
  • SFUTranslate SFU NLL’s MT toolkit (designed to be modern and flexible)
  • JoeyNMT Minimalist system for understanding NMT
  • OpenNMT Comprehensive NMT framework