Reference books
- There is no official textbook for the course, but if you would like to read further about NLP, here are some good reference books on NLP:
- Speech and Language Processing by Dan Jurafsky and James Martin.
- Natural Language Processing by Jacob Eisenstein
- A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg (see also Neural Network methods for Natural Language Processing).
- Foundations of Large Language Models by Tong Xiao and Jingbo Zhu
- To learn more about deep learning
- Dive into deep learning by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola
- Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- Understanding Deep Learning by Simon Prince
Debugging and learning about neural networks
- Neural Networks: Zero to Hero from Andrej Karpathy
- General tips from Andrej Karpathy for training neural networks
- Tips from Graham Neubig (CMU CS11-747) for debugging training of NLP models
Podcasts
- NLP highlights (from 2017 to 2023) with Matt Gardner, Pradeep Dasigi, and Waleed Ammar
Learning about LLMs and transformers
- 3Blue1Brown’s videos on neural networks and LLMs
- Andrej Karparthy’s videos on LLMs
- Jay Alammar’s illustrated guides
- Borealis tutorials
Tools
- Deep learning for NLP
- Sequence models
- Static Word Embeddings
- Pretrained language models
- HuggingFace course on LLMs
- HuggingFace transformers Docummentation for various transformer based models from HuggingFace
- OpenAI API
- Llama
- HuggingFace course on LLMs
- Using LLMs
- Prompting Guide
- LangChain / LangGraph for building LLM agents
- DSPy for more flexible pipelines
- NLP pipelines
Tasks and datasets
A list of shared task datasets are provided below. In some cases you can also extend your homework code to produce innovative project ideas for these tasks.
Shared Task Collections
- Torchtext
- Sebastian Ruder’s curated collection
- Datasets for Natural Language Processing
- Kaggle NLP Tasks
Embeddings
Language models
Multitask Benchmarks
Question Answering
- SQuAD
- CoQA
- HotpotQA
- Natural Questions
- Qanta shared task
- Reverse QA: Jeopardy style QA. json and csv
Math questions
Commonsense reasoning
CoNLL Shared Tasks
- CoNLL Shared Tasks
- CoNLL 2003 Named Entity Recognition Task
- CoNLL 2000 Chunking Task
- CoNLL 2018 Multilingual Parsing
SemEval Shared Tasks
- SemEval 2025
- SemEval 2024
- SemEval 2023
- SemEval 2022
- SemEval 2021
- SemEval 2020
- SemEval 2019
- SemEval 2018
- SemEval 2017
- SemEval 2016
- SemEval 2015
- SemEval 2014
Classification Tasks
Information Extraction
- Drug-Drug Interaction Extraction
- Web named entities
- Twitter sequence prediction tasks
- WNUT Emerging and Rare entity recognition shared task
- Gun violence text data
Parsing
Machine Translation
- WMT 2025 Shared Task
- WMT 2024 Shared Task
- WMT 2023 Shared Task
- WMT 2022 Shared Task
- WMT 2021 Shared Task
- WMT 2020 Shared Task
- WMT 2019 Shared Task
- WMT 2018 Shared Task
- WMT 2017 Shared Task
- WMT 2016 Shared Task
- WMT 2015 Shared Task
- NMT 2018 Neural MT Shared Task
- Web Inventory of Transcribed and Translated Talks