SFU NLP class: Syllabus

Georges Artsrouni's mechanical brain, a translation device patented in 1933 in France.

Schedule

The schedule is preliminary and subject to change. Slides will be updated as the term progresses.

Week	Date	Topic	Assignments	Readings and Resources
1
	1/5	Lecture: Introduction (NLP histroy and basics of language modeling) [slides]		(optional) How Language Shapes the Way We Think. Lera Boroditsky. (optional) Meaning and Intelligence in Language Models (KDD 2025 Keynote). Chris Manning.
	1/6	Tutorial (optional): Review of a) probability, linear algebra and calculus and b) useful python/unix commands
	1/7	Lecture: Language Modeling [LM slides]		N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins.
	1/9	Tutorial (optional): Building language models
2
	1/12	Lecture: Text classification - Naive Bayes and evaluation [NB slides] [Evaluation slides]		Logistic Regression and Text Classification (J+M chapter 4)
	1/13	Tutorial (optional): Learning pytorch
	1/14	Lecture: Text classification - Logistic Regression [Logistic regression slides]	Due: HW0
	1/16	Tutorial (optional): Building classifiers with pytorch
3
	1/19	Lecture: NN review and word representations [WV slides 1]		Neural Networks (J+M chapter 6) (optional) Feedforward neural networks. Michael Collins. (optional) A Primer on Neural Network Models for Natural Language Processing. Yoav Goldberg.
	1/20	Tutorial (optional): Building text classifiers with word embeddings
	1/21	Lecture: Word Representations and neural language models [WV slides 2]		Embeddings (J+M chapter 4) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	1/23	Tutorial (optional): Building text classifiers with pretrained word embeddings
4
	1/26	Lecture: Sequence modeling (HMMs and RNNs) [slides] [slides]		Hidden Markov Models (J+M Appendix A.1-A.4)
	1/27	Tutorial (optional): Neural language models with BoW/Fixed Window FFN
	1/28	Lecture: Neural Sequence Modeling (LSTM/GRU) [slides]	Due: HW1	RNNs and LSTMs (J+M chapter 13) (optional) Understanding LSTM Networks. Christopher Olah.
	1/30	Tutorial (optional): Using RNNs in pytorch
5
	2/2	Lecture: Sequence generation, attention, and intro to transformers [sequence generation slides] [transformer slides]
	2/4	Lecture: Transformers [transformer slides]		Transformers (J+M chapter 8) (optional) Attention Is All You Need. Vaswani et al. 2017. (optional) The Annotated Transformer. Alexander Rush.
6
	2/9	Lecture: Pretraining LLMs [pretraining slides]		N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins. N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins.
	2/11	Lecture: Tasks, benchmarks and project information [project info slides] [benchmark slides]	Due: HW2
7
	2/16	No class - No class - Reading break
	2/19	No class - No class - Reading break
8
	2/23	Lecture: Using LLMs: Prompting and parameter efficient fine-tuning [prompting] [fine-tuning]	Due: Project proposal
	2/25	Lecture: Modern LLM architecture [slides]	Due: HW3
9
	3/2	Lecture: Posttraining - instruction tuning and preference alignment [instruction tuning] [preference alignment]
	3/4	Lecture: Posttraining - preference alignment and reasoning [slides]
10
	3/9	Lecture: Scaling laws and data [slides]
	3/11	Lecture: Final project tips, model debugging and analysis [slides]	Due: HW4
11
	3/16	Lecture: Parsing [constituency parsing] [Dependency parsing]		Constituency Grammars (J+M chapter 18) (optional) Statistical Constituency Parsing (J+M chapter Appendix C.1-C.4) Constituency Grammars (J+M chapter 18) (optional) Statistical Constituency Parsing (J+M chapter Appendix C.1-C.4)
	3/18	Lecture: Information retrieval and RAG [slides]
	3/19		Due: Project milestone
12
	3/23	Guest Lecture by Issam Laradji: Emerging directions in Large Language Models and AI agents
	3/25	Lecture: Guest Lecture (TBD)
13
	3/30	Lecture: Grounding and multimodal models - with talks from TAs (TA talks from Spring 2025 is currently linked) [slides] Visual grounding in 3D (Austin Wang) Text and 3D Representation Learning (Han-Hung Lee) Text-to-Scene Generation (Austin Wang) LLMs for modeling DNA (Chuanqi Tang)
	4/1	Lecture: Conclusion [slides]
14
	4/6	No class - No class - Easter Monday
	4/8	Lecture: Final project presentations
	4/9		Due: Final project report