STAT 453 | Schedule

Date	Lecture	Readings	Logistics
Module 1: Introduction and Foundations
9/3	Lecture #1 (Prof. Lengerich): Course Introduction, Introduction to DL [ slides \| notes ]
9/8	Lecture #2 (Prof. Lengerich): A Brief History of DL [ slides \| notes ]		HW1 Out
9/10	Lecture #3 (Prof. Lengerich): Statistics / linear algebra / calculus review [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 2
9/15	Lecture #4 (Prof. Lengerich): Single-layer networks [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 5
9/17	Lecture #5 (Prof. Lengerich): Parameter Optimization and Gradient Descent [ slides \| notes ]
9/22	Lecture #6 (Prof. Lengerich): Automatic differentiation with PyTorch [ slides \| notes ]
9/24	Lecture #7 (Prof. Lengerich): Cluster and cloud computing resources [ slides \| notes ]		HW2 Out
Module 2: Neural Networks
9/29	Lecture #8 (Prof. Lengerich): Multinomial logistic regression [ slides \| notes ]
10/1	Lecture #9 (Prof. Lengerich): Multi-layer perceptrons and backpropagation [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 6
10/6	Lecture #10 (Prof. Lengerich): Regularization [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 7	HW3 Out
10/8	Lecture #11 (Prof. Lengerich): Normalization / Initialization [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 11
10/13	Lecture #12 (Prof. Lengerich): Optimization, Learning Rates [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 8
10/15	Lecture #13 (Prof. Lengerich): CNNs [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 9	Project Proposal Due
10/20	Lecture #14 (Prof. Lengerich): Review [ slides \| notes ]
10/22	Midterm Exam
Module 3: Intro to Generative Models
10/27	Lecture #15 (Prof. Lengerich): A Linear Intro to Generative Models [ slides \| notes ]	A. Ng, On Discriminative vs. Generative Classifiers J. Xue, Comment on On Discriminative vs. Generative Classifiers
10/29	Lecture #16 (Prof. Lengerich): Factor Analysis, Autoencoders, VAEs [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 13 Goodfellow et al., Deep learning book, Ch. 14 Kingma and Welling, Variational Autoencoders
11/3	Lecture #17 (Prof. Lengerich): Generative Adversarial Networks [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 20.9-10 Goodfellow et al., Generative Adversarial Nets Arora., Generative Adversarial Networks (GANs), Some Open Questions
11/5	Lecture #18 (Prof. Lengerich): Diffusion Models [ slides \| notes ]		Project Midway Report Due
Module 4: Large Language Models
11/10	Lecture #19 (Prof. Lengerich): Sequence Learning with RNNs [ slides \| notes ]	Pascanu, Mikolov, Bengio, On the difficulty of training recurrent neural networks Goodfellow et al., Deep learning book, Ch. 10	HW4 Out
11/12	Lecture #20 (Prof. Lengerich): Attention, Transformers [ slides \| notes ]	Vasvani et al., Attention is all you need Devlin et al., BERT - Pre-training of Deep Bidrectional Transformers for Language Understanding. Raschka, Build an LLM from Scratch 3 (video) Sanderson, Visualizing transformers and attention (video)
11/17	Lecture #21 (Prof. Lengerich): GPT Architectures [ slides \| notes ]	Radford et al., Improving Language Understanding by Generative Pre-Training (the GPT-1 paper) Radford et al., Language Models are Unsupervised Multitask Learners (the GPT-2 paper) Brown et al., Language Models are Few-Shot Learners (the GPT-3 paper) Raschka, Build an LLM from Scratch 4 (video) Karpathy, Let's Build GPT from Scratch (video)
11/19	Lecture #22 (Prof. Lengerich): Unsupervised Training of LLMs [ slides \| notes ]	Bi. et al, DeepSeek LLM Scaling Open-Source Language Models with Longtermism Liu et al., DeepSeekV2 A Strong, Economical, and Efficient Mixture-of-Experts Language Model Liu et al., DeepSeekV3 Technical Report
11/24	Lecture #23 (Prof. Lengerich): Supervised Fine-tuning of LLMs [ slides \| notes ]	Raffel et al., Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Ouyang et al., Training language models to follow instructions with human feedback	HW5 Out
11/26	Lecture #24 (Prof. Lengerich): Prompts and In-context learning [ slides \| notes ]	Li & Liang, Prefix-tuning - Optimizing continuous prompts for generation
12/1	Lecture #25 (Prof. Lengerich): Foundation models, alignment, explainability [ slides \| notes ]
12/3	Lecture #26 (Prof. Lengerich): Open directions in LLM research [ slides \| notes ]
Module 5: Student Presentations
12/8	Project Presentations
12/10	Project Presentations
12/17	Final Exam