STAT 453 | Schedule

Date	Lecture	Readings	Logistics
Module 1: Introduction and Foundations
9/3	Lecture #1 (Prof. Lengerich): Course Introduction [ slides \| notes ]
9/8	Lecture #2 (Prof. Lengerich): A Brief History of Deep Learning [ slides \| notes ]
9/10	Lecture #3 (Prof. Lengerich): Statistics, Linear Algebra, and Calculus Review [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 2
9/15	Lecture #4 (Prof. Lengerich): Single-layer networks [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 5
9/17	Lecture #5 (Prof. Lengerich): Parameter Optimization and Gradient Descent [ slides \| notes ]		HW1 due Fri 9/18
9/22	Lecture #6 (Prof. Lengerich): Automatic differentiation with PyTorch; Project Discussion [ slides \| notes ]
Module 2: Neural Networks
9/24	Lecture #7 (Prof. Lengerich): Multi-layer perceptrons and backpropagation [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 6
9/29	Lecture #8 (Prof. Lengerich): Structure and weight sharing; CNNs [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 9
10/1	Lecture #9 (Prof. Lengerich): Regularization [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 7	HW2 due Fri 10/2
10/6	Lecture #10 (Prof. Lengerich): Normalization and Initialization [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 11
10/8	Lecture #11 (Prof. Lengerich): Optimization and learning rates [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 8	Project proposal due Fri 10/9
10/13	Lecture #12 (Prof. Lengerich): Review [ slides \| notes ]
10/15	Midterm Exam (in-class)
Module 3: Intro to Generative Models
10/20	Lecture #13 (Prof. Lengerich): A linear introduction to generative models [ slides \| notes ]	A. Ng, On Discriminative vs. Generative Classifiers J. Xue, Comment on On Discriminative vs. Generative Classifiers	HW3 due Fri 10/23
10/22	Lecture #14 (Prof. Lengerich): Factor Analysis, Autoencoders, VAEs [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 13 Goodfellow et al., Deep learning book, Ch. 14 Kingma and Welling, Variational Autoencoders
10/27	Lecture #15 (Prof. Lengerich): Generative Adversarial Networks [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 20.9-10 Goodfellow et al., Generative Adversarial Nets Arora., Generative Adversarial Networks (GANs), Some Open Questions
10/29	Lecture #16 (Prof. Lengerich): Diffusion Models [ slides \| notes ]
Module 4: Sequence Models and Large Language Models
11/3	Lecture #17 (Prof. Lengerich): Sequence Learning with RNNs [ slides \| notes ]	Pascanu, Mikolov, Bengio, On the difficulty of training recurrent neural networks Goodfellow et al., Deep learning book, Ch. 10
11/5	Lecture #18 (Prof. Lengerich): Attention, Transformers [ slides \| notes ]	Vasvani et al., Attention is all you need Devlin et al., BERT - Pre-training of Deep Bidrectional Transformers for Language Understanding. Raschka, Build an LLM from Scratch 3 (video) Sanderson, Visualizing transformers and attention (video)	Project midway report due Fri 11/6
11/10	Lecture #19 (Prof. Lengerich): GPT Architectures [ slides \| notes ]	Radford et al., Improving Language Understanding by Generative Pre-Training (the GPT-1 paper) Radford et al., Language Models are Unsupervised Multitask Learners (the GPT-2 paper) Brown et al., Language Models are Few-Shot Learners (the GPT-3 paper) Raschka, Build an LLM from Scratch 4 (video) Karpathy, Let's Build GPT from Scratch (video)
11/12	Lecture #20 (Prof. Lengerich): Unsupervised Training of LLMs [ slides \| notes ]	Bi. et al, DeepSeek LLM Scaling Open-Source Language Models with Longtermism Liu et al., DeepSeekV2 A Strong, Economical, and Efficient Mixture-of-Experts Language Model Liu et al., DeepSeekV3 Technical Report	HW4 due Fri 11/13
11/17	Lecture #21 (Prof. Lengerich): Aligning LLMs: SFT, RLHF, and DPO [ slides \| notes ]	Raffel et al., Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Ouyang et al., Training language models to follow instructions with human feedback Rafailov et al., Direct Preference Optimization: Your Language Model is Secretly a Reward Model
11/19	Lecture #22 (Prof. Lengerich): Reasoning Models and RL from Verifiable Rewards [ slides \| notes ]	OpenAI, Learning to Reason with LLMs (the o1 report) Guo et al., DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
11/24	Lecture #23 (Prof. Lengerich): Parameter-Efficient Fine-tuning: LoRA, Adapters, and Prompting [ slides \| notes ]	Hu et al., LoRA: Low-Rank Adaptation of Large Language Models Li & Liang, Prefix-tuning - Optimizing continuous prompts for generation	HW5 due Fri 11/27
Module 5: Student Presentations & Wrap-up
12/1	Student Project Presentations
12/3	Student Project Presentations
12/8	Lecture #24 (Prof. Lengerich): Review & Open Directions [ slides \| notes ]	Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models	Project final report due Fri 12/11
12/11	Final Exam (exam period Dec 11-17; exact date/time TBD pending registrar block schedule)