Lectures
Important notes:
- We will upload lectures prior to their corresponding classes.
- [SIT770]: Indicates that the content provided is specifically tailored for students currently enrolled in SIT770.
-
Week 0: Course Overview
Summary: Introduction and course overview.
[slides] [slides 6up]
Video recordings (23 Minutes and 19 Seconds):
-
Week 1: Information Retrieval Part 1
Summary: Inverted indices, scoring, term weighting, and the vector space model.
[slides] [slides 6up]
Video recordings (1 Hour, 59 Minutes and 23 Seconds):
- Introduction to Information Retrieval (6:32)
- Term-document incidence matrices (7:10)
- The Inverted Index, the key data structure underlying modern IR (9:52)
- Query processing with an inverted index (5:23)
- Structured vs. Unstructured Data (3:31)
- Modeling in Information Retrieval (4:37)
- The Boolean Model (10:28)
- Phrase queries and positional indexes (10:02)
- The Vector Model (1:28)
- Ranked retrieval (6:08)
- Scoring documents (7:41)
- Term frequency (9:29)
- Collection Statistics (12:39)
- Weighting schemes (4:25)
- Vector space scoring (19:59)
-
Week 2: Information Retrieval Part 2
Summary: Probabilistic IR and Evaluation methods.
[slides] [slides 6up]
Video recordings (1 Hour, 59 Minutes and 22 Seconds):
- Probabilistic IR model (1 Hour, 08 Minutes and 39 Seconds):
- IR Evaluation methods (50 Minutes and 43 Seconds):
-
Week 3: Text processing
Summary: Regular Expressions, Text Normalization, Edit Distance.
[slides] [slides 6up]
Video recordings (1 Hour, 54 Minutes and 32 Seconds):
- Regular Expressions (28 Minutes and 32 Seconds):
- Text Normalization (37 Minutes and 11 Seconds):
- [SIT770] Edit Distance (48 Minutes and 49 Seconds):
-
Week 4: N-gram Language Models
Summary: N-gram Language Models.
[slides] [slides 6up]
Video recordings (2 Hours, 01 Minutes and 23 Seconds):
- Language Models (1 Hour, 14 Minutes and 31 Seconds):
- Spelling Correction and the Noisy Channel (46 Minutes and 52 Seconds):
-
Week 5: Vector Embeddings and Sequence Labeling
Summary: Vector Embeddings and Sequence Labeling.
[slides] [slides 6up]
Video recordings (2 Hour, 23 Minutes and 58 Seconds):
- Vector Embeddings (1 Hour, 12 Minutes and 00 Seconds):
- Sequence Labeling (1 Hour, 11 Minutes and 58 Seconds):
-
Week 6: Neural Networks for NLP
Summary: Neural Networks for NLP.
[slides] [slides 6up] [Notes]
Video recordings:
- Introduction (1:01)
- Optional: Introduction to Neural Nets (1 Hour, 13 Minutes and 06 Seconds):
- Neural Networks Overview (4:26)
- Neural Network Representation (5:14)
- Computing a Neural Network’s Output (9:57)
- Vectorizing across multiple examples (9:05)
- Explanation for Vectorized Implementation (7:37)
- Activation functions (10:56)
- Derivatives of activation functions (7:57)
- Gradient descent for Neural Networks (9:57)
- Random Initialization (7:57)
- Applying feedforward networks to NLP tasks (15:32)
- Recurrent Neural Networks (2 Hours, 34 Minutes and 11 Seconds):
- Why sequence models? (3:00)
- Notation (9:15)
- Recurrent Neural Network Model (16:31)
- Different types of RNNs (8:33)
- Language model and sequence generation (12:01)
- Sampling novel sequences (8:38)
- Vanishing gradients with RNNs (6:28)
- Gated Recurrent Unit (GRU) (17:06)
- Long Short Term Memory (LSTM) (9:53)
- Bidirectional RNN (8:19)
- Deep RNNs (5:16)
- Basic Models (6:18)
- Picking the most likely sentence (8:56)
- Beam Search (11:54)
- Attention Model Intuition (9:41)
- Attention Model (12:22)
-
Week 7: Transformers and Pretrained LMs
Summary: Transformers and Pretrained LMs.
[slides] [slides 6up]
Video recordings (2 Hours, 05 Minutes and 12 Seconds):
- Transformers: Attention Is All You Need! (1 Hour, 10 Minutes and 27 Seconds)
- Pre-trained LMs (54 Minutes and 45 Seconds)
-
Week 8: Large Language Models
Summary: Large Language Models.
[slides] [slides 6up]
Video recordings (1 Hours, 08 Minutes and 32 Seconds):
- Introduction to Large Language Models (6:14)
- Large Language Models: What tasks can they do? (7:46)
- Sampling for LLM Generation (9:32)
- Pretraining Large Language Models: Algorithm (5:43)
- Pretraining data for LLMs (6:35)
- Finetuning (2:46)
- Evaluating Large Language Models (3:56)
- Dealing with Scale (16:03)
- Harms of Large Language Models (9:57)
-
Week 9: Speech Processing & ASR
Summary: Speech Processing & ASR.
[slides] [slides 6up]
Video recordings (48 Minutes and 36 Seconds):
-
Week 10: Dialogue Systems & Conversational AI
Summary: Dialogue Systems & Conversational AI.
[slides] [slides 6up]
Video recordings (1 hours, 40 Minutes and 48 Seconds):
- Introduction to Chatbots and Dialogue Systems (5:40)
- Properties of Human Conversation (17:07)
- Rule-based Chatbots - ELIZA and PARRY (12:58)
- Corpus-based Chatbots (14:51)
- The Frame-based (“GUS”) Dialogue Architecture (10:45)
- The Dialogue-State Architecture (11:13)
- The Dialogue-State Architecture Continued - Policy and Generation (13:21)
- Evaluating Dialogue Systems (8:22)
- Design and Ethical Issues (6:31)