Introduction to Large Language Models (LLMs)
Learn how large language models work: language modeling, Transformers in PyTorch, pre-training, prompting, alignment, retrieval and efficient tuning.
Large language models sit on a stack of earlier ideas about modeling text, and the lectures build that stack in order. Statistical language models and deep learning basics come first, then word representations, neural language models, sequence-to-sequence models and attention. You implement a Transformer in PyTorch, compare pre-training strategies, and move on to prompting, alignment with human feedback, knowledge retrieval and parameter-efficient fine-tuning.
What You Will Achieve
- Build and evaluate statistical and neural language models
- Implement a Transformer in PyTorch and explain self-attention and positional encoding
- Compare instruction tuning, prompting and parameter-efficient fine-tuning as ways to adapt a pre-trained model
- Recognize hallucination, bias and toxicity risks in LLM output
Program Overview
Language Models Before Transformers
The statistical and neural groundwork that modern LLMs build on.
- NLP pipeline and statistical language models
- Deep learning and PyTorch basics
- Word representation and tokenization strategies
- Neural and sequence-to-sequence models with attention
Transformers, Pre-Training and Prompting
How today’s models are built, pre-trained and steered.
- Self and multi-head attention, positional encoding
- Implementing a Transformer in PyTorch
- Encoder-only, encoder-decoder and decoder-only pre-training
- Instruction tuning, prompting and alignment with human feedback
Knowledge, Efficiency and Responsible LLMs
Research directions that make models better informed, cheaper and safer.
- Knowledge graphs for retrieval
- Parameter-efficient fine-tuning, quantization, pruning and distillation
- Interpretability and the residual stream view
- Bias, toxicity and responsible LLMs
Certificate Awarded (Paid Exam)
Pass the proctored exam to earn it. The course is free to study; the certificate requires a paid, proctored exam.
Job Outlook
Language model skills are used in NLP research, machine learning engineering, and teams that build search, question answering and other text applications on pre-trained models.
Good for: Undergraduate and postgraduate students in computer science, electronics, IT or mathematics who want to understand how LLMs are built and studied.
Keep in mind: The final week surveys named models such as GPT-4 and Llama-3, which newer releases will overtake; the weeks on how models work date far less.
Bottom line: A research-oriented account of how LLMs work, with a Transformer you code yourself and a closing look at alignment and responsible use.
Frequently Asked Questions
Do I write code in this course?
Yes. PyTorch is introduced early and you implement a Transformer with it; HuggingFace is introduced alongside the pre-training lectures.
Does the course cover alignment and RLHF?
Yes. Alignment of language models, including alignment with human feedback (RLHF), follows the lectures on instruction tuning and prompting.
Which model compression methods are taught?
Quantization, pruning and distillation, in the same week as parameter-efficient fine-tuning methods such as LoRA.
Which word representation methods are taught?
Word2Vec, fastText and GloVe, followed by tokenization strategies, before the lectures turn to neural language models.
User Reviews
Write your review
You must be logged in to post a review.
Reviews can only be posted by registered learners. Log in or create a free account, then you will return here to post.







































There are no reviews yet.