Part 1 covered how the Bank of Korea makes its rate decision and which instruments it uses to hold the base rate in place. This post is about what happens next. How does a change in the overnight r...

Eight times a year, seven people vote on one number. The Bank of Korea’s base rate anchors the cost of overnight interbank borrowing, and from there it spreads into mortgage rates, corporate bond y...

GPT kept the decoder and learned to generate. BERT kept the encoder and learned to read in both directions. This post covers why bidirectionality matters, how BERT’s training objectives work, and w...

Part 3 ended with a question: GPT-3 is capable, so why isn’t it ChatGPT? The answer is alignment, the gap between a model that predicts likely text and one that follows what people actually want. T...

Part 2 ended with the full Transformer: an encoder and a decoder connected by cross-attention, built for sequence-to-sequence tasks like translation. GPT asked what happens if you keep only the dec...

In Part 1, we derived Cross-Entropy Loss from first principles. The next question is how to actually minimize it. This post walks through the training loop step by step: Forward Propagation, Backpr...

Part 1 ended with a question: what if the decoder could look at any part of the input directly, instead of routing everything through one compressed vector? This post follows the answer from the fi...

Before Transformers, researchers spent years working out why sequences are harder than other kinds of data. This post follows that path from the simplest recurrent network to the point where Seq2Se...