L02 Brief: How LLMs Code (Under the Hood)
EECS AASE Podcast, season 1, episode 2

Transcript of the published audio. The episode is AI-generated from
course materials; this transcript is produced locally from the audio.

This is the brief on Lecture 2 of EECS 498 AASE. Lecture 2 literally rips
open the black box of how an LLM actually writes code, proving that
understanding the underlying mechanics is the difference between blindly
retyping a failed prompt and surgically debugging a bad answer.

First, we're breaking down tokenization and the context window. You know,
tokens are basically a model's true currency and attention span. Imagine
trying to hold 20 conflicting thoughts in your head at once. It's chaos,
right? Well, that's kind of what happens when a model's context gets
totally bloated. It really makes you wonder, why do long conversations
suddenly just fall apart? And it's exactly why tools like Aider actually
force you to aggressively manage what the model sees by typing /add.

Second, when that context window hits its absolute limit, the model just
starts guessing. Yep, that leads straight to hallucination. Since it's
just predicting the next word using autoregressive sampling, it
completely lacks a fact-checker, which results in confidently invented
APIs. So you've got to ask yourself, what can tweaking your context
actually fix? And what's just fundamentally beyond its reach?

Finally, we're looking past output errors to contrast a model's raw
capability against its stored knowledge. There's a massive difference
between actual reasoning skills and static training data. If you assume
bigger is always better, think again. Counterintuitively, a smaller 4
billion parameter model actually makes for a way better teacher.

All these theoretical mechanics pay off big time in lecture three,
collapsing into a powerful diagnostic framework using context, model, and
prompt as your three levers. Mastering these mechanics is exactly how you
stop guessing and start engineering.