Notes / 04
Engineering notes
Short records of what broke, what I misunderstood first and the decisions that made the system clearer.
One LLM Score Was Hiding Two Different Problems
I stopped evaluating the RAG pipeline as one thing and started asking separate questions about retrieval and generation.
pip install Is a Trust Decision
AI projects accumulate providers, SDKs, agents, vector stores, and secrets quickly. I started treating every new dependency as part of the security model.
terraform destroy Finished. I Still Checked the Bill.
Terraform removed everything it owned. That was not the same thing as removing everything that could still cost money.
A Pod Can Be Running and Still Be Going Nowhere
LiteLLM worked locally, then exposed exactly how little room was left on my e2-small GKE nodes.
The Trace Was Correct. The Story Wasn't.
What happened when LangChain knew the agent, LiteLLM knew the model, and Langfuse showed both as separate stories.