4 Dispatches from the engineering frontier — Deep dives, insights, and explorations
Retrieval-Augmented Generation (RAG) is prone to hallucinations when the retrieved context is irrelevant or noisy. I’ve spent the last few months moving away from standard "naiv...
Latency is the silent killer of LLM-powered products. When I’m building production apps, I find that waiting for a 70B parameter model to stream tokens at 15 tokens per second f...
Most LLM-based agents suffer from a classic case of amnesia. Once the context window hits its limit or the session resets, your agent loses everything it learned about the user’...
Building LLM-powered applications is fun until you hit the "jailbreak" wall. I remember my first production chat deployment—everything worked perfectly until a user started test...