Observability for LLM Applications: Tracing, Tokens, and Feedback Loops
What to actually instrument in an LLM application beyond latency and error rate, and how to close the loop from production traffic to eval sets.
Tag
8 articles
What to actually instrument in an LLM application beyond latency and error rate, and how to close the loop from production traffic to eval sets.
How to build an eval suite that catches LLM application regressions before deploy, from labeled datasets to LLM-as-judge pitfalls.
Why average response time hides the experience of your worst-served users, and how to measure, diagnose, and reduce tail latency in production systems.
How to set service level objectives and error budgets that drive real engineering decisions, instead of becoming a number nobody looks at after launch.
Logs, metrics, and traces are usually taught as separate pillars, but the real value comes from how they connect during an actual incident investigation.
How Uber built M3, its open-sourced metrics platform, when off-the-shelf time-series tooling couldn't handle its monitoring scale.
How Micrometer and OpenTelemetry fit together in a Spring Boot service, and which metrics and traces are worth instrumenting by hand versus for free.
Which Spring Boot Actuator endpoints earn their place in production, and the security configuration that keeps them from becoming an attack surface.