Problems-as-a-Service: a GenAI troubleshooting demo

I’ve been demoing Observability-for-AI quite a lot recently, and searching around for an OTel-demo equvalent; an app that produces gen-AI spans and ideally has some problems to find and solve.

In a similar vein to Tiny-telemetry, I’ve made my own and in a fit of originality I’ve called it Buggy AI App.

It’s a small RAG app, just complex enough to have spans, that I wrote for demoing Coralogix’s AI Center.

It has a list of questions, and a gateway hands each question to a rewriter, a retriever and a generator, and every step sends OpenTelemetry traces using the GenAI conventions.

Out of the box it’s perfectly healthy and you can just see the flow, inspect the spans and inspect the app behaviour from Coralogix.

If you turn on one of the numbered scenarios it starts having problems, and then you get to find out what by looking at the telemetry. They’re numbered rather than named so the names don’t give the game away; you can run this in an interactive session where you encourage participants to use observability to investigate and find the problems.

You can run it as a group session with someone holding the answers, or on your own with the spoiler-free hints. The LLM calls are mocked by default so it doesn’t cost any tokens - the point here isn’t really the AI itself, but the gen-AI spans

You just need to docker-compose up and then kick-off script that keeps asking it questions until you ctrl-c it.

The README has a lot more detail than this, there are hints to solve the problems in the participant notes, and the bugs are detailled in the facilitator notes, which you shouldn’t read first.