AIEN
One AI reading another's answer
What I brought back from TDC in Florianópolis — AI as a Judge and the shape of a judge step in a chatbot flow, harness as the word of the day, and an observability workshop that ended up answering a product question back at work.
An AI reading the answer another AI generated, to say whether it actually answers the question. That's what I brought back from TDC in Florianópolis.
I was there on July 24th, with a ticket my tech lead Giovani Valgas got me — thanks, Giovani. The schedule was dominated by AI: AI First, harness, model evaluation. Three things from that day stayed with me, and one of them ended up in my own work.
AI as a Judge
Livia Rodrigues Fernandes Silva, from Oracle, gave a talk on AI as a Judge. The idea is simple: one model reads another model's output and evaluates it. Does it answer what the customer asked? Does it respect the guardrails?
She showed several examples of evaluator prompts — prompts whose only job is to grade an answer, not to produce one. And she brought numbers: with a judge in the loop, answer quality improved in 90% of the cases, meaning the customer's question actually got resolved more easily.
The value isn't in the idea, it's in doing it systematically. There are a lot of support chatbots running today with nobody looking at what they reply. A judge step is a lot of return for very little engineering.
The talk didn't go into tooling, so I went looking afterwards. For my scenario, the tool that fit best was n8n. But the tool matters less than the shape: any chatbot flow that needs an LLM as a judge ends up looking roughly the same.
- The customer asks something.
- The assistant model generates an answer.
- Before that answer reaches anyone, a second model receives the question, the answer and the rules, and returns a verdict.
- The verdict decides what happens next.
Everything that isn't the LLM
The other word that came up all day was harness. The cleanest definition I heard: everything you give an AI that isn't the model itself, and that makes it produce the best possible result. Plugins, MCPs, skills, prompts, the tooling around it.
Put shortly: everything that isn't the LLM is harness.
Observability, hands-on
I also took Ariane Izac's observability workshop, run inside Datadog's own course platform. We instrumented an e-commerce application and then ran simulated tests: following a user's path through the store, and hunting down infrastructure bottlenecks that were also simulated. Traces, dashboards and alerts, all configured by us.
Where it went afterwards
Back at work, I instrumented our whole front end with Grafana — logs, traces and the rest — to validate whether a new feature would get the adoption the team expected.
It did. The data showed the button that triggers the feature was clicked far more than the others on the screen — the clearest sign that users found it and wanted it. I also brought Google Analytics into the Grafana dashboards, so product usage and technical telemetry lived side by side, and tracking that adoption was just another panel.
Observability is usually sold as a way to find what broke. Here it answered a product question: is anyone actually using this?
The speech that isn't new
What repeated across the day was the speech about the value of the developer. Every speaker said the same sentence in different words: the developer is a conductor now.
Except that sentence isn't new. Ten years ago it was "keep up or fall behind", and the object was frameworks and architecture. Today the object is AI. The noun changed, not the argument.