NewAgent Observability — find failures before customers do.

Agent observability · Prompts & experiments

Prompt management

Find the issues a prompt is causing, update the prompt, and replay it against past conversations. If the scores are better, release it to all your agents right away.

How to set it up

How prompts work in Oodle

Manage, update, edit, compare, release.

1 · Manage

Keep every prompt in Oodle. Your agent fetches it at runtime.

Your code asks for a prompt by name and label, fills in the variables, and runs. The prompt text lives in Oodle, not in your repo, so changing it never means a deploy.

agent.py
  1. 1# Fetch by label
  2. 2prompt = oodle.get_prompt(
  3. 3 "voice-agent-appointment-booking",
  4. 4 label="production",
  5. 5)
  6. 6
  7. 7# Compile with variables
  8. 8compiled = prompt.compile(
  9. 9 caller_name="Priya",
  10. 10 transcript="Hi, can I move my Friday appointment to next week?",
  11. 11)

2 · Update

Update prompts directly from Oodle.

Every edit is a version. Labels like production and candidate point at one version each, so releasing a change is moving a label.

prompts · sprout-express-support
  • v8candidatetoday
  • v7production3 days ago
  • v62 weeks ago
  • v51 month ago
promote

production label

v7 v8

Your code fetches by label. Moving it is the release; there is nothing to deploy and it is one click to move back.

Promote v8

3 · Edit

Edit the prompt next to the conversation that broke.

Open a failing session with its inputs already filled in. Change the prompt, swap the model, try again. Nothing to deploy.

trace · open in playground
A session transcript in Oodle
The Oodle playground with the session's inputs filled in

4 · Compare

Replay it against real conversations.

Keep the sessions that went wrong as test cases, real ones, not made up. Replay every candidate against them and score both versions with the evals you already trust.

trace · add to dataset
A session transcript in Oodle
Datasets and experiments in Oodle built from real sessions

5 · Release

Make the winner live. Keep watching.

Move the production label to the version that scored best. The evals keep scoring production, so if the problem comes back you see the drift before customers tell you.

prove the fix
0bxF9Zq5… · order_support_agent · 6 turns-0.82
“Your return has been filed.” — it wasn’t. initiateReturn returned 503.

returns-failures · 24 conversations like this one

Tool failed → agent claims success20
Deadlock → agent retries 4×4

Step 1 · Failing conversations

Real conversations, kept as test cases.

Now change the prompt.

Try it on your agents

Two commands. No credit card. Every session kept from the first one.