What it does
When a user ends a long conversation with “thanks!” or “bye”, the client resends the whole conversation to the model, just to get “You’re welcome!” back. If the provider’s prompt cache has expired by then, that one-word turn is billed as the full context at full price. Thecourtesy_reply guardrail recognizes these turns and answers them at the
gateway, so the conversation is never resent. By default the reply is canned
text and nothing reaches a provider; with reply: model, only the last few
messages go to a small reply_model. It works on
/v1/chat/completions, /v1/responses, and /v1/messages, and recognizes
thank-yous and goodbyes in English, Polish, German, Spanish, French,
Portuguese, and Italian.
When it answers
By default it answers only when skipping the model saves real money:- The cache is cold. The gateway remembers when each conversation last reached its provider and model. A courtesy turn is answered only when the prompt cache has likely expired (five minutes by default, longer when the request asks for a longer cache) or the provider is not caching at all. While the cache is warm, the turn costs mostly cheap cache reads and goes to the model as usual.
- The conversation is big enough. Conversations under 2,000 tokens go to the model; the saving is not worth replacing its reply.
- The message is only a courtesy. Short messages made of nothing but thanks or goodbye phrases. Anything with a question, a request, or new information goes to the model.
unknown_cache is set to warm.
What always reaches the model
The guardrail stays out of the way whenever a canned reply could be wrong:- pending tool or function results, and non-text content sent with the turn
- a turn right after the assistant asked a question, where “thanks” may mean “yes, go ahead”
- forced tool calls and structured output
- provider-stateful Responses requests; a Responses turn is answered only
with
"store": false
reply_to_acknowledgments.
Enable it
Add a rule to theguardrails section of config/config.yaml:
config.yaml join the default
workflow; a workflow of your own runs
the rule only when one of its prompt steps references it. You can also create
the rule from Plugins & Guardrails in the Dashboard, which shows every
setting with its help text.
Main settings
Settings go under the rule’sconfig: key:
The full list, including custom reply texts, extra phrases, and an optional
decision model for phrasings the built-in list does not know, is returned by
GET /admin/guardrails/types and shown in the Dashboard.
Courtesy replies are part of the compression entitlement. Without it, a
configured rule still loads and lets every request through.