Making a language model’s output safe to show a user

The rationale field is displayed verbatim. Live output read: “Fits Priya’s preference for afternoons only if needed? Actually this is the soonest open weekday slot…” The slot was valid; the model was thinking out loud into the UI.

The first fix was a list of prohibitions — don’t question yourself, don’t revise mid-sentence. It made things worse, producing clipped, malformed text that repeated itself. Replacing the prohibitions with a positive specification and a worked good/bad example fixed it. Telling a model what to produce beats enumerating what to avoid; a list of “don’ts” leaves it guessing at the shape of a “do”.

From a thread where an attendee wrote “I’m out Thursday”, the assistant proposed Thursday — and called it Wednesday.

The cause was in the prompt. Timestamps were rendered as bare dates, so the model had to derive a day-of-week before it could apply any constraint in the thread — and every real scheduling constraint is phrased in weekdays. It believed Aug 27 was a Wednesday, and reasoned correctly from there to a wrong answer.

Before:  2026-08-27 09:00 +00:00
After:   Thursday 2026-08-27 09:00 +00:00

This had surfaced before and been treated as a symptom — a validator’s comment already recorded the model proposing a Saturday and calling it Friday. The output was being checked; the cause, that nothing ever told the model what day anything was, went unaddressed.

With weekdays supplied, one failure remained — and it was the most instructive.

The heading is derived from the slot’s timestamp by application code and is always right. Giving the model weekday-labelled input didn’t stop it re-deriving the weekday when writing a new sentence about a slot it had just produced — a fresh arithmetic step, and one bad roll contradicts the line directly above it.

The fix was to delete the redundancy rather than defend it. The rationale is now forbidden from naming a day or date at all: the interface already shows it, deterministically, immediately above. Restating it created a second chance to be wrong with no upside.

• Give the model facts rather than making it derive them. Anything computable in code should arrive already computed.
• Never ask for the same fact twice. Two independent derivations will eventually disagree in front of a user, and the disagreement reads worse than either value alone.
• Validate output against ground truth anyway. Proposed slots are re-checked server-side against the real calendar — three of these four failures produced output that satisfied the prompt as written.

Each fix is pinned by a test asserting the rule survives in the prompt, with the failing output quoted in the test comment — so an edit that quietly drops a rule fails loudly and explains why the rule existed.