Fix 4 failure modes from the 2026-07-05 live call (the 'Nina' call)

- callstate: corrections win (most recent caller statement replaces old
  slots — 'no, my name is Carlos' was ignored); patient_name only from the
  caller explicitly giving their own name; booking classified only on
  explicit scheduling intent (small talk flipped a call to booking and
  drove a re-ask loop); new will_call_back call type short-circuits to a
  warm close instead of a message-taking flow.
- bot: per-call _today_line() injects the real current date (spoken form,
  answer-only) — AVA guessed 'Tuesday' on a Sunday and argued; hours rule
  now also bans stating open/closed days ('closed on Sundays' was invented).
- EndCallProcessor: phone-confirm gate reads CallState (booking/callback +
  a captured name or reason) instead of the keyword heuristic when tracking
  is on, and the injected confirmation now ends the call — the goodbye had
  already played, and holding the line open confused the caller.

Verified: staged replay of the failed transcript on activeblue-avc:latest
(4 stages x 3 runs, all pass, incl. clean-booking control); ab_replay.py
--state 0 failures, latency med 0.52s. Deployed via systemd restart.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CLAUDE.md

@@ -40,13 +40,19 @@ Watches LLM text stream for closing keywords ("goodbye"), waits for TTS to finis
 clipped, then pushes `EndTaskFrame` upstream. `TwilioFrameSerializer` with `auto_hang_up`
 drops the carrier leg. Verified working in the Phase 1 gate (4/4 clean hang-ups).
 
-It also **deterministically guarantees the callback number is confirmed** on booking calls:
+It also **deterministically guarantees the callback number is confirmed** on real capture
-the 8B reads the number back only ~half the time, so if a closing is reached on a booking
+calls: the 8B reads the number back only ~half the time, so if a closing is reached without
-call (booking keyword seen) without the agent having spoken the number (`phone_marker` not
+the agent having spoken the number (`phone_marker` not seen in its replies), a scripted
-seen in its replies), the hang-up is suppressed and a scripted confirmation line
+confirmation line (`phone_confirm_line`, the caller-ID spelled out) is injected as a
-(`phone_confirm_line`, the caller-ID spelled out) is injected as a `TTSSpeakFrame` first.
+`TTSSpeakFrame` — **and the call then hangs up** (the goodbye already played; holding the
-The agent's own readback satisfies the gate, so there's no double-ask in the common case;
+line open after it confused a live caller on 2026-07-05: "did you say goodbye, or are we
-info-only calls (no booking keyword) are never asked for a number.
+continuing?"). The injected wording tells the caller to call back if the number is wrong,
+since no reply is possible. The agent's own readback satisfies the gate, so there's no
+double-ask in the common case. Whether the gate applies comes from **CallState** when
+tracking is on (`state_provider` → call_type booking/callback AND a captured name or
+reason — a misheard word once flipped a chit-chat call to "booking" and forced a number
+confirmation nobody wanted); on the anthropic path (no tracking) it falls back to the
+booking-keyword heuristic. Info-only and will-call-back calls are never asked for a number.
 
 **Mulaw 8kHz ↔ 16kHz conversion** — handled internally by `TwilioFrameSerializer`.
 `PIPELINE_SAMPLE_RATE = 16000`, `WIRE_SAMPLE_RATE = 8000` are already set correctly.
@@ -118,7 +124,20 @@ before generation) it synchronously merges VAD-fragmented consecutive user messa
 insurance, preferred day/time`. It also carries call type (`callback` → "do NOT ask booking
 questions"). Verified via `scripts/ab_replay.py` (replays the real failed calls): llama3.1-8B
 raw = 3 failures, +CALL STATE = **0 failures**, chat latency 0.31s→0.55s med (system-message
-churn re-evals the prompt; acceptable, still ≪ the 3s gate). Env: `CALL_STATE_TRACKING`
+churn re-evals the prompt; acceptable, still ≪ the 3s gate).
+**Hardened after the 2026-07-05 "Nina" call** (STT misheard "Nah, I'll call you right back"
+as "Nina, …"; the extractor locked `patient_name: Nina`, never accepted "no, my name is
+Carlos", classified the call `booking` off small talk, and the checklist drove a booking
+re-ask loop): the extractor prompt now (1) treats the caller's MOST RECENT statement as
+authoritative — corrections replace old values; (2) takes `patient_name` ONLY from the
+caller explicitly giving their own name, never from assistant turns or stray words;
+(3) classifies `booking` ONLY on explicit scheduling intent (doubt → `unknown`); (4) adds
+call_type **`will_call_back`** ("I'll call you back" / "I have to go") whose state block
+short-circuits to a warm close — no name/number/booking questions, no message-taking.
+Groomer exposes `.state` (read by `EndCallProcessor`'s phone-confirm gate). All four cases
+verified against the real transcript on `activeblue-avc:latest` (3 runs each, incl. a clean
+booking control); `ab_replay.py --state` still 0 failures, latency med 0.52s.
+Env: `CALL_STATE_TRACKING`
 (default: on for ollama, off for anthropic — Claude tracks state fine on its own; extraction
 always runs on the local Ollama model). Qwen3-14B was A/B'd as an alternative and rejected
 for now: no better raw, ~3s/turn with state, needs `think:false` handling, ~11GB VRAM.
@@ -390,6 +409,14 @@ the model's weak adherence) also hallucinated availability. Tested directly, the
 computes appointment dates wrong ~5/5 times, so stating a computed date is a liability. The
 calendar injection and in-call validation were removed (commit on 2026-06-25).
 
+**One narrow exception (2026-07-05, `_today_line()` in `bot.py`):** the system prompt now
+gets ONE line with the real current day/date in spoken form, built fresh per call. A live
+call had AVA guess "Tuesday" on a Sunday and argue about it — with the true date on hand it
+can just answer "what day is it?". The line explicitly says it changes nothing else: no
+computing/stating/correcting appointment dates. This is NOT the 45-day calendar coming back
+(~30 tokens, answer-only). The hours rule was also tightened: never say which days an office
+is open or closed (AVA had invented "we're closed on Sundays").
+
 The post-call extractor still gets today's date and records a **best-effort** `resolved_date`
 for staff convenience — it is staff-verified on callback, not authoritative, and never spoken to
 the caller. A deterministic (non-LLM) date resolver is the right hardening if accuracy is needed;

bot.py

@@ -11,6 +11,7 @@ the FastAPI/TwiML/WebSocket side and calls run_call() once per call.
 """
 
 import asyncio
+import datetime
 import os
 
 import re
@@ -262,7 +263,7 @@ SYSTEM_PROMPT = (
     "is 'booked', 'scheduled', 'set', or 'confirmed' — not even in the recap. It is always a "
     "REQUEST: say you've NOTED it and a staff member will call back to confirm the date and time.\n"
     "- Hours are not published — say they vary by office and staff will confirm; never give "
-    "specific hours.\n"
+    "specific hours and never say which days an office is open or closed.\n"
     "- You don't give medical advice and can't transfer calls. If the caller mentions an eye "
     "problem, just note it as the reason and say a staff member or doctor will follow up.\n"
     "- If you're not sure you heard something, simply ask them to repeat it.\n"
@@ -274,6 +275,28 @@ SYSTEM_PROMPT = (
     "PRACTICE FACTS:\n" + practice_summary()
 )
 
+_DAY_ORDINALS = (
+    "first", "second", "third", "fourth", "fifth", "sixth", "seventh", "eighth", "ninth",
+    "tenth", "eleventh", "twelfth", "thirteenth", "fourteenth", "fifteenth", "sixteenth",
+    "seventeenth", "eighteenth", "nineteenth", "twentieth", "twenty-first", "twenty-second",
+    "twenty-third", "twenty-fourth", "twenty-fifth", "twenty-sixth", "twenty-seventh",
+    "twenty-eighth", "twenty-ninth", "thirtieth", "thirty-first",
+)
+
+
+def _today_line() -> str:
+    """One spoken-form line with the real current date, built fresh per call (the service
+    runs for days). Lets the agent answer 'what day is it?' truthfully — a live call had it
+    guess 'Tuesday' on a Sunday and argue — WITHOUT reintroducing in-call date math:
+    appointment dates stay capture-and-defer (see the DATES rule)."""
+    now = datetime.datetime.now()
+    return (
+        f"\n\nTODAY: right now it is {now:%A}, {now:%B} {_DAY_ORDINALS[now.day - 1]}. If the "
+        "caller asks the day or date, answer from this line only. This changes nothing else: "
+        "still take appointment days in the caller's own words, and never compute, state, or "
+        "correct an appointment's calendar date."
+    )
+
 
 def _build_tools() -> ToolsSchema:
     # Only the booking action is a tool. Practice facts already live in the system prompt,
@@ -309,17 +332,24 @@ class EndCallProcessor(FrameProcessor):
     (TwilioFrameSerializer auto_hang_up drops the call).
 
     Deterministic phone confirmation: the prompt asks the agent to read the callback number
-    back, but the 8B skips it ~half the time. So if a closing is reached and the agent never
+    back, but the 8B skips it ~half the time. So if a closing is reached on a call with a real
-    spoke the number this call (`phone_marker` not seen in its replies), we suppress the
+    capture and the agent never spoke the number (`phone_marker` not seen in its replies), the
-    hang-up and inject a scripted confirmation turn first — guaranteeing it happens exactly
+    scripted confirmation line is injected after the goodbye and the call then hangs up —
-    once (the agent's own readback satisfies the gate, so no double-ask in the common case)."""
+    since the goodbye has already been spoken, holding the line open afterwards just confuses
+    the caller (live call 2026-07-05: 'did you say goodbye, or are we continuing?'). The
+    agent's own readback satisfies the gate, so no double-ask in the common case.
+
+    Whether a capture is real comes from CallState via `state_provider` when tracking is on
+    (a misheard word once flipped a call to 'booking' and forced a number confirmation nobody
+    wanted); without tracking it falls back to the booking-keyword heuristic."""
 
     _CLOSINGS = ("goodbye", "good-bye", "good bye", "adiós", "adios", "hasta luego")
-    # Only force phone confirmation when a booking was actually underway (not info-only calls).
+    # Fallback heuristic when no CallState is available (e.g. anthropic path).
     _BOOKING_KWS = ("appointment", "schedule", "book", "insurance", "what day", "what time",
                     "come in", "preferred")
 
-    def __init__(self, phone_confirm_line: str | None = None, phone_marker: str | None = None):
+    def __init__(self, phone_confirm_line: str | None = None, phone_marker: str | None = None,
+                 state_provider=None):
         super().__init__()
         self._buf = ""
         self._should_end = False
@@ -330,12 +360,25 @@ class EndCallProcessor(FrameProcessor):
         self._phone_confirmed = not phone_confirm_line
         self._assistant_seen = ""
         self._pending_phone_inject = False
+        self._state_provider = state_provider
 
     @classmethod
     def _is_closing(cls, text: str) -> bool:
         t = (text or "").lower()
         return any(c in t for c in cls._CLOSINGS)
 
+    def _capture_underway(self) -> bool:
+        """Does this call actually need the phone confirmation? With CallState: only a
+        booking/callback call that captured a name or reason. Without: keyword heuristic."""
+        state = self._state_provider() if self._state_provider else None
+        if state:
+            ctype = (state.get("call_type") or "").strip().lower()
+            def _s(key):
+                v = state.get(key)
+                return v.strip() if isinstance(v, str) else ""
+            return ctype in ("booking", "callback") and bool(_s("patient_name") or _s("reason"))
+        return any(k in self._assistant_seen for k in self._BOOKING_KWS)
+
     async def _hang_up_after_delay(self):
         await asyncio.sleep(HANGUP_DELAY_SECS)
         logger.info(f"{AGENT_NAME} ending task / hanging up")
@@ -353,20 +396,23 @@ class EndCallProcessor(FrameProcessor):
                 self._phone_confirmed = True  # the agent read the number back itself
         elif isinstance(frame, LLMFullResponseEndFrame):
             if self._is_closing(self._buf):
-                booking = any(k in self._assistant_seen for k in self._BOOKING_KWS)
+                if self._phone_confirmed or not self._capture_underway():
-                if self._phone_confirmed or not booking:
                     self._should_end = True
                     logger.info(f"{AGENT_NAME} signalled closing -- will hang up "
                                 f"{HANGUP_DELAY_SECS:.0f}s after she finishes speaking")
                 else:
-                    # Booking call closing without the number confirmed — do it deterministically.
+                    # Real capture closing without the number confirmed — speak it, then end.
                     self._pending_phone_inject = True
-                    logger.info(f"{AGENT_NAME} reached closing w/o phone confirmation -- injecting it")
+                    logger.info(f"{AGENT_NAME} reached closing w/o phone confirmation -- "
+                                "injecting it, then hanging up")
             self._buf = ""
         elif isinstance(frame, BotStoppedSpeakingFrame):
             if self._pending_phone_inject:
                 self._pending_phone_inject = False
                 self._phone_confirmed = True
+                # The goodbye already played — after the injected line finishes (next
+                # BotStoppedSpeakingFrame), hang up instead of leaving the line open.
+                self._should_end = True
                 await self.push_frame(TTSSpeakFrame(self._phone_confirm_line), FrameDirection.DOWNSTREAM)
             elif self._should_end:
                 self._should_end = False
@@ -707,7 +753,7 @@ async def run_agent(transport, caller_number=None, call_sid=None, do_capture=Tru
             "\n\nCALLER ID: no number is available — near the end, ask the caller for the best "
             "phone number to reach them."
         )
-    system_content = SYSTEM_PROMPT + caller_line
+    system_content = SYSTEM_PROMPT + _today_line() + caller_line
     context_kwargs = {"messages": [{"role": "system", "content": system_content}]}
     if ENABLE_TOOLS:
         context_kwargs["tools"] = _build_tools()
@@ -756,13 +802,19 @@ async def run_agent(transport, caller_number=None, call_sid=None, do_capture=Tru
     # having read the caller-ID back, EndCallProcessor speaks this scripted line first.
     if caller_number:
         _spoken = _spoken_phone(caller_number)
+        # Spoken AFTER the goodbye and the call then hangs up, so the correction path must
+        # not need a reply on this call — staff have the number for the callback anyway.
         phone_confirm_line = (
-            f"Also, I have your number as {_spoken}; if that's not the best number, just let me know."
+            f"One last thing — I have your number as {_spoken} for the callback; if that's "
+            "not the best number, please give us a quick call back. Thank you, goodbye!"
         )
         phone_marker = _spoken.split(",")[0].strip()  # e.g. "nine seven three"
     else:
         phone_confirm_line = phone_marker = None
-    endcall = EndCallProcessor(phone_confirm_line=phone_confirm_line, phone_marker=phone_marker)
+    endcall = EndCallProcessor(
+        phone_confirm_line=phone_confirm_line, phone_marker=phone_marker,
+        state_provider=(lambda: groomer.state) if groomer else None,
+    )
 
     watchdog = SilenceWatchdog(
         silence_secs=SILENCE_REPROMPT_SECS,

callstate.py

@@ -33,17 +33,30 @@ from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
 _STATE_INSTRUCTIONS = (
     "You are tracking the state of a LIVE phone call between a caller and the receptionist "
     "of an optometry practice. From the transcript, extract only what the CALLER has clearly "
-    "provided so far. Respond with ONLY a JSON object with these keys:\n"
+    "provided so far. The assistant's lines are context only and MAY CONTAIN MISTAKES — never "
-    '  "call_type": "booking" (wants to schedule a visit), "callback" (wants something staff '
+    "treat something as true just because the assistant said it. Corrections win: if the "
-    "must check off-phone: order/frames/lens/prescription status, billing, account lookup, "
+    "caller corrects a value ('no, my name is Carlos', 'actually Tuesday'), output the "
-    'reach a person), "question" (just asking something), or "unknown"\n'
+    "caller's MOST RECENT statement and discard the old value entirely. "
+    "Respond with ONLY a JSON object with these keys:\n"
+    '  "call_type": one of:\n'
+    '     "booking" — ONLY if the caller explicitly asked to schedule/book a visit or come '
+    "in. Greetings, small talk, or asking what you can help with are NOT booking.\n"
+    '     "callback" — wants something staff must check off-phone: order/frames/lens/'
+    "prescription status, billing, account lookup, reach a person.\n"
+    '     "will_call_back" — the caller said THEY will call back later, have to go, or want '
+    "to end the call.\n"
+    '     "question" — just asking something.\n'
+    '     "unknown" — none of the above clearly applies yet. When in doubt, use "unknown", '
+    'not "booking".\n'
     '  "reason": string or null — WHAT the caller wants. For booking: the visit type or eye '
     'problem (e.g. "annual exam", "eye pain"). For callback: what they want checked or done, '
     'even if phrased as a question (e.g. "are my glasses ready", "status of an order", '
     '"billing question"). Only \'an appointment\' with no visit reason is NOT a reason — '
     "use null then.\n"
     '  "location": string or null — the office/city the caller wants\n'
-    '  "patient_name": string or null — the caller\'s name as given (full or first-only)\n'
+    '  "patient_name": string or null — ONLY a name the caller explicitly gave as their own '
+    "(\"my name is…\", \"this is…\", or answering a name question); NEVER a name guessed "
+    "from other words, a garbled transcription, or a name only the assistant used\n"
     '  "name_is_full": boolean — true only if it clearly has first AND last name\n'
     '  "insurance": string or null — the plan the caller named, exactly as said\n'
     '  "preferred_time": string or null — day/time in the caller\'s own words\n'
@@ -101,6 +114,18 @@ def build_state_block(state) -> str:
     if not state:
         return ""
     ctype = (state.get("call_type") or "unknown").strip().lower()
+
+    if ctype == "will_call_back":
+        # Observed failure (2026-07-05 call): "I'll call you right back" got treated as a
+        # message-taking flow — the agent pushed for a callback number and invented a
+        # third-person caller. Cut straight to a warm close instead.
+        return (
+            "CALL STATE (auto-tracked from this conversation — trust it over your memory):\n"
+            "- The caller said THEY will call back. Do NOT ask for anything else — no name, "
+            "no phone number, no booking questions — and do not offer to take a message. "
+            "Warmly tell them they're welcome to call back anytime, and end your reply with "
+            "'Goodbye'."
+        )
     got, needed = [], []
     for key, label in _BOOKING_ORDER:
         val = (state.get(key) or "").strip() if isinstance(state.get(key), str) else ""
@@ -197,6 +222,12 @@ class CallStateGroomer(FrameProcessor):
         self._state = None
         self._task = None
 
+    @property
+    def state(self):
+        """Latest extracted slot state (dict or None). Read by EndCallProcessor's
+        phone-confirm gate so it only fires on real booking/callback captures."""
+        return self._state
+
     def _extract_done(self, task):
         self._task = None
         if task.cancelled():