Fix 4 failure modes from the 2026-07-05 live call (the 'Nina' call)

- callstate: corrections win (most recent caller statement replaces old
  slots — 'no, my name is Carlos' was ignored); patient_name only from the
  caller explicitly giving their own name; booking classified only on
  explicit scheduling intent (small talk flipped a call to booking and
  drove a re-ask loop); new will_call_back call type short-circuits to a
  warm close instead of a message-taking flow.
- bot: per-call _today_line() injects the real current date (spoken form,
  answer-only) — AVA guessed 'Tuesday' on a Sunday and argued; hours rule
  now also bans stating open/closed days ('closed on Sundays' was invented).
- EndCallProcessor: phone-confirm gate reads CallState (booking/callback +
  a captured name or reason) instead of the keyword heuristic when tracking
  is on, and the injected confirmation now ends the call — the goodbye had
  already played, and holding the line open confused the caller.

Verified: staged replay of the failed transcript on activeblue-avc:latest
(4 stages x 3 runs, all pass, incl. clean-booking control); ab_replay.py
--state 0 failures, latency med 0.52s. Deployed via systemd restart.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
tocmo0nlord
2026-07-05 05:44:12 +00:00
parent db986ecf76
commit e714262155
3 changed files with 137 additions and 27 deletions

View File

@@ -40,13 +40,19 @@ Watches LLM text stream for closing keywords ("goodbye"), waits for TTS to finis
clipped, then pushes `EndTaskFrame` upstream. `TwilioFrameSerializer` with `auto_hang_up`
drops the carrier leg. Verified working in the Phase 1 gate (4/4 clean hang-ups).
It also **deterministically guarantees the callback number is confirmed** on booking calls:
the 8B reads the number back only ~half the time, so if a closing is reached on a booking
call (booking keyword seen) without the agent having spoken the number (`phone_marker` not
seen in its replies), the hang-up is suppressed and a scripted confirmation line
(`phone_confirm_line`, the caller-ID spelled out) is injected as a `TTSSpeakFrame` first.
The agent's own readback satisfies the gate, so there's no double-ask in the common case;
info-only calls (no booking keyword) are never asked for a number.
It also **deterministically guarantees the callback number is confirmed** on real capture
calls: the 8B reads the number back only ~half the time, so if a closing is reached without
the agent having spoken the number (`phone_marker` not seen in its replies), a scripted
confirmation line (`phone_confirm_line`, the caller-ID spelled out) is injected as a
`TTSSpeakFrame`**and the call then hangs up** (the goodbye already played; holding the
line open after it confused a live caller on 2026-07-05: "did you say goodbye, or are we
continuing?"). The injected wording tells the caller to call back if the number is wrong,
since no reply is possible. The agent's own readback satisfies the gate, so there's no
double-ask in the common case. Whether the gate applies comes from **CallState** when
tracking is on (`state_provider` → call_type booking/callback AND a captured name or
reason — a misheard word once flipped a chit-chat call to "booking" and forced a number
confirmation nobody wanted); on the anthropic path (no tracking) it falls back to the
booking-keyword heuristic. Info-only and will-call-back calls are never asked for a number.
**Mulaw 8kHz ↔ 16kHz conversion** — handled internally by `TwilioFrameSerializer`.
`PIPELINE_SAMPLE_RATE = 16000`, `WIRE_SAMPLE_RATE = 8000` are already set correctly.
@@ -118,7 +124,20 @@ before generation) it synchronously merges VAD-fragmented consecutive user messa
insurance, preferred day/time`. It also carries call type (`callback` → "do NOT ask booking
questions"). Verified via `scripts/ab_replay.py` (replays the real failed calls): llama3.1-8B
raw = 3 failures, +CALL STATE = **0 failures**, chat latency 0.31s→0.55s med (system-message
churn re-evals the prompt; acceptable, still ≪ the 3s gate). Env: `CALL_STATE_TRACKING`
churn re-evals the prompt; acceptable, still ≪ the 3s gate).
**Hardened after the 2026-07-05 "Nina" call** (STT misheard "Nah, I'll call you right back"
as "Nina, …"; the extractor locked `patient_name: Nina`, never accepted "no, my name is
Carlos", classified the call `booking` off small talk, and the checklist drove a booking
re-ask loop): the extractor prompt now (1) treats the caller's MOST RECENT statement as
authoritative — corrections replace old values; (2) takes `patient_name` ONLY from the
caller explicitly giving their own name, never from assistant turns or stray words;
(3) classifies `booking` ONLY on explicit scheduling intent (doubt → `unknown`); (4) adds
call_type **`will_call_back`** ("I'll call you back" / "I have to go") whose state block
short-circuits to a warm close — no name/number/booking questions, no message-taking.
Groomer exposes `.state` (read by `EndCallProcessor`'s phone-confirm gate). All four cases
verified against the real transcript on `activeblue-avc:latest` (3 runs each, incl. a clean
booking control); `ab_replay.py --state` still 0 failures, latency med 0.52s.
Env: `CALL_STATE_TRACKING`
(default: on for ollama, off for anthropic — Claude tracks state fine on its own; extraction
always runs on the local Ollama model). Qwen3-14B was A/B'd as an alternative and rejected
for now: no better raw, ~3s/turn with state, needs `think:false` handling, ~11GB VRAM.
@@ -390,6 +409,14 @@ the model's weak adherence) also hallucinated availability. Tested directly, the
computes appointment dates wrong ~5/5 times, so stating a computed date is a liability. The
calendar injection and in-call validation were removed (commit on 2026-06-25).
**One narrow exception (2026-07-05, `_today_line()` in `bot.py`):** the system prompt now
gets ONE line with the real current day/date in spoken form, built fresh per call. A live
call had AVA guess "Tuesday" on a Sunday and argue about it — with the true date on hand it
can just answer "what day is it?". The line explicitly says it changes nothing else: no
computing/stating/correcting appointment dates. This is NOT the 45-day calendar coming back
(~30 tokens, answer-only). The hours rule was also tightened: never say which days an office
is open or closed (AVA had invented "we're closed on Sundays").
The post-call extractor still gets today's date and records a **best-effort** `resolved_date`
for staff convenience — it is staff-verified on callback, not authoritative, and never spoken to
the caller. A deterministic (non-LLM) date resolver is the right hardening if accuracy is needed;