How the grounded assistant is built

The Q&A Assistant is a conversational layer on top of an explicitly governed knowledge base. Sources are curated in the repo (markdown and structured data), modeled as records and chunks, synced to Supabase, and retrieved with a hybrid pipeline (vectors + full-text + aliases + intent-aware scoring). Responses are generated with Anthropic Claude, with structured output, citations, and deterministic fallbacks when parsing or the API fails.

• Grounding: answers are tied to approved evidence, not the model’s latent “memory” of the web.
• Governance: public vs excluded content is enforced in the corpus and at runtime (explicit refusals for sensitive file names).
• Observability: sessions, messages, query logs, blocked events, feedback, handoffs, and eval runs are persisted when Supabase is configured.

Each user message travels through a single server pipeline (runGroundedChat) with timing breakdowns for profiling (session, intent, history, retrieval, LLM, normalize, logging).

The corpus is built in curated-records.ts: approved paths (e.g. about-me/*.md, project slices) and an explicit exclusion list for private or internal interview analyses. Each fact becomes a record with entity_type, canonical_key, visibility, assistant_summary, evidence text, tags, and query_aliases for retrieval.

Chunks split long material for embedding and FTS; each chunk carries enriched embed_text (not raw-only dumps), metadata for narrative ordering, recruiter evidence, and answer_roles so retrieval can prefer chunks suited to the current intent.

Query and chunk vectors use the OpenAI Embeddings API (text-embedding-3-large truncated to 1536 dimensions). The model is pinned in code next to the version tag stored with every vector, so a query is never embedded in a different space than the chunks it is compared with. An in-memory cache reduces duplicate calls. If the API key is missing, vector similarity degrades gracefully while lexical and alias signals remain.

Tables include qa_assistant_records, qa_assistant_chunks, qa_assistant_sessions, qa_assistant_messages, qa_assistant_query_logs, qa_assistant_feedback, qa_assistant_handoffs, qa_assistant_blocked_events, qa_assistant_eval_runs, and admin override state. The RPC qa_assistant_hybrid_candidates returns vector, FTS, alias, and title-match scores for candidate chunks filtered by visibility and entity type.

Retrieval combines: (1) Supabase hybrid candidates when available, (2) local cosine similarity against stored chunk embeddings, (3) composite scoreChunk weighting titles, summaries, content, tags, intent alignment, leadership / AI-tool boosts, and penalties for common confusions (e.g. spoken languages vs programming languages). Results are deduplicated by canonical entity, optionally forced to include an entity anchor, then trimmed via applyChunkBudgetAndOrdering for modes like narrative, recruiter fit, or project focus.

Anthropic models are fixed in code: an Opus-class Claude writes the answer and a Haiku-class model takes two internal decisions, the query planner and the context grader. Environment variables exist only as an override for local work or emergencies. Generation runs with a fixed reasoning effort and a token ceiling sized for reasoning models, and every provider call has its own timeout. System prompts enforce grounding to provided evidence. The client receives structured payloads for rich rendering (bullets, sections, project/fit cards). If the provider fails or JSON parsing fails, a deterministic fallback builds an answer from chunk summaries so the UI never silently hallucinates a full narrative.

POST /api/qa-assistant/chat is rate limited per IP twice: a Vercel firewall rule at the edge (12 requests per minute, shared by every function instance) and an in-memory limiter inside the function. Every chat request also has to carry the proof of Vercel BotID, an invisible bot check, and passes a daily spend ceiling read from the usage logs: once the day's model budget is spent, the assistant answers with a fixed notice instead of calling a model. Handoff submissions use stricter limits, honeypot fields, and Cloudflare Turnstile when configured. Admin routes are protected via Supabase auth and an allowlist of admin emails. Service-role keys never reach the browser.

Full-page chat at /qa-assistant and a floating widget share the same API, differentiated by an entrypoint flag for analytics. Citations, follow-ups, structured layouts, feedback thumbs, and a lead handoff flow are integrated in the UI layer.

CI runs on every push and pull request: type check, lint, the tool-consistency and intent-routing evals, the chat-routing and abstention checks, and a production build. A nightly job runs the eval regression suite. A weekly job checks knowledge-base drift against Supabase and the vector store, pricing-constant staleness, telemetry health, and that the configured models still exist and accept the exact request shape each code path sends. Quality evals (48 structural checks over eight canonical questions) and an LLM judge run on demand, with optional persistence to qa_assistant_eval_runs.

Typical production variables (names only — set values in your host):

• Designed a full RAG pipeline with explicit content governance, not generic file upload.
• Hybrid retrieval + intent/entity routing + mode-specific chunk budgets.
• Production-minded API: validation, rate limits, structured logging, lead capture with bot mitigation.
• Typed end-to-end (TypeScript + Zod) and eval hooks for quality over time.

• Intent classification is primarily rule-based; scaling may need labeled data and metrics.
• The edge rate limit counts per point of presence and the in-memory one per instance, so a burst spread across regions is capped by the daily spend ceiling, not by the rate limit.
• Each message may incur embedding + LLM cost — caching mitigates repeated phrasing, not novel questions.
• Answers take roughly 5–20 seconds end to end and are not streamed yet: the model reasons before it writes, retrieval adds a few seconds, and the client waits for the whole JSON. A progress signal and streaming are the next step.