Knowledge Base

The knowledge base is where a school writes down its policies, FAQs, and how-tos — and the retrieval engine that lets two AI consumers (Rover and the social agent) answer questions from it. It's the API's one pgvector feature, and its architecture separates cleanly: writes are ordinary CRUD, embedding happens in a worker, and search reads both indexes at once.

Two tables, one deliberately unmappable

knowledge_base (@TenantScoped) holds the document: title, description, a Markdown content blob, updatedBy — and isPublic, default false. That flag is the second access axis: docs are staff-only until explicitly published, and the public-facing social agent can only ever see isPublic rows.

knowledge_base_chunk is the search index — and it's declared @Entity('knowledge_base_chunk', { synchronize: false }) because its two working columns can't be expressed by TypeORM: embedding vector(1536) (pgvector, HNSW index) and content_tsv tsvector (generated column, GIN index). The schema lives in hand-written SQL migrations, migration:generate skips the table entirely, and CI runs a pgvector/pgvector:pg16 Postgres so those migrations apply (see The Database). Chunks deliberately store no tenancy or visibility of their own — both are read live via a join to the parent at search time, so publishing a doc never touches its chunk rows.

Writes enqueue, the worker embeds

CRUD is a plain staff controller (Resource.KNOWLEDGE_BASE per action, every write audited — with the content blob excluded from snapshots). The interesting part is when embedding happens:

• Create always enqueues an embed job on the kb-embedding queue.
• Update enqueues only when content actually changed — retitling or publishing a doc costs nothing, because visibility is read live.
• Delete soft-deletes the doc and hard-deletes its chunks — derived data doesn't get soft-delete ceremony.

The worker processor loads the doc scoped by the job's schoolId (no request context in a worker — the job carries the tenant), splits content into ≤2000-character chunks, prefixes every chunk with title — description so a retrieved fragment still carries its context, embeds the batch with text-embedding-3-small (1536 dims — must match the column), and replaces the index transactionally: delete-all, then insert with raw ::vector SQL.

Hybrid search

hybridSearch({ query, publicOnly?, limit? }) is the single retrieval door, and it fails closed: no active school context (and no operator bypass) → it throws rather than search across tenants. Because the chunk SQL is raw, the tenant boundary is bound explicitly — kb.school_id = $n — one of the few places @TenantScoped can't do it for you.

1. limit clamps to [1, 10] (default 5); each arm fetches a pool of limit × 4 candidates.
2. The query embeds with the same model, then both arms run in parallel: vector (ORDER BY embedding <=> $1::vector, cosine over HNSW) and keyword (content_tsv @@ websearch_to_tsquery('english', …) ranked by ts_rank_cd over GIN).
3. Reciprocal rank fusion merges the two rankings — each result scores Σ 1/(60 + rank) across the lists it appears in — then the top limit return as hits.

Why hybrid: vectors find "how do refunds work" when the doc says fee reversal policy; full-text wins on exact names and codes. RRF needs no score normalization between the two — only ranks — which is what makes the fusion robust.

Two consumers, two visibility levels

That one boolean is the entire boundary between "what staff can ask Rover" and "what the public-facing agent will ever repeat" — treat isPublic review accordingly.

Where to go next