Expose stored embeddings through listChunks

Merged
IA
6d ago
+442-5

rag.listChunks(ctx, { entryId, paginationOpts, includeEmbeddings: true }) now returns each chunk's embedding, so callers can search from an existing chunk or copy content into another namespace without calling an embedding provider again. The option defaults to false and preserves the existing response shape and read cost when omitted.

This extends the existing paginated query and chunk type; it adds no schema migration or separate search API. Ready and replaced chunks return normalized embeddings with importance weighting removed, while pending chunks return their original embeddings. The README includes search and paginated-copy examples and documents the existing storage limits: zero importance erases vector direction, and 4096-dimensional vectors have their discarded final coordinate padded with zero.

The copy example explicitly runs in an action. Although rag.add accepts mutation contexts for small pre-embedded entries, nested reads and writes share the enclosing mutation's transaction limits. A convex-test regression with default transaction limits enabled copies 1,501 chunks with 1,536-dimensional embeddings (over 16 MiB of embedding coordinates): the mutation fails with a read-limit error, while the action completes across 31 pages with zero embedding tokens. The test verifies every copied chunk's order, text, metadata, state, and embedding dimensions, plus representative coordinates.

Validation: all 119 tests pass; repository and example typechecks, ESLint, build, and formatting pass. Component codegen and circular-dependency checks also passed for the API change. Coverage includes pagination, pending/ready/replaced chunks, default omission, zero importance and 4096 dimensions, and search/copy reuse with independent destination importance.

Fixes #59.