feat(policies): read all pages in the chainloop.discover builtin

Merged
MI
1d ago
+306-20

The chainloop.discover policy builtin (Rego and WASM) returned only the first page of the paginated DiscoverPrivate API, which holds the 20 newest references by default. A policy that looked for an older reference of a digest, such as a CI run behind several newer deploy or scan runs, could not find it and got no error.

The builtin now reads all the pages:

  • It requests pages of 100 references (the maximum the API accepts) and follows next_cursor until there is no next page.
  • It merges the references of all pages into one result with the same shape as before, so existing policies need no change.
  • A cap of 10 pages (1000 references) bounds the requests of one call. When the read stops before the last page, the result has "truncated": true; otherwise "truncated": false.
  • A control plane without pagination support returns all references in one response with no next cursor, so the read ends after one request with nothing dropped. A control plane that ignores the cursor returns the same next cursor again; the read stops there, does not merge the repeated page, and sets truncated.
  • With no control plane connection, the Rego builtin returns null instead of panicking.

The builtin does not send the run's project_name. That would filter out references from other projects, which policies such as provenance checks can depend on, so it would change what the builtin returns.

This change is limited to the page-reading loop in pkg/policies/engine/builtins and does not overlap the organization scoping in #3777.

Note: pkg/policies/engine/rego/builtins/discover.go holds an older copy of the discover builtin that nothing in the repo registers. It still reads one page. This PR leaves it as is; it can be removed in a follow-up.

Testing: table-driven unit tests with a fake referrer client cover a single page, several pages, the last page allowed by the cap, the cap that stops the read (truncated), an error on a later page, a control plane without pagination, and a control plane that ignores the cursor. They also cover the JSON shape of the result and the output of the Rego builtin.

AI disclosure: this contribution was made with the help of Claude Code.

Closes #3780

šŸ¤– Posted by Maximus bot (Claude Code) on behalf of @migmartri using the eng-autopilot skill