Feat: name forEach items by a key, and read the name as each.key

Merged
BR
13d ago
+910-140

Description of your changes

A loop can name its items' steps by their values (kubevela/pkg#154 adds forEach.key), and every step in a loop can read that name as each.key.

- name: deploy
  type: apply-object
  forEach:
    from: targets            # [{cluster: prod-us, namespace: shop}, ...]
    key: [cluster, namespace]
  properties:
    value:
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: 'web-$(each.key)'          # web-prod-us-shop
  • Names: deploy-prod-us-shop and rollout-prod-us-shop-verify in place of deploy-0 and rollout-0-verify. The item's outputs and its each variable are named the same way, so restart takes the keyed step name.
  • each.key sits beside each.item and each.index: the item's key, or its index as a string when the loop has none. It is readable in $( ) properties and conditions, from inputs[].from, and as context.each.key in a template.
  • One name for the engine and the user. Expressions find the pass a step runs in by each.key, so a step in a keyed loop reads its own pass's outputs ($(steps.fetch.outputs.data)), where matching by index would miss it. types.IterationName takes that label, so naming lives in one place.
  • Rules: each key part is a string or an integer, none is missing, the joined key uses step-name characters (no .), fits in 63 characters, and is unique in the loop. Any break fails the loop with a message naming the items, for example items 1 and 3 have the same key "x-y-z". A key is never rewritten to fit, since two values rewriting alike would break restart.
  • No name is parsed back into a key. The keys are stored beside the pinned items (<stepID>.for-each-keys) and used for restart, suspend backoff and clearing a pass's variables. Collected outputs stay in item order.
  • A failed keyed loop names the item by its key: item 1 (prod-us-shop) failed at verify: ....
  • The WorkflowRun and Workflow CRDs gain the field.

Two behaviour changes a reviewer should see:

  • Admission reserves names for a keyed loop. Its step names are only known once the list resolves, so the clash check reserves <loop>-.+ (and <loop>-.+-<sub> for a group) in place of <loop>-<n>, and another forEach named <keyed loop>-... is refused. A keyed loop deploy beside a step called deploy-x is refused at admission.
  • The variable-clash rule is narrower, for every loop. Admission and restart cleanup match only what a pass writes (<loop>-<n>-each and the body's outputs), not any <loop>-<n>-<anything>. So an output scale-0-other beside a loop scale that has no other output is now admitted, where it was refused. One validation case was updated and the opposite case added.

Fixes #

I have:

  • Read and followed KubeVela's contribution process.
  • Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
  • Run make reviewable to ensure this PR is ready for review.
  • Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

  • Names, variables and collected outputs with a single and a composite key, for a step and a group, in both modes.
  • Each rule failing the loop with its message, including two keys that join to the same name.
  • Restart clearing a keyed item's status and variables.
  • each.key read as $(each.key), and a keyed pass reading its own sibling's output and not another pass's.
  • Unit and envtest suites pass (bar the email provider tests, which need a local SMTP port); golangci-lint reports nothing. The keyed specs ran on k3d before this rebase onto #272.

Special notes for your reviewer

  • Merge order: kubevela/pkg#154, then #272, then this, then kubevela/kubevela#7468.
  • The last commit pins kubevela/pkg to the fork branch of kubevela/pkg#154 and is never merged; it is swapped for the upstream version once that merges.
  • Docs (a "Naming items by a key" section and each.key in the forEach page) will follow in kubevela.github.io.