Skip to content

Conversation

@teskje
Copy link
Contributor

@teskje teskje commented Feb 9, 2026

So far the expression cache, which is used to speed up envd restarts, is only populated during bootstrap. That means that dataflows that get created are not in the cache the first time envd restarts, only on subsequent restarts, making the expr cache less useful than it could be.

Another motivation for wanting this is that it ensures plan pinning within the same Mz version. This lets us change the order in which dataflows are sequenced, e.g. by moving indexes to the front, without risking compute reconciliation failures due to accidentally changing plans.

Motivation

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

So far the expression cache, which is used to speed up envd restarts, is
only populated during bootstrap. That means that dataflows that get
created are not in the cache the first time envd restarts, only on
subsequent restarts, making the expr cache less useful than it could be.

Another motivation for wanting this is that it ensures plan pinning
within the same Mz version. This lets us change the order in which
dataflows are sequenced, e.g. by moving indexes to the front, without
risking compute reconciliation failures due to accidentally changing
plans.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant