Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix patched dependencies across repositories #42463

Merged
merged 2 commits into from
Nov 9, 2024

Conversation

becker33
Copy link
Member

@becker33 becker33 commented Feb 2, 2024

Currently, if a package has a dependency from another repository and patches it, generation of the patch cache will fail. Concretization succeeds if a fixed patch cache is in place.

  • don't assume that patched dependencies are in the same repo when indexing
  • add some test fixtures to support multi-repo tests.

@spackbot-app spackbot-app bot added the core PR affects Spack core functionality label Feb 2, 2024
alalazo
alalazo previously requested changes Feb 5, 2024
Copy link
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we fix the underlying issue without resorting to using globals? Also, can we add a regression test?

@alalazo alalazo self-assigned this Feb 5, 2024
@pbrady
Copy link
Contributor

pbrady commented Feb 13, 2024

Just want to say that this fixes an important issue for me.

@tgamblin tgamblin added this to the v0.23 milestone Oct 27, 2024
Copy link
Member

@tgamblin tgamblin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@becker33 can you address @alalazo's comment for 0.23?

@tgamblin tgamblin force-pushed the bugfix/patches-across-repos branch from 5f93a75 to a00d6b2 Compare November 8, 2024 08:34
@tgamblin tgamblin force-pushed the bugfix/patches-across-repos branch from a00d6b2 to fa5b6ad Compare November 8, 2024 09:08
@spackbot-app spackbot-app bot added dependencies new-version tests General test capability(ies) labels Nov 8, 2024
@tgamblin tgamblin dismissed their stale review November 8, 2024 10:04

added a test

@tgamblin tgamblin requested review from alalazo and haampie November 8, 2024 10:04
@tgamblin
Copy link
Member

tgamblin commented Nov 8, 2024

@alalazo see if you like the test here. I do not see a good way to do this without using spack.repo.PATH right now, but perhaps I can revisit it when I rework repo metadata handling to be faster. I am planning on refactoring the way we do the misc_cache.

@tgamblin tgamblin force-pushed the bugfix/patches-across-repos branch from 7469aec to 7ddf20d Compare November 8, 2024 10:19
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
@tgamblin tgamblin force-pushed the bugfix/patches-across-repos branch from 7ddf20d to 2e6760c Compare November 9, 2024 00:19
@tgamblin tgamblin enabled auto-merge (squash) November 9, 2024 00:20
@tgamblin tgamblin dismissed alalazo’s stale review November 9, 2024 00:21

Greg and I reviewed each other's changes and this is addressed.

Copy link
Member

@tgamblin tgamblin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with @becker33's approach (see above). He's ok with/my tests. We also now have scaffolding for multi-repo tests in CI, which likely we will need more of.

@tgamblin tgamblin merged commit da1d533 into develop Nov 9, 2024
27 of 33 checks passed
@tgamblin tgamblin deleted the bugfix/patches-across-repos branch November 9, 2024 01:07
@@ -530,7 +530,7 @@ def update_package(self, pkg_fullname: str) -> None:

# update the index with per-package patch indexes
pkg_cls = self.repository.get_pkg_class(pkg_fullname)
partial_index = self._index_patches(pkg_cls, self.repository)
partial_index = self._index_patches(pkg_cls)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tgamblin @becker33 Note that this and the line above:

pkg_cls = self.repository.get_pkg_class(pkg_fullname)
partial_index = self._index_patches(pkg_cls)

was the main cause for my request ☝️ Now we are using both self.repository and spack.repo.PATH. We should keep that in mind, because if they ever get out of sync (and they probably do if this fixes the issue), they will cause bugs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be out of sync here? In any given Spack run, the dependency a name refers to is the one in spack.repo.PATH. So this is the only way I see right now to resolve a name that's not in the local repo, and it's the only one we really support (unless the dependency explicitly specifies a namespace, but then you want to go through PATH as well).

I'm not sure what you want this to do.

@tgamblin
Copy link
Member

tgamblin commented Nov 9, 2024

I'm baffled how this auto-merged without passing tests. Working on a fix. If I can't I'll revert.

alalazo pushed a commit that referenced this pull request Nov 9, 2024
alalazo pushed a commit that referenced this pull request Nov 9, 2024
fryeguy52 pushed a commit to fryeguy52/spack that referenced this pull request Dec 17, 2024
Currently, if a package has a dependency from another repository and patches it,
generation of the patch cache will fail. Concretization succeeds if a fixed patch
cache is in place.

- [x] don't assume that patched dependencies are in the same repo when indexing
- [x] add some test fixtures to support multi-repo tests.

---------

Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
fryeguy52 pushed a commit to fryeguy52/spack that referenced this pull request Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core PR affects Spack core functionality dependencies new-version tests General test capability(ies)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants