Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add RFD describing support for multi-repo target #963

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

j-lanson
Copy link
Collaborator

@j-lanson j-lanson commented Feb 26, 2025

Resolves #962 .

Currently a very rough draft, perhaps too vague to be an RFD as-is. I'm looking for feedback from people who know the target resolution, salsa, and Session subsystems, then I can iterate on the content.

(EDIT from Andrew: Rendered)

Signed-off-by: jlanson <jlanson@mitre.org>
@j-lanson j-lanson added the product: project Relates to the project itself label Feb 26, 2025
@j-lanson j-lanson self-assigned this Feb 26, 2025
Signed-off-by: Julian Lanson <jlanson@mitre.org>
@alilleybrinker
Copy link
Collaborator

Regarding the ref flag and multi-target seeds, we could add support for the single-target "VCS URL" spec from PyPI: https://pip.pypa.io/en/stable/topics/vcs-support/

Then, when resolving targets from a multi-target seed, any that point directly to Git URLs could be resolved to git+ VCS URLs, including with a ref specifier if necessary. This would permit ref handling in the multi-target case, and is probably what we'd need to do to support things like a Cargo.toml with Git-dependencies anyway. Similarly, I know Go dependencies are basically Git repos, so we'd also need to do this for future go.mod support.


As an alternative to intra-process parallelization with async code, we could
spawn Hipcheck subprocesses that each communicate with their own copies of the
plugin processes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably would also want to produce reports as we run.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, remembering more of what I meant by the comment above. I think this section should have one more paragraph added explaining that when we're executing analyses in parallel we should be printing reports as they're finished. This avoids holding reports in memory and means we provide a more responsive user experience.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd presume for multi-target we'd want to offer some kind of "summary" printing option? and/or print the full report as JSON to a target folder?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the option to print a JSON report to a target folder, even for a single target run, is something I brought up in one of the discussions last week. At least one Hipcheck user is going that already as part of a pipeline that includes Hipcheck. They do it now by piping the JSON output from stdout to a file, but we could make a flag to handle that ourselves. Happy to make an issue for that if we think it's a good idea.

Copy link
Collaborator

@alilleybrinker alilleybrinker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some thoughts. Happy to discuss further.


As an alternative to intra-process parallelization with async code, we could
spawn Hipcheck subprocesses that each communicate with their own copies of the
plugin processes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, remembering more of what I meant by the comment above. I think this section should have one more paragraph added explaining that when we're executing analyses in parallel we should be printing reports as they're finished. This avoids holding reports in memory and means we provide a more responsive user experience.

Signed-off-by: jlanson <jlanson@mitre.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product: project Relates to the project itself
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Write RFD describing proposed changes to HC core to support multi-Target
3 participants