Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parallel CI jobs for crate publishing #22

Open
joao-paulo-parity opened this issue Dec 8, 2022 · 2 comments
Open

Support parallel CI jobs for crate publishing #22

joao-paulo-parity opened this issue Dec 8, 2022 · 2 comments
Labels
question Further information is requested

Comments

@joao-paulo-parity
Copy link
Contributor

PRs which touch lots of crates, for example paritytech/substrate#12768, take a long time to publish overall (example: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2130765). We can improve the task's overall time to completion by splitting crates into groups and publishing those groups in parallel, perhaps using paritytech/substrate#12709 as reference for an implementation.

@joao-paulo-parity joao-paulo-parity added the enhancement New feature or request label Dec 8, 2022
@joao-paulo-parity joao-paulo-parity changed the title Support parallel CI jobs for create publishing Support parallel CI jobs for crate publishing Dec 15, 2022
@joao-paulo-parity
Copy link
Contributor Author

To publish a crate you need to publish all of its dependencies first, so it makes more sense as a serial processing operation which happens in the context of a single job rather than a distributed operation spread across parallel jobs. In the latter case they'd need to somehow synchronize between themselves so that Crate Y is only published from Job B after Crate X is published from Job A. Feasibility aside, I don't know if it's worthwhile to do that.

@joao-paulo-parity joao-paulo-parity added question Further information is requested and removed enhancement New feature or request labels Dec 15, 2022
@joao-paulo-parity
Copy link
Contributor Author

Changing the ticket to "question" because I'd like to first have the plan for a future implementation proposal before moving to the implementation itself

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant