Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change protobuf pin in training requirements #13596

Merged
merged 1 commit into from
Nov 9, 2022

Conversation

baijumeswani
Copy link
Contributor

@baijumeswani baijumeswani commented Nov 8, 2022

protobuf 3.18.3 is incompatible with onnx 1.12.0

Some pipelines rely on other sources (example vcpkg) which are older protobuf packages (3.18.3) and as a result, the pipeline fails.

Forcing re-install of protobuf to a newer version.

@baijumeswani baijumeswani added the training issues related to ONNX Runtime training; typically submitted using template label Nov 8, 2022
@baijumeswani baijumeswani requested a review from a team as a code owner November 8, 2022 23:11
@@ -4,6 +4,6 @@ h5py
numpy >= 1.16.6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this file used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requirements_file = "requirements-training.txt"

So, when the pipeline does pip install *.whl, it will try to first install all the dependencies listed in setup.py.

@baijumeswani baijumeswani merged commit e0361e6 into main Nov 9, 2022
@baijumeswani baijumeswani deleted the bmeswani/fix-win-gpu-pipeline branch November 9, 2022 17:37
@baijumeswani
Copy link
Contributor Author

Thanks for the review @snnn, @askhade, @jywu-msft

@@ -4,6 +4,6 @@ h5py
numpy >= 1.16.6
onnx
packaging
protobuf >= 3.12.2, <= 3.20.1
protobuf >= 3.20.0, <= 3.21.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't looks right. <= 3.21.0 means you want 3.21.0 (although it won't exist because they call it 4.21.0) but not 3.21.1

Copy link
Contributor Author

@baijumeswani baijumeswani Nov 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for sharing this. I'll make it < 4.21.1 in that case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to be <4, because that clearly states "major version introduces breaking change"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And no it's not <4.21.1, 4.21.0 won't work either.
What happen is, protobuf had a breaking change in 3.21, then its python variant decided to name 3.21 as 4.21 in pip wheel, and there're still new releases of 3.20.x after 4.21.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
training issues related to ONNX Runtime training; typically submitted using template
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants