Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat empty environment variables as unset in setup.py #3763

Closed
wants to merge 13 commits into from

Conversation

Eta0
Copy link

@Eta0 Eta0 commented Jun 17, 2023

This change fixes a bug that causes setup.py to fail when any of its configuration environment variables are the empty string.

For example, running the following:

export DS_BUILD_OPS=""
pip install deepspeed

Currently fails with the following message:

BUILD_OP_DEFAULT = int(os.environ.get('DS_BUILD_OPS', BUILD_OP_PLATFORM))
ValueError: invalid literal for int() with base 10: ''

Because os.environ.get does not use its fallback argument when an environment variable exists, but is empty.
Similarly, key in os.environ returns True for empty (but still set) keys.

Shells usually do not distinguish between unset variables and empty variables, so setting variables to the empty string is usually seen as another way to unset them. This specifically arose as an issue for me while I was writing a Dockerfile, since those have no way to unset ENV and ARG directives—only to set them to the empty string (or to figure out and hardcode what the default would have resolved to for each ARG). I figure fixing this here is more helpful than writing weird workarounds.

This changes setup.py to treat unset and empty environment variables identically. This behaviour could potentially be changed in other files, as well, but that is not included here.

@loadams loadams requested a review from jeffra June 20, 2023 15:37
@loadams loadams requested a review from mrwyattii June 21, 2023 17:47
@loadams
Copy link
Contributor

loadams commented Jun 28, 2023

@Eta0 - if you want to merge this you'll need to reply to the Microsoft contributor agreement in the comment above

@Eta0
Copy link
Author

Eta0 commented Jun 28, 2023

@loadams Got it. Currently waiting on approval from my company to agree to it.

@mrwyattii
Copy link
Contributor

@Eta0 any update? It would be great to get this merged

@Eta0
Copy link
Author

Eta0 commented Jul 11, 2023

Thank you all for your patience! I'm still waiting on legal approval, so I'll check in with them again.


Update: it should be wrapped up some time this week 🎇 then I can sign and we can get this merged. Yay!

@loadams
Copy link
Contributor

loadams commented Jul 24, 2023

Hi @Eta0 - no rush, but figured I'd follow up since its been a week!

@loadams
Copy link
Contributor

loadams commented Aug 9, 2023

Hi @Eta0 - following up on this again, curious if you were able to get legal approval.

@loadams
Copy link
Contributor

loadams commented Aug 21, 2023

@Eta0 - Since we'd like to incorporate these changes, we've pulled them into a PR here: #4185

@loadams loadams closed this Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants