Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-38112: compileall: Skip long path path on Windows if the path can't be created #16414

Merged
merged 4 commits into from
Sep 26, 2019

Conversation

encukou
Copy link
Member

@encukou encukou commented Sep 26, 2019

This avoids the buildbot failure on Windows:

FileNotFoundError: [WinError 206] The filename or extension is too long: 'd:\\temp\\tmp5r3z438t\\long\\1\\2\\3\\4\\5\\6\\7\\8\\9\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\20\\21\\22\\23\\24\\25\\26\\27\\28\\29\\30\\31\\32\\33\\34\\35\\36\\37\\38\\39\\40\\41\\42\\43\\44\\45\\46\\47\\48\\49\\50\\51\\52\\53\\54\\55\\56\\57\\58\\59\\60\\61\\62\\63\\64\\65\\66\\67\\68\\69\\70\\71\\72\\73\\74\\75\\76\\77\\78'

This creates a path that's long but avoids OS restrictions.

https://bugs.python.org/issue38112

Automerge-Triggered-By: @encukou

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than creating a subdirectory of a fixing length (many_directories) at once, would it be possible to create subdirectories one by one until the path is long enough or until we get a "path too long" error?

For example, replace makedirs() with a loop and stop at the first error? So we could test a path which is longer than usual, even if we cannot create the full many_directories tree.

@vstinner
Copy link
Member

Ah! I found where I already had to do something similiar: https://bugs.python.org/issue37412

I got a similar error on Windows. I wrote a function which tries to create a "long path" piece by piece:
ec3e20a

Note: test_getcwd_long_path() has the constraint of needing to goes into this directory using os.chdir().

Also, actually run the test
@encukou
Copy link
Member Author

encukou commented Sep 26, 2019

Thanks, I'll look at your code

@encukou
Copy link
Member Author

encukou commented Sep 26, 2019

Hm, this one has the constraint of needing to fit __pycache__/*.pyc inside.

@encukou
Copy link
Member Author

encukou commented Sep 26, 2019

@vstinner, does this look reasonable?

# On Windows, a FileNotFoundError("The filename or extension
# is too long") is raised for long paths
if sys.platform == "win32":
break
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this raise otherwise? Same for OSError below. Just curious.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should. Thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and I was wrapping my head around the reason it doesn't :D

@encukou
Copy link
Member Author

encukou commented Sep 26, 2019

OK, merging now to try fixing the buildbots.

@encukou encukou merged commit 4267c98 into python:master Sep 26, 2019
@encukou encukou deleted the skip-long-path branch September 26, 2019 09:53
jacobneiltaylor pushed a commit to jacobneiltaylor/cpython that referenced this pull request Dec 5, 2019
…'t be created (pythonGH-16414)

This avoids the buildbot failure on Windows:
```
FileNotFoundError: [WinError 206] The filename or extension is too long: 'd:\\temp\\tmp5r3z438t\\long\\1\\2\\3\\4\\5\\6\\7\\8\\9\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\20\\21\\22\\23\\24\\25\\26\\27\\28\\29\\30\\31\\32\\33\\34\\35\\36\\37\\38\\39\\40\\41\\42\\43\\44\\45\\46\\47\\48\\49\\50\\51\\52\\53\\54\\55\\56\\57\\58\\59\\60\\61\\62\\63\\64\\65\\66\\67\\68\\69\\70\\71\\72\\73\\74\\75\\76\\77\\78'
```
Creates a path that's long but avoids OS restrictions.

https://bugs.python.org/issue38112
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants