urls being used as path

[ghost]

when i try to download via the command line, returns the error

C:\Users\mattl>gallery-dl https://twitter.com/veemogallery
[twitter][error] Unable to download data:  OSError: [WinError 123] La syntaxe du nom de fichier, de répertoire ou de volume est incorrecte: '\\\\?\\C:\\Users\\mattl\\gallery- dl\\twitter\\veemogallery\\\\'

this would translate to "The syntax of the file name, of directory or of volume is incorect: '\\?\C:\Users\mattl\gallery- dl\twitter\veemogallery\\'

[ghost]

actually a verbose would probably be more usefull

C:\Users\mattl>gallery-dl -v https://twitter.com/veemogallery
[gallery-dl][debug] Version 1.10.3-dev
[gallery-dl][debug] Python 3.6.7 - Windows-10-10.0.17134-SP0
[gallery-dl][debug] requests 2.20.1 - urllib3 1.24.3
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/veemogallery'
[gallery-dl][debug] Updating urllib3 ciphers
[twitter][debug] Using TwitterTimelineExtractor for 'https://twitter.com/veemogallery'
[twitter][error] Unable to download data:  OSError: [WinError 123] La syntaxe du nom de fichier, de répertoire ou de volume est incorrecte: '\\\\?\\C:\\Users\\mattl\\gallery-dl\\twitter\\veemogallery\\\\'
[twitter][debug]
Traceback (most recent call last):
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\site-packages\gallery_dl\job.py", line 48, in run
    self.dispatch(msg)
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\site-packages\gallery_dl\job.py", line 97, in dispatch
    self.handle_directory(msg[1])
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\site-packages\gallery_dl\job.py", line 251, in handle_directory
    self.initialize(keywords)
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\site-packages\gallery_dl\job.py", line 310, in initialize
    self.pathfmt.set_directory(keywords)
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\site-packages\gallery_dl\util.py", line 635, in set_directory
    os.makedirs(self.realdirectory, exist_ok=True)
  File "c:\users\mattl\appdata\local\programs\python\python36\lib\os.py", line 220, in makedirs
    mkdir(name, mode)
OSError: [WinError 123] La syntaxe du nom de fichier, de répertoire ou de volume est incorrecte: '\\\\?\\C:\\Users\\mattl\\gallery-dl\\twitter\\veemogallery\\\\'

[Hrxn]

Can you try it with gallery-dl -v --ignore-config <URL>?

[ghost]

it returns the exact same error

[f2d]

Just encountered this error in v1.10.2 (latest from pip) on Win7, uninstalled, reinstalled v1.10.1, working again.

At first I thought it was something bad with my path formats, but no.

[Hrxn]

Okay, seems like a regression then.

makedirs function:
https://github.com/python/cpython/blob/e64f948e762a6b9fd02e2902ccf42438df6fcb61/Lib/os.py#L196-L226

error raised here:

try:
    mkdir(name, mode)
except OSError:
    # Cannot rely on checking for EEXIST, since the operating system
    # could give priority to other errors like EACCES or EROFS
    if not exist_ok or not path.isdir(name):
        raise

Called here from utils.py

    # Enable longer-than-260-character paths on Windows
    if os.name == "nt":
        self.realdirectory = "\\\\?\\" + os.path.abspath(directory) + sep
    else:
        self.realdirectory = directory


    # Create directory tree
    os.makedirs(self.realdirectory, exist_ok=True)

Changes in 3284c62 and e77a656 , added in 0bb8737

Hm, what's with that additional trailing sep?

[mikf]

The additional sep was necessary in Python 3.7, the one version I tested with on Windows, because os.path.abspath(directory) removes any \ at the end of a path. It doesn't on Python 3.6, which resulted in two \ at the end and Windows doesn't like that, it seems.

[Hrxn]

Ah, okay. Yeah that explains why it was working for me..

So much for Windows being a "first class citizen" in Python, and already considered as "stable" and "reliable" 🙄

[mikf]

Oh, it is, as long as you stick to the standard library functions for path manipulation and so on. But I decided to spin my own solution and made too many assumptions along the way. Basically it's my own fault for "disobeying the rules", but os.path.join(a, b) for example is so much slower than a + b that I couldn't help it ... and lets ignore the fact that I shouldn't be caring about speed when using Python in the first place.

[Hrxn]

... and lets ignore the fact that I shouldn't be caring about speed when using Python in the first place.

True, true. But I guess the limiting factor in what gallery-dl actually does is basically always the speed of the packets crawling through those thick pipes towards your network interface. The language to solve that needs to be invented first..

And on the other hand, they apparently changed the behaviour of the abspath function in os.path between Python 3.6 and Python 3.7, so any reason notwithstanding, they deserve at least some of the blame. At least I feel that my feigned righteous indignation is somewhat justified.