Regression in requirements.txt since 2.1.0

[egonk]

requirements.txt

2.0.3

>"\Users\egon\Downloads\osv-scanner_windows_amd64 (2).exe" scan source -L requirements.txt
Scanned C:\m\1\tests\requirements.txt file and found 17 packages
╭─────────────────────────────────────┬──────┬───────────┬──────────────┬─────────┬──────────────────╮
│ OSV URL                             │ CVSS │ ECOSYSTEM │ PACKAGE      │ VERSION │ SOURCE           │
├─────────────────────────────────────┼──────┼───────────┼──────────────┼─────────┼──────────────────┤
│ https://osv.dev/PYSEC-2023-254      │ 7.5  │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-jfhm-5ghh-2f97 │      │           │              │         │                  │
│ https://osv.dev/GHSA-3ww4-gg4f-jr7f │ 8.7  │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-5cpq-8wj7-hf2v │      │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-9v9h-cgj8-h64p │ 5.5  │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-jm77-qphf-c4w8 │      │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-v8gr-m533-ghj9 │      │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-w7pp-m8wf-vj6r │ 6.9  │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-x4qr-2fvf-3mr5 │ 7.4  │ PyPI      │ cryptography │ 3.3.2   │ requirements.txt │
│ https://osv.dev/GHSA-3rq5-2g8h-59hc │ 5.9  │ PyPI      │ dnspython    │ 1.14.0  │ requirements.txt │
│ https://osv.dev/PYSEC-2018-12       │ 6.1  │ PyPI      │ lxml         │ 3.6.0   │ requirements.txt │
│ https://osv.dev/GHSA-xp26-p53h-6h2p │      │           │              │         │                  │
│ https://osv.dev/PYSEC-2020-62       │ 6.1  │ PyPI      │ lxml         │ 3.6.0   │ requirements.txt │
│ https://osv.dev/GHSA-pgww-xf46-h92r │      │           │              │         │                  │
│ https://osv.dev/PYSEC-2021-19       │ 6.1  │ PyPI      │ lxml         │ 3.6.0   │ requirements.txt │
│ https://osv.dev/GHSA-jq4v-f5q6-mjqq │      │           │              │         │                  │
│ https://osv.dev/PYSEC-2022-230      │ 6.9  │ PyPI      │ lxml         │ 3.6.0   │ requirements.txt │
│ https://osv.dev/GHSA-wrxv-2j5q-m38w │      │           │              │         │                  │
│ https://osv.dev/GHSA-55x5-fj6c-h6m8 │ 8.2  │ PyPI      │ lxml         │ 3.6.0   │ requirements.txt │
│ https://osv.dev/PYSEC-2022-166      │      │ PyPI      │ paramiko     │ 2.7.1   │ requirements.txt │
│ https://osv.dev/GHSA-45x7-px36-x8w8 │ 5.9  │ PyPI      │ paramiko     │ 2.7.1   │ requirements.txt │
│ https://osv.dev/PYSEC-2018-28       │ 7.5  │ PyPI      │ requests     │ 2.9.1   │ requirements.txt │
│ https://osv.dev/GHSA-x84v-xcm2-53pg │      │           │              │         │                  │
│ https://osv.dev/PYSEC-2023-74       │ 6.1  │ PyPI      │ requests     │ 2.9.1   │ requirements.txt │
│ https://osv.dev/GHSA-j8r2-6x86-q33q │      │           │              │         │                  │
│ https://osv.dev/GHSA-9hjg-9r4m-mvj7 │ 5.3  │ PyPI      │ requests     │ 2.9.1   │ requirements.txt │
│ https://osv.dev/GHSA-9wx4-h78v-vm56 │ 5.6  │ PyPI      │ requests     │ 2.9.1   │ requirements.txt │
╰─────────────────────────────────────┴──────┴───────────┴──────────────┴─────────┴──────────────────╯

2.2.0 (and similar 2.1.0)

>"\Users\egon\Downloads\osv-scanner_windows_amd64.exe" scan source -L requirements.txt
unsupported file extension for requirements: .zip
(extracting as python/requirementsenhanceable) open m\1\tests\requirements.txt: invalid argument

Note the broken path with missing C:\. -L on Windows can be also a bit tricky, C:\path\path might be confused with extractor:path, the last time I checked I had to always pass extractor name on Windows with absolute paths.

[another-rex]

Quick question, what shell are you using? Currently running in powershell I cannot seem to replicate this issue. I suspect maybe your shell is expanding requirements.txt path to an absolute path?

Passing absolute paths and having it intepreted as a extractor is something we should fix either way though.

For now, can you try running it like this:

osv-scanner.exe scan source -L :requirements.txt 
# OR
osv-scanner.exe scan source -L :.\requirements.txt 

note the prepended :

[egonk]

@another-rex 2.2.2 works for me with absolute paths:

C:\Users\ekocj>downloads\osv-scanner_windows_amd64.exe -L c:\Users\ekocj\Downloads\go.mod
Scanned c:\Users\ekocj\Downloads\go.mod file and found 186 packages
No issues found

2.2.1 did not work

C:\Users\ekocj>"downloads\osv-scanner_windows_amd64 (1).exe" -L c:\Users\ekocj\Downloads\go.mod
could not determine extractor, requested c

That was fix #2188

[egonk]

@another-rex The problem with this specific requirements.txt is still there in 2.2.2 however, I used cmd.exe but the output is the same in powershell.

C:\Users\ekocj\Downloads>osv-scanner_windows_amd64.exe -L requirements.txt
unsupported file extension for requirements: .zip
(extracting as python/requirementsenhanceable) open Users\ekocj\Downloads\requirements.txt: invalid argument

C:\Users\ekocj\Downloads>osv-scanner_windows_amd64.exe -L :requirements.txt
unsupported file extension for requirements: .zip
(extracting as python/requirementsenhanceable) open Users\ekocj\Downloads\requirements.txt: invalid argument

C:\Users\ekocj\Downloads>osv-scanner_windows_amd64.exe -L :.\requirements.txt
unsupported file extension for requirements: .zip
(extracting as python/requirementsenhanceable) open Users\ekocj\Downloads\requirements.txt: invalid argument

C:\Users\ekocj\Downloads>osv-scanner_windows_amd64.exe -L c:\Users\ekocj\Downloads\requirements.txt
unsupported file extension for requirements: .zip
(extracting as python/requirementsenhanceable) open Users\ekocj\Downloads\requirements.txt: invalid argument

[another-rex]

Ah I see, I think it's something to do with transitive dependencies. I'll look into this further.

[another-rex]

Made the fix PR, in the mean time before the next fix is released, you can fix this by passing --no-resolve flag.

The .zip error is it trying to resolve a transitive dependency and failing (dnspython@1.14.0), and it is falling back to the non resolving extractor, but that fallback logic is failing.