Skip to content

cycle in requirements.txt causes infinite recursion #354

Closed

Description

The logic that handles referring to other requirement files can recurse infinitely (or at least until the stack hits the goroutine 1Gb limit).

func ParseRequirementsTxt(pathToLockfile string) ([]PackageDetails, error) {
packages := map[string]PackageDetails{}
file, err := os.Open(pathToLockfile)
if err != nil {
return []PackageDetails{}, fmt.Errorf("could not open %s: %w", pathToLockfile, err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := removeComments(scanner.Text())
if strings.HasPrefix(line, "-r ") {
details, err := ParseRequirementsTxt(
filepath.Join(filepath.Dir(pathToLockfile), strings.TrimPrefix(line, "-r ")),
)

Consider the following requirements.txt:

-r requirements.txt
iterpipes3==0.4
numpy==1.16.4

Running osv-scanner on this file will crash the program (and my VM when trying to replicate)

runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

For what it's worth, it's a problematic requirements.txt for pip too:

pip install -r requirements.txt 
ERROR: Exception:
Traceback (most recent call last):
...
RecursionError: maximum recursion depth exceeded while calling a Python object

Would some sort of cycle detection in osv-scanner make sense? Or is this too much work to handle what is essentially bad input?
The alternative on my end is just removing the offending repo from my dataset, which I'm fine doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions