retdec-fileinfo runs for a minute and uses 110 GB of RAM to analyze a 41 MB PE file

[s3rvac]

retdec-fileinfo uses 110 GB of RAM to analyze a 41 MB PE file.

Steps to reproduce

On Linux, run

$ /usr/bin/time -v retdec-fileinfo FILE

where FILE is FE015EB24B7EEA2907698A6D7142198644A757066DA4EB8D3A4B63900008CF5E.

Expected output

The file is analyzed in a reasonable time and using a reasonable amount of memory.

Actual output

$ /usr/bin/time -v retdec-fileinfo FILE
[..]
        User time (seconds): 61.19
[..]
        Maximum resident set size (kbytes): 116301540

That is, retdec-fileinfo ran for over a minute and used around 110 GB of memory during the analysis.

Configuration

• Commit: 37dbfd1 (current master)
• 64b Debian Buster, GCC 9, Debug build of RetDec

[s3rvac]

Possibly related issue: #911.