terraform_deprecated_index
: improve perf for files with many expressions
#49
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A coworker at @observeinc alerted me to a module where new versions of TFLint were running extremely slowly. Some initial investigation revealed that
terraform_deprecated_index
was spending a really long time when there were a very large number of expressions to walk in a module. In this case, there were a few large JSON objects being encoded withjsonencode
and all the key/value pairs are expressions, though mostly static.# times out tflint --only terraform_deprecated_index
Most of that time was going into repeatedly calling
runner.GetFile()
, which in the example case was being called hundreds of times for the same file. There's no parser instance stored on the runner so eachGetFile
call meant the file was read from disk and parsed as HCL.Since
WalkExpressions
callsGetFiles()
anyway, calling it once before walking is a straightforward fix that brings the performance to an acceptable point. In the module in question, previously TFLint ran for multiple minutes on my Mac Studio at 200% CPU before I'd give up after 5m and interrupt it before it printed any output. After, it finishes in ~8s. Caching files after load would probably further boost performance by avoiding multiple rounds of loading from disk and parsing. The memory penalty would probably be worthwhile for the reduced CPU usage.