rubocop --parallel failing with "negative array size"

[mark-young-atg]

Our github action to run rubocop started failing when parallel was upgraded from 1.25.1 to 1.26.0

This is the error seen

Run bundle exec rubocop --parallel
negative array size
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `initialize'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `new'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `create_workers'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:520:in `work_in_processes'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:291:in `map'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:235:in `each'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/runner.rb:103:in `warm_cache'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/runner.rb:76:in `run'
/home/runner/work/sm-web/sm-web/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command/execute_runner.rb:26:in `block in execute_runner'
...snip...

I presume this is because of the following change that uses Concurrent.available_processor_count.floor to determine the number of available processors if the gem concurrent-ruby is installed, which in my case it is.

I can see that I can use the environment variable PARALLEL_PROCESSOR_COUNT to override this behaviour for my configuration but I think there is an issue here to be understood and mitigated around the behaviour in github actions. Whilst I appreciate the incorrect value may come from concurrent-ruby, I think perhaps that parallel should handle a negative value in a different way. Perhaps by failing back to using Etc.nprocessors as the previous version did.

I have not created a PR for this because I felt it needed discussion and knowledge from those that understand parallel vastly more than I do.

[mark-young-atg]

As a further comment on this, I checked our Amazon AWS instance and the use of Concurrent.available_processor_count.floor looks as though it will cause problems by returning a negative number. This is what I see on our ECS instance:

irb(main):001> Concurrent.available_processor_count.floor
=> -1
irb(main):002> Etc.nprocessors
=> 2

[grosser]

yanked 1.26.0

…

On Thu, Aug 8, 2024 at 4:20 PM mark-young-atg ***@***.***> wrote: As a further comment on this, I checked our Amazon AWS instance and the use of Concurrent.available_processor_count.floor looks as though it will cause problems by returning a negative number. This is what I see on our ECS instance: irb(main):001> Concurrent.available_processor_count.floor => -1 irb(main):002> Etc.nprocessors => 2 — Reply to this email directly, view it on GitHub <#349 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACYZYBFXYCNBCDKSZIWBLZQN5CXAVCNFSM6AAAAABMGSC452VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZVHE2TGNJUG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ovsant]

Same issue

bundle exec rubocop --color
negative array size
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `initialize'
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `new'
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:581:in `create_workers'
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:520:in `work_in_processes'
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:291:in `map'
/app/vendor/bundle/ruby/3.3.0/gems/parallel-1.26.0/lib/parallel.rb:235:in `each'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/runner.rb:103:in `warm_cache'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/runner.rb:76:in `run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command/execute_runner.rb:26:in `block in execute_runner'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command/execute_runner.rb:52:in `with_redirect'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command/execute_runner.rb:25:in `execute_runner'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command/execute_runner.rb:17:in `run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/command.rb:11:in `run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli/environment.rb:18:in `run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli.rb:122:in `run_command'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli.rb:129:in `execute_runners'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli.rb:51:in `block in run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli.rb:81:in `profile_if_needed'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/lib/rubocop/cli.rb:43:in `run'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/exe/rubocop:19:in `block in <top (required)>'
/usr/local/lib/ruby/3.3.0/benchmark.rb:313:in `realtime'
/app/vendor/bundle/ruby/3.3.0/gems/rubocop-1.65.1/exe/rubocop:19:in `<top (required)>'
/app/vendor/bundle/ruby/3.3.0/bin/rubocop:25:in `load'
/app/vendor/bundle/ruby/3.3.0/bin/rubocop:25:in `<main>'
/usr/local/bundle/bin/ruby_executable_hooks:22:in `eval'
/usr/local/bundle/bin/ruby_executable_hooks:22:in `<main>'

[mark-young-atg]

Thank you very much @grosser. I think that was a sensible move.

[mark-young-atg]

@grosser although I think it might be sensible to fix forward by releasing v1.26.1 which reverts that commit. The reason being I'm still finding that bundle update and bundle install try to give me v1.26.0, possibly due to the gem still being installed locally on my laptop. Obviously I know to remove it with gem uninstall parallel:1.26.0 but this might catch out others and CI/CD systems.

[grosser]

Kk, will do

…

On Thu, Aug 8, 2024, 4:57 PM mark-young-atg ***@***.***> wrote: @grosser <https://github.com/grosser> although I think it might be sensible to fix forward by releasing v1.26.1 which reverts that commit. The reason being I'm still finding that bundle update and bundle install try to give me v1.26.0, possibly due to the gem still being installed locally on my laptop. Obviously I know to remove it with gem uninstall parallel:1.26.0 but this might catch out others and CI/CD systems. — Reply to this email directly, view it on GitHub <#349 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACYZ2VS2P65OEIKZCHBH3ZQOBOPAVCNFSM6AAAAABMGSC452VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZWGAZTSMBTGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grosser]

1.26.1 is out