Use of Parallel.map with too many threads can lead to connection timeouts

[nicolasfranck]

Please complete the following fields as applicable:

Expected behaviour:

Actual behaviour:

Steps to reproduce:

• use a computer that has 2 or less CPU's
• run

$ git checkout tags/v2.2.1
$ bundle exec rake upgrade:v2_2_0_part2
"Upgrading to v2.2.0 (part 1) ... A summary report will be generated when complete"
"------------------------------------------------------------------------"
"Transferring existing org_identifiers over to the identifiers table"
"please wait ..."
"Transfer complete. Orginal org_identifier count 0, new identifiers count 0"
"--------------------------"
"Transferring existing user_identifiers over to the identifiers table"
"this may take in excess of 10 minutes depending on the size of your users table ..."
rake aborted!
ActiveRecord::ConnectionTimeoutError: could not obtain a database connection within 5.000 seconds (waited 5.003 seconds)
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:189:in `block in wait_poll'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:180:in `loop'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:180:in `wait_poll'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:135:in `block in poll'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:145:in `synchronize'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:133:in `poll'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:425:in `acquire_connection'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:349:in `block in checkout'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:348:in `checkout'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:263:in `block in connection'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:262:in `connection'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:571:in `retrieve_connection'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_handling.rb:113:in `retrieve_connection'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.11.3/lib/active_record/connection_handling.rb:87:in `connection'
/home/install/roadmap-ugent/lib/tasks/upgrade.rake:857:in `block (3 levels) in <top (required)>'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/parallel-1.19.2/lib/parallel.rb:508:in `call_with_index'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/parallel-1.19.2/lib/parallel.rb:361:in `block (2 levels) in work_in_threads'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/parallel-1.19.2/lib/parallel.rb:519:in `with_instrumentation'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/parallel-1.19.2/lib/parallel.rb:360:in `block in work_in_threads'
/home/install/roadmap-ugent/vendor/bundle/ruby/2.4.0/gems/parallel-1.19.2/lib/parallel.rb:211:in `block (4 levels) in in_threads'

If I reduce the number of threads on https://github.com/DMPRoadmap/roadmap/blob/v2.2.1/lib/tasks/upgrade.rake#L852 to 2 for example, it starts working. This is only one example where things go wrong like this. Make this configurable, or derive from the number of available cpu's?