Commits on Feb 9, 2024

1. Fix HashBuild unspilling stuck (facebookincubator#8715) …

   Summary:
   
   When the input of `HashBuild` is from spilling, they all come from the
   same partition.  That means the spill partition bits are all same for the hashes
   from these rows.  In case the hash table is large, there could be overlap between the hash bits
   we use to calculate bucket index and the bits for spill partitioning.  These
   bits are fixed for all rows and because they are higher bits, we end up
   restricting ourselves to a smaller region of the hash table.  This results in
   heavy hash collision and the hash build will take super long time and block
   driver threads.
   
   Fix this by adding a check to make sure that there will be no overlap between
   the spill partitioning bits and the bits used for bucket indexing, and increase
   the default spill start partition bit to 48.
   
   Reviewed By: oerling
   
   Differential Revision: D53589502

   

   Yuhta authored and facebook-github-bot committed Feb 9, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 314f24c
   • Browse repository at this point

   Copy the full SHA

   314f24c View commit details

   Browse the repository at this point in the history