Ballista: Finish implementing shuffle mechanism [DRAFT]

[andygrove]

Which issue does this PR close?

Closes #707 but is actually much larger than one issue because it turns out that the shuffle mechanism wasn't fully implemented and wasn't really being used.

With this PR, I now see the executors using hash partitioning in the shuffle writes.

=== Physical plan with metrics ===
ShuffleWriterExec: Some(Hash([Column { name: "l_orderkey", index: 0 }], 2)), metrics=[writeTime=21538883]
  CoalesceBatchesExec: target_batch_size=4096, metrics=[]

Other changes:

• Fixed bug where all shuffle tasks within the same query stage were writing to the same output files, causing corruption
• TaskStatus now includes task meta-data so that that the scheduler can pass the correct partition meta-data to shuffle readers (and skip empty partitions)

Remaining work:

• Get it all working

Rationale for this change

Shuffles were broken. The executor always ran the shuffle writes with partioning of None instead of the hash partitioning they were supposed to use. This information was never sent as part of the protobuf. Queries still worked correctly but this was not scaling since there was always a single partition.

What changes are included in this PR?

Are there any user-facing changes?

No

[andygrove]

@edrevo fyi