improve the LocalExecutor builders

[HippoBaro]

This PR makes a series of changes to the builders and placement
strategies:

• The Placement struct is renamed to PoolPlacement as it maps
  placements for a pool of executors.
• Embed the number of executors to create directly in the placement
  policy. Instead of having two orthogonal configurations for a
  LocalExecutorPoolBuilder(the number of executors to create and the CPU
  placement policy) that can conflict with each other (for instance if we
  provide a custom placement and a different number of shards to create),
  merge the two together. A PoolPlacement now also takes (when
  appropriate), the number of executors to create in addition to the
  CpuSet to use. The LocalExecutorPoolBuilder constructor now takes a
  PoolPlacement directly instead of the number of executors to create.
  This simplifies the API and makes it harder to misuse.
• Align the LocalExecutorBuilder API with the pooled version. The
  pooled version is presently strictly superior to the non-pooled one
  because it uses the PoolPlacement logic. It is therefore possible to
  spawn a single executor that's fenced to a collection of CPUs using the
  pooled API, but not with the non-pooled one. With the existing
  LocalExecutorBuilder, an executor is either pinned to a specific CPU,
  or to none at all. To remedy this, this commit creates a new Placement
  enum that contains a subset of placement strategies that make sense for
  a single executor. In turn, LocalExecutorBuilder::new now takes a
  Placement strategy and the pin_to_cpu API is removed.

[glommer]

@trtsl

[glommer]

I like the fact that Placement is now the way to specify cpus in the single executor case. The only reason why it wasn't done that way of course is that the single executor preceded Placement so the consistency is good.

I am not sure I like the name Exact. I think I would prefer Fixed. Maybe there is even a case to be made that this is really just a special case of Fenced. Happy to hear from others.

[davidblewett]

This looks good. There are some doc changes that need to be adjusted to the new PoolPlacement variant format.

[trtsl]

I think the changes make sense. Though I do have a couple of thoughts related to PoolPlacements / CpuSets that may be worth revisiting but are not directly related to this pull request:

• Given that the api now has several set operations, it may we worth having a closer look at PartialEq. It is possible for two placements to compare equal but for the pool builder to use different cpu sets (because it relies on iterating a hashset and the number of cpus requested may be less than the set size); it is also possible for cpus sets to compare not equal but run on the same cpus (when the cpu sets are different but the number of cpus selected is less than set size). Finally, Placement::Custom still relies on a Vec which is dependent on ordering. One option: it would be possible for the implementation of PartialEq to check what CpuSets will in fact be used and then return the result ... perhaps even allowing e.g. Placement::Custom to equal Placement::MaxSpread if all the CpuSets that the pool builder will use match.
• The changes from this weekend now allow construction of arbitrary CpuSets by the user (i.e. with invalid CpuLocations). An invalid CpuSet in the pool builder would likely fail silently.

[github-actions]

Greetings @HippoBaro!

It looks like your PR added a new or changed an existing dependency, and CI has failed to validate your changes.
Some possible reasons this could happen:

• One of the dependencies you added uses a restricted license. See deny.toml for a list of licenses we allow;
• One of the dependencies you added has a known security vulnerability;
• You added or updated a dependency and didn't update the LICENSE-3rdparty.csv file. To do so, run the following and commit the changes:

$ cargo install cargo-license
$ cargo license --all-features -a -j --no-deps -d | jq -r '(["Component","Origin","License","Copyright"]) as $cols | map(. as $row | ["name", "repository", "license", "authors"] | map($row[.])) as $rows | $cols, $rows[] | @csv' > LICENSE-3rdparty.csv

Thank you!

[HippoBaro]

The changes from this weekend now allow construction of arbitrary CpuSets by the user (i.e. with invalid CpuLocations). An invalid CpuSet in the pool builder would likely fail silently.

Yeah, this would be pretty bad. I've added a runtime check that asserts any CpuLocation we deal with is part of the CPU set available to the current process.

I haven't acted on the PartialEq feedback here because I feel this would deserve a dedicated PR.