[xray] Sets good object manager defaults. #2255
Conversation
|
Test PASSed. |
src/ray/raylet/main.cc
Outdated
|
|
||
| // This may be 0 when core detection fails. | ||
| int num_cores = std::thread::hardware_concurrency(); | ||
| object_manager_config.max_sends = std::max(1, num_cores / 4); |
There was a problem hiding this comment.
Let's leave this configurable and initialize with this variable in the RayConfig. Also we should use num_cpus as in e1024d8 and not the hardware_concurrency (first because a user may want to use only a fraction of a machine for ray and second because std::thread::hardware_concurrency seems to be less portable, as you point out it could even be 0).
| @@ -17,6 +17,7 @@ ObjectManager::ObjectManager(asio::io_service &main_service, | |||
| // release_delay of 2 * config_.max_sends is to ensure the pool does not release | |||
| // an object prematurely whenever we reach the maximum number of sends. | |||
| buffer_pool_(config_.store_socket_name, config_.object_chunk_size, | |||
| config_.max_sends, | |||
There was a problem hiding this comment.
Can we use max_chunks to make it consistent with object_buffer_pool.h.
There was a problem hiding this comment.
The configuration max_sends is used in other places in the code where the name max_sends makes more sense than max_chunks. I think keeping it max_sends here makes more sense.
|
Test PASSed. |
* 'master' of https://github.com/ray-project/ray: (157 commits) Fix build failure while using make -j1. Issue 2257 (ray-project#2279) Cast locator with index type (ray-project#2274) fixing zero length partitions (ray-project#2237) Make actor handles work in Python mode. (ray-project#2283) [xray] Add error table and push error messages to driver through node manager. (ray-project#2256) addressing comments (ray-project#2210) Re-enable some actor tests. (ray-project#2276) Experimental: enable automatic GCS flushing with configurable policy. (ray-project#2266) [xray] Sets good object manager defaults. (ray-project#2255) [tune] Update Trainable doc to expose interface (ray-project#2272) [rllib] Add a simple REST policy server and client example (ray-project#2232) [asv] Pushing to s3 (ray-project#2246) [rllib] Remove need to pass around registry (ray-project#2250) Support multiple availability zones in AWS (fix ray-project#2177) (ray-project#2254) [rllib] Add squash_to_range model option (ray-project#2239) Mitigate randomly building failure: adding gen_local_scheduler_fbs to raylet lib. (ray-project#2271) [rllib] Refactor Multi-GPU for PPO (ray-project#1646) [rllib] Envs for vectorized execution, async execution, and policy serving (ray-project#2170) [Dataframe] Change pandas and ray.dataframe imports (ray-project#1942) [Java] Replace binary rewrite with Remote Lambda Cache (SerdeLambda) (ray-project#2245) ...
Adds good defaults to ObjectManager.
send_max = receive_max = 1/4 num coresdefault_chunk_size = 1MBmax_sends.