[Data] RandomAccessDataset.multiget return unexpected values for missing keys. #44768
Open
Description
What happened + What you expected to happen
the ray.data.RandomAccessDataset.multiget expected return a None for missing records, in fact, I got an unexpected value for the missing key.
I find this PR update the _RandomAccessWorker.multiget: #24825, and it use the np.searchsorted to speed up the multiget, but the np.searchsorted will return the insertion points for missing records and it use the search result directly to get the row from the block without test col[i] == key, just like the code here:
ray/python/ray/data/random_access_dataset.py
Lines 266 to 269 in d8c7234
Versions / Dependencies
Ray: latest master
Python: 3.9.2
OS: linux
Reproduction script
import ray
import ray.data
kv_store = ray.data.from_items(
[i for i in range(0, 1000, 2)]
).repartition(5).to_random_access_dataset(key="item", num_workers=1)
print(ray.get(kv_store.get_async(1)), ray.get(kv_store.get_async(901)))
# output: None None
print(kv_store.multiget([1, 901]))
# output: [{'item': 2}, {'item': 902}]
Issue Severity
None