Skip to content

[SYCL] Fix race condition in submission time read/write #18716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions sycl/source/detail/device_impl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -331,12 +331,21 @@ uint64_t device_impl::getCurrentDeviceTime() {

// To account for potential clock drift between host clock and device clock.
// The value set is arbitrary: 200 seconds
std::shared_lock<std::shared_mutex> ReadLock(MDeviceHostBaseTimeMutex);
constexpr uint64_t TimeTillRefresh = 200e9;
assert(HostTime >= MDeviceHostBaseTime.second);
uint64_t Diff = HostTime - MDeviceHostBaseTime.second;

// If getCurrentDeviceTime is called for the first time or we have to refresh.
if (!MDeviceHostBaseTime.second || Diff > TimeTillRefresh) {
ReadLock.unlock();
std::unique_lock<std::shared_mutex> WriteLock(MDeviceHostBaseTimeMutex);
// Recheck the condition after acquiring the write lock.
if (MDeviceHostBaseTime.second && Diff <= TimeTillRefresh) {
// If we are here, it means that another thread has already updated
// MDeviceHostBaseTime, so we can just return the current device time.
return MDeviceHostBaseTime.first + Diff;
}
const auto &Adapter = getAdapter();
auto Result = Adapter->call_nocheck<UrApiKind::urDeviceGetGlobalTimestamps>(
MDevice, &MDeviceHostBaseTime.first, &MDeviceHostBaseTime.second);
Expand Down
3 changes: 2 additions & 1 deletion sycl/source/detail/device_impl.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

#include <memory>
#include <mutex>
#include <shared_mutex>
#include <utility>

namespace sycl {
Expand Down Expand Up @@ -2203,7 +2204,7 @@ class device_impl : public std::enable_shared_from_this<device_impl> {
// This is used for getAdapter so should be above other properties.
std::shared_ptr<platform_impl> MPlatform;

// TODO: Does this have a race?
std::shared_mutex MDeviceHostBaseTimeMutex;
std::pair<uint64_t, uint64_t> MDeviceHostBaseTime{0, 0};

const ur_device_handle_t MRootDevice;
Expand Down