Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ARM CPU] ACL TBB scheduler #17445

Merged
merged 25 commits into from
Jul 19, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8e81a38
tbb experiment - old version
allnes May 9, 2023
5a81547
enable throughput mode
allnes May 9, 2023
04eb077
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 16, 2023
a25f2cc
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 19, 2023
b527a92
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 19, 2023
4a585b5
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 22, 2023
e9316b5
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 23, 2023
69efd08
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 23, 2023
607f56c
Merge branch 'master' into an/tbb_scheduler_exp
allnes May 23, 2023
871dd8d
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes May 25, 2023
c48bd28
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes Jun 1, 2023
7f20bf8
acl scheduler trnasform
allnes Jun 5, 2023
fa19a46
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes Jun 5, 2023
17926b5
Merge branch 'master' of https://github.com/openvinotoolkit/openvino …
allnes Jun 6, 2023
f455b09
new version tbb scheduler
allnes Jun 7, 2023
8280284
Merge remote-tracking branch 'origin/an/tbb_scheduler_exp' into an/tb…
allnes Jun 7, 2023
1dd7dac
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 3, 2023
d10234b
update scheduling
allnes Jul 3, 2023
50b3bc8
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 3, 2023
d60b8f0
enable multi-streaming
allnes Jul 3, 2023
4f03280
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 3, 2023
44d2b16
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 12, 2023
9d6eee7
change
allnes Jul 12, 2023
1f3a64d
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 18, 2023
369ebc0
Merge branch 'master' into an/tbb_scheduler_exp
allnes Jul 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update scheduling
  • Loading branch information
allnes committed Jul 3, 2023
commit d10234b3b860f931f5bb4a9f7d1af69a494da753
Original file line number Diff line number Diff line change
Expand Up @@ -26,34 +26,15 @@ void ACLScheduler::schedule(ICPPKernel *kernel, const Hints &hints) {
}

void ACLScheduler::schedule_op(ICPPKernel *kernel, const Hints &hints, const Window &window, ITensorPack &tensors) {
const Window & max_window = window;
const unsigned int num_iterations = max_window.num_iterations(hints.split_dimension());
const unsigned int num_threads = std::min(num_iterations, static_cast<std::uint32_t>(parallel_get_num_threads()));

if (!kernel->is_parallelisable() || num_threads == 1) {
ThreadInfo info;
info.cpu_info = &cpu_info();
kernel->run_op(tensors, max_window, info);
} else {
const unsigned int num_windows = num_threads;
std::vector<IScheduler::Workload> workloads(num_windows);
for (unsigned int t = 0; t < num_windows; t++) {
workloads[t] = [t, &hints, &max_window, &num_windows, &kernel, &tensors](const ThreadInfo &info) {
Window win = max_window.split_window(hints.split_dimension(), t, num_windows);
win.validate();
kernel->run_op(tensors, win, info);
};
}
run_workloads(workloads);
}
schedule_common(kernel, hints, kernel->window(), tensors);
dmitry-gorokhov marked this conversation as resolved.
Show resolved Hide resolved
}

void ACLScheduler::run_workloads(std::vector<arm_compute::IScheduler::Workload> &workloads) {
InferenceEngine::parallel_for(workloads.size(), [&] (int wid) {
ThreadInfo info;
info.cpu_info = &cpu_info();
info.num_threads = parallel_get_num_threads();
info.thread_id = parallel_get_thread_num();
info.thread_id = wid;
workloads[wid](info);
});
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ using namespace arm_compute;
class ACLScheduler final : public IScheduler {
public:
ACLScheduler();
~ACLScheduler() = default;
~ACLScheduler() override = default;
std::uint32_t num_threads() const override;
void set_num_threads(unsigned int num_threads) override;
void schedule(ICPPKernel *kernel, const Hints &hints) override;
Expand Down