Skip to content

Load test fixes#253

Open
al1img wants to merge 4 commits into
aosedge:developfrom
al1img:load_test
Open

Load test fixes#253
al1img wants to merge 4 commits into
aosedge:developfrom
al1img:load_test

Conversation

@al1img

@al1img al1img commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.11111% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@0bf748c). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/sm/launcher/runtimes/container/crunrunner.cpp 20.00% 4 Missing ⚠️
src/sm/launcher/runtimes/container/monitoring.cpp 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #253   +/-   ##
==========================================
  Coverage           ?   81.46%           
==========================================
  Files              ?      345           
  Lines              ?    34205           
  Branches           ?     3574           
==========================================
  Hits               ?    27865           
  Misses             ?     6340           
  Partials           ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

al1img added 4 commits July 2, 2026 11:58
Release Runner::mMutex before invoking RunStatusReceiverItf::UpdateRunStatus
to break an AB/BA lock order cycle:

- Thread pool workers hold Instance::mMutex (in Instance::Start/Stop) and
  block on Runner::mMutex (via Runner::StartInstance/StopInstance).
- Runner::MonitorContainers holds Runner::mMutex and blocks on the same
  Instance::mMutex, reached through UpdateRunStatus ->
  ContainerRuntime::UpdateRunStatus -> Instance::UpdateRunStatus.

Once triggered, Runner::mMutex is held forever, wedging every worker
thread as well as the shutdown path (Runner::Stop joins the monitoring
thread), so the whole aos_sm_app hangs, including on close.

MonitorContainers now copies the running-instances snapshot under the
lock, releases it, then invokes the receiver outside the critical
section, mirroring how instancesToRestart is already handled.

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
std::filesystem::path::operator/= replaces the whole path instead of
appending when the right-hand side is itself absolute (starts with
'/'), so JoinPath("/sys/fs/cgroup", "/system.slice/...", ...) silently
dropped "/sys/fs/cgroup" and returned just the cgroup path itself.
Every caller here means "append this component", so strip a leading
'/' from entry/entries (never from base) before appending.

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
mInstanceMonitoringCache (std::unordered_map) was read and mutated from
StartInstanceMonitoring, StopInstanceMonitoring, and
GetInstanceMonitoringData with no synchronization, all of which run
concurrently from the launcher's worker thread pool. Concurrent
insert/erase/lookup on the same unordered_map is undefined behavior.

Guard all three with a mutex.

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
@al1img al1img changed the title WIP Load test fixes Jul 2, 2026

@mlohvynenko mlohvynenko left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed-by: Mykhailo Lohvynenko <mykhailo_lohvynenko@epam.com>

@sonarqubecloud

sonarqubecloud Bot commented Jul 2, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants