Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart

[SteveYurongSu]

No description provided.

[Copilot]

Pull Request Overview

This PR adds local persistence of pipe progress indexes during shutdown to enable accurate recovery after restarts.

• Exposes and propagates persistProgressIndex from task to runtime level
• Hooks into the shutdown sequence to trigger persistence via the task agent
• Implements persistAllProgressIndexLocally with locking and logging in the task agent

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
PipeTaskMeta.java	Changed persistProgressIndex visibility from private to public to allow external calls
PipeRuntimeMeta.java	Added persistProgressIndex to iterate over all task metas and persist progress
DataNodeShutdownHook.java	Calls persistAllProgressIndexLocally before shutting down the node
PipeDataNodeTaskAgent.java	Introduced persistAllProgressIndexLocally with a read lock and informational logging

Comments suppressed due to low confidence (5)

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:847

• [nitpick] Consider pluralizing 'progress index' to 'progress indexes' and rephrasing to 'Failed to persist progress indexes locally due to timeout.' for clarity.

      LOGGER.info("Failed to persist all progress index locally because of timeout.");

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:854

• [nitpick] Rephrase to 'Successfully persisted progress indexes locally.' to match plural usage and improve readability.

      LOGGER.info("Persist all progress index locally successfully.");

iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/agent/task/meta/PipeRuntimeMeta.java:143

• Add a JavaDoc comment explaining the purpose and thread-safety guarantees of this public method.

  public void persistProgressIndex() {

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:845

• Provide JavaDoc for this shutdown method describing its behavior, locking semantics, and any exceptions it may throw.

  public void persistAllProgressIndexLocally() {

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/DataNodeShutdownHook.java:121

• Fix grammar in the comment: change 'to accurate recovery' to 'to ensure accurate recovery' for clarity.

    // Persist progress index before shutdown to accurate recovery after restart

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

Project coverage is 39.11%. Comparing base (bbe031a) to head (766c8c1).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
...otdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java	0.00%	11 Missing ⚠️
.../commons/pipe/agent/task/meta/PipeRuntimeMeta.java	0.00%	5 Missing ⚠️
.../apache/iotdb/db/service/DataNodeShutdownHook.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #15779      +/-   ##
============================================
- Coverage     39.12%   39.11%   -0.01%     
  Complexity      198      198              
============================================
  Files          4837     4837              
  Lines        314486   314506      +20     
  Branches      39449    39452       +3     
============================================
- Hits         123033   123021      -12     
- Misses       191453   191485      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.