refactor: Do in-place compact in Dump for csr

[zhanglei1949]

Fix #736

[Copilot]

Pull request overview

This PR addresses #736 by removing pre-dump CSR compaction in the graph/loader checkpoint flow and instead performing CSR normalization during CSR Dump(), enabling reuse of previously-dumped CSR files when the data is already clean/unchanged.

Changes:

• Removed unconditional PropertyGraph::Compact(...) calls before dumping checkpoints in loader and DB checkpoint publication paths.
• Refactored PropertyGraph compaction into an internal helper to support “checkpoint preparation” (schema/metadata work) without compacting edge CSRs.
• Added a shared csr_dump_utils.h helper and expanded storage tests to verify tombstone removal/timestamp normalization, compaction effects, and snapshot file reuse behavior.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/storage/test_open_graph.cc	Adds an end-to-end COPY → CHECKPOINT → reopen test to cover the initial-load checkpoint scenario from #736.
tests/storage/test_mutable_csr.cc	Adds tests ensuring Dump normalizes timestamps, removes tombstones/compacts, and reuses neighbor files when clean.
tests/storage/test_immutable_csr.cc	Adds tests for Dump compaction of deleted edges and file reuse, plus reopen/dump behavior across memory levels.
src/storages/loader/abstract_property_graph_loader.cc	Removes the pre-dump graph_.Compact(1) call so bulk-load dumps avoid unnecessary CSR compaction.
src/storages/graph/property_graph.cc	Introduces compact_internal(...) and updates DumpAndClear to prepare checkpoints without compacting edge CSRs.
src/storages/graph/edge_table.cc	Extends EdgeTable::Compact with a compact_csr flag; moves sort-key logic into SortByEdgeData.
src/storages/csr/mutable_csr.cc	Moves tombstone removal + timestamp normalization into Dump() and reuses snapshot files via shared dump utility.
src/storages/csr/immutable_csr.cc	Normalizes/compacts CSR data during Dump() and reuses degree/nbr files when unchanged via shared dump utility.
src/storages/csr/csr_dump_utils.h	New helper for hashing/rewriting-or-linking normalized CSR dump files based on stored MD5 headers.
src/main/neug_db.cc	Removes pre-dump live_graph->Compact(MAX_TIMESTAMP) before publishing checkpoints.
include/neug/storages/graph/property_graph.h	Declares compact_internal(...) as a private helper.
include/neug/storages/graph/edge_table.h	Updates EdgeTable::Compact signature to accept compact_csr (defaulting to true).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

tests/storage/test_mutable_csr.cc:1117

• inode_of() uses errno and std::strerror(errno) but this file does not include <cerrno> / <cstring>, which can make the test fail to compile depending on transitive includes. Either add the missing headers or avoid referencing errno/strerror here.

    if (stat(path.c_str(), &st) != 0) {
      throw std::runtime_error("stat() failed for path: " + path + " — " +
                               std::strerror(errno));
    }

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/storages/graph/edge_table.cc:1010

• EdgeTable::Compact() now always registers an atexit handler and records timing stats. This introduces non-trivial overhead in production and will emit profiler logs on every process exit after any compaction. Consider enabling this profiling only when explicitly requested (e.g. via an environment variable) and keep the default path free of atexit/global side effects.

void EdgeTable::Compact(const std::optional<std::string>& sort_key_for_nbr) {
  RegisterEdgeTableCompactProfilerOnFirstUse();
  GetEdgeTableCompactProfiler().RecordCompactCall();
  {
    EdgeTableCompactStageTimer timer(EdgeTableCompactStage::kOutCsrCompact);
    out_csr_->compact();
  }
  {
    EdgeTableCompactStageTimer timer(EdgeTableCompactStage::kInCsrCompact);
    in_csr_->compact();
  }
  if (sort_key_for_nbr.has_value()) {
    EdgeTableCompactStageTimer timer(EdgeTableCompactStage::kSortByEdgeData);
    SortByEdgeData(1);
  }

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.