Skip to content

external: normalize unambiguous file id lookup#2992

Open
cheese-cakee wants to merge 1 commit into
checkpoint-restore:criu-devfrom
cheese-cakee:fix/issue-2951-external-lookup
Open

external: normalize unambiguous file id lookup#2992
cheese-cakee wants to merge 1 commit into
checkpoint-restore:criu-devfrom
cheese-cakee:fix/issue-2951-external-lookup

Conversation

@cheese-cakee
Copy link
Copy Markdown
Contributor

@cheese-cakee cheese-cakee commented Mar 29, 2026

Why this PR is necessary

external_lookup_id() used strict string comparison, so logically identical file[mnt_id:inode] IDs could fail to
match when represented differently.

What this PR changes

  • Adds numeric normalization for file[mnt_id:inode] lookup in criu/external.c, with strict compatibility guards.
  • Keeps exact strcmp() match as the first path.
  • Applies normalization only when both components are unambiguous:
  • 0x... => hex
  • contains a-f/A-F => hex
  • digits-only => ambiguous, no normalization
  • Prevents ambiguous cross-matching (e.g. 11 vs 17) for backward compatibility.

Scope

  • Single logical change in criu/external.c only.

Testing

  • git diff --check HEAD~1..HEAD
  • python3 -m py_compile test/zdtm.py test/zdtm/criu_config.py

Copilot AI review requested due to automatic review settings March 29, 2026 16:34
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make external regular-file id matching more robust across equivalent numeric representations (decimal vs hex) and improve restore diagnostics when CRIU falls back from --inherit-fd to path-based reopen.

Changes:

  • Add normalization logic for file[mnt_id:inode] matching in external id lookup (keeping exact string compare as a fast path).
  • Emit a warning during restore when an external file has no --inherit-fd mapping and CRIU falls back to reopening by path.
  • Update CLI help and documentation to describe accepted numeric formats for file[mnt_id:inode].

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
criu/files-reg.c Adds a warning when --inherit-fd mapping is missing for an external file.
criu/external.c Introduces parsing/normalization for file[mnt_id:inode] during external id lookup.
criu/crtools.c Updates --external / --inherit-fd help text for file[mnt_id:inode] formats.
Documentation/criu.txt Clarifies documentation for --external file[mnt_id:inode] numeric formats.

Comment thread criu/crtools.c Outdated
Comment thread criu/external.c Outdated
Comment thread criu/external.c Outdated
Comment thread criu/crtools.c Outdated
Comment thread Documentation/criu.txt Outdated
@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch 2 times, most recently from 11276ef to 7bf7613 Compare March 29, 2026 16:52
@cheese-cakee
Copy link
Copy Markdown
Contributor Author

please review when possible @avagin

@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch from 7bf7613 to 0f87717 Compare April 16, 2026 16:05
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 16, 2026

Codecov Report

❌ Patch coverage is 93.75000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.26%. Comparing base (b7f6b72) to head (aeba1ac).
⚠️ Report is 736 commits behind head on criu-dev.

Files with missing lines Patch % Lines
criu/external.c 93.75% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           criu-dev    #2992      +/-   ##
============================================
- Coverage     57.76%   57.26%   -0.51%     
============================================
  Files           142      154      +12     
  Lines         37664    40486    +2822     
  Branches          0     8874    +8874     
============================================
+ Hits          21758    23185    +1427     
- Misses        15906    17037    +1131     
- Partials          0      264     +264     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@avagin
Copy link
Copy Markdown
Member

avagin commented Apr 16, 2026

@cheese-cakee please update the commit message with more details.

Comment thread criu/external.c Outdated
@avagin
Copy link
Copy Markdown
Member

avagin commented Apr 16, 2026

I get the goal here, but we need to be careful. Since the file[hex:hex] format lacks the 0x prefix, any hex value consisting only of numbers could be parsed as a decimal either...

@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch 2 times, most recently from f3b1e9f to 4499038 Compare April 16, 2026 18:11
@cheese-cakee
Copy link
Copy Markdown
Contributor Author

I get the goal here, but we need to be careful. Since the file[hex:hex] format lacks the 0x prefix, any hex value consisting only of numbers could be parsed as a decimal either...

I addressed this by making CRIU emit the 0x prefix (changed the snprintf in dump_one_reg_file from %x to 0x%x), so the format is unambiguous going forward. The parser now requires 0x for hex

@avagin
Copy link
Copy Markdown
Member

avagin commented Apr 16, 2026

I get the goal here, but we need to be careful. Since the file[hex:hex] format lacks the 0x prefix, any hex value consisting only of numbers could be parsed as a decimal either...

I addressed this by making CRIU emit the 0x prefix (changed the snprintf in dump_one_reg_file from %x to 0x%x), so the format is unambiguous going forward. The parser now requires 0x for hex

I don't understand how it will help us. We can't break existing users, so we need to proper handle the old format...

@rst0git
Copy link
Copy Markdown
Member

rst0git commented Apr 17, 2026

Adding a note that I believe this is also related to the comments in #2951 (comment) and #2973

Comment thread criu/external.c Outdated
int base;

/*
* 0x prefix means hexadecimal; everything else is decimal.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is the breaking change that Andrei is referring to. There is no reliable way for us to parse decimal numbers here. For example, 0x11 == 17. When 0x is removed, we have no way of knowing if the input is 11 or 17.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll drop the emission change and instead make the parser try decimal first (matching user expectation for --inherit-fd arguments) and fall back to hex only when the decimal parse fails

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, it can break backward compatibility. Please explain in the commit message in details why existing users will be not affected by this change. Try to think on a case when there are two external resources with ids '11' and '17'.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now normalization is applied only when both components (mnt_id and inode) are unambiguous:

  • 0x... => unambiguous hex
  • contains a-f/A-F => unambiguous hex
  • digits-only => ambiguous, no normalization

So file[11:17] and file[17:23] keep literal/exact semantics and won’t be cross-matched by numeric normalization.
external_lookup_id() still does exact strcmp() first, then numeric compare only for unambiguous IDs

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cheese-cakee Were you able to test these changes? It sounds overcomplicated, and I am not sure the logic is correct. Note that AI chatbots can make mistakes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested the new changes and I have tried to simplify the implementation :
Normalization only when both mnt_id AND inode are unambiguous:

  • 0x... or contains a-f → hex
  • digits-only (like 11) → treated as decimal, no normalization

please review if its alright

@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch 7 times, most recently from 43b95cb to 6409fc5 Compare April 27, 2026 20:04
@cheese-cakee cheese-cakee changed the title external: normalize file id lookup and warn on fallback reopen external: normalize unambiguous file id lookup Apr 27, 2026
@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch 4 times, most recently from ea45cc5 to 079e01b Compare April 28, 2026 08:01
external_lookup_id() uses strict string comparison, so file[0xa:0xa]
and file[10:10] (the same file) wont match when one side emits hex
and the other decimal.  Add a numeric comparison path, but only for
unambiguous forms to preserve backward compatibility.

The key insight from avagins review: a component like 11 is ambiguous
it could mean decimal 11 or hex (0x11 = 17 decimal). Without the
explicit 0x prefix, there is no way to know the users intent.
file[11:17] and file[17:23] could silently match incorrectly.

Parse rules:
- 0x... => unambiguous hex (explicit prefix)
- contains hex digits (a-f/A-F) => unambiguous hex
- digits only => ambiguous, do NOT normalize-match

Only normalize when BOTH the mnt_id AND inode components are unambiguous.
Ambiguous IDs retain their literal/exact semantics.

This narrow fix addresses backward compatibility concerns:
- CRIU emits bare hex from %x (e.g., file[a:a]): unambiguous, normalizes
- CRIU emits bare digits from %x (e.g., file[10:10]): ambiguous, doesnt normalize
- User provides file[0xa:0xa]: unambiguous, matches to stored file[0xa:0xa]
- User provides file[10:10]: ambiguous, matches to stored file[10:10]
- file[10:10] will NOT normalize-match file[0xa:0xa] because the latter is unambiguous

Fixes: checkpoint-restore#2951

Signed-off-by: Farzan Aman Khan <farzanaman99@gmail.com>
@cheese-cakee cheese-cakee force-pushed the fix/issue-2951-external-lookup branch from 079e01b to aeba1ac Compare April 28, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants