Skip to content

[Caching] Encoding cache key for input file with index instead of path #70384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 4, 2024

Conversation

cachemeifyoucan
Copy link
Contributor

Avoid path encoding difference (for example, real_path vs. path from symlink) by eliminating the path from cache key. Cache key is now encoded with the index of the input file from all the input files from the command-line, reguardless if those inputs will produce output or not. This is to ensure stable ordering even the batching is different.

Add a new cache computation API that is preferred for using input index directly. Old API for cache key is deprecated but still updated to fallback to real_path comparsion if needed.

As a result of swift scan API change, rename the feature in JSON file to avoid version confusion between swift-driver and libSwiftScan.

rdar://119387650

@cachemeifyoucan
Copy link
Contributor Author

@swift-ci please smoke test

@cachemeifyoucan
Copy link
Contributor Author

@swift-ci please smoke test

@cachemeifyoucan
Copy link
Contributor Author

@swift-ci please smoke test

@cachemeifyoucan
Copy link
Contributor Author

@swift-ci please smoke test windows platform

@cachemeifyoucan
Copy link
Contributor Author

ping

@@ -40,7 +40,7 @@
"name": "ld-path-driver-option"
},
{
"name": "cache-compile-job"
"name": "compilation-caching"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a result of swift scan API change, rename the feature in JSON file to avoid version confusion between swift-driver and libSwiftScan.

Should we embed a version number into the dictionary instead of changing the string? That would make it more obvious which string is the new vs. old API.

Alternatively, could you just dlsym the new API and use it when available? That's what we've typically done for libclang

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the dlsym check in swiftDriver because it is getting bigger and bigger. Want to take this as a last chance to reset the feature before start compatibility support as I don't want to keep both code paths to support old and new API in swift driver.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any strong opinion going either way?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel strongly enough about this to block the change. If this is the last time we need to make an incompatible API change it's fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, one last time. Thanks!

Avoid path encoding difference (for example, real_path vs. path from
symlink) by eliminating the path from cache key. Cache key is now
encoded with the index of the input file from all the input files from
the command-line, reguardless if those inputs will produce output or
not. This is to ensure stable ordering even the batching is different.

Add a new cache computation API that is preferred for using input index
directly. Old API for cache key is deprecated but still updated to
fallback to real_path comparsion if needed.

As a result of swift scan API change, rename the feature in JSON file to
avoid version confusion between swift-driver and libSwiftScan.

rdar://119387650
@cachemeifyoucan
Copy link
Contributor Author

@swift-ci please smoke test

@cachemeifyoucan
Copy link
Contributor Author

ping

@cachemeifyoucan cachemeifyoucan merged commit e2f888b into swiftlang:main Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants