Skip to content

DependenciesScanner: implement protocol for batch module dependencies scan #33569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 21, 2020

Conversation

nkcsgexi
Copy link
Contributor

This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.

This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.

… scan

This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.

This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.
@nkcsgexi
Copy link
Contributor Author

@swift-ci please smoke test

@nkcsgexi nkcsgexi requested review from DougGregor and artemcm August 20, 2020 21:09

/// Parse an entry like this, where the "platforms" key-value pair is optional:
/// {
/// "module": "Foo.pcm",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the only reason we include a file extension here to distinguish whether or not that's a Swift or Clang module? It seems that for either scanner we then strip it away. The file extension is kind of a downstream implementation detail here; sure, once we build them they end up being a .swiftmodule or a .pcm, but at the scanning stage we have neither and all we have to search by is a module name.

Do you think it would be cleaner to have the JSON key indicate what kind of module it is and leave module name value just have the name the scanners can use directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think adding a more explicit key is totally reasonable. Should we add a boolean value key isSwiftModule?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, either an additional boolean value, or have the module name key itself be either clangModuleName or swiftModuleName, kinda like this:

enum BatchScanModuleInfo {
  case swift(SwiftScanModuleInfo)
  case clang(ClangScanModuleInfo)
}
struct ClangScanModuleInfo: Codable {
  var clangModuleName: String
  var pcmArgs: [String]
  var outputPath: String
}
struct SwiftScanModuleInfo: Codable {
  var swiftModuleName: String
  var outputPath: String
}

Whichever makes more sense to deal with in the scanner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, That's a great idea. Will differentiate the key for a concise representation.

@nkcsgexi
Copy link
Contributor Author

@swift-ci please smoke test

@artemcm artemcm self-requested a review August 20, 2020 21:42
@nkcsgexi
Copy link
Contributor Author

@swift-ci Please smoke test OS X platform

2 similar comments
@nkcsgexi
Copy link
Contributor Author

@swift-ci Please smoke test OS X platform

@nkcsgexi
Copy link
Contributor Author

@swift-ci Please smoke test OS X platform

@nkcsgexi nkcsgexi merged commit 5a1f38b into swiftlang:master Aug 21, 2020
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 27, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Aug 28, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Sep 2, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
artemcm added a commit to artemcm/swift-driver that referenced this pull request Sep 4, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
xymus pushed a commit to xymus/swift-driver that referenced this pull request Nov 6, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
xymus pushed a commit to xymus/swift-driver that referenced this pull request Nov 6, 2020
…d PCM clang modele re-scan

swiftlang/swift#33569 introduced a batch dependency scanning mode to save up on the overhead of clang instance creation.
This patch switches `swift-driver` over to using the new batch mode instead of generating individual `-scan-clang-dependencies` jobs for each re-scan.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants