Skip to content

Implement Swift serialization and deserialization of Clang types #29670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

rjmccall
Copy link
Contributor

@rjmccall rjmccall commented Feb 6, 2020

As part of this, we have to change the type export rules to prevent @convention(c) function types from being used in exported interfaces if they aren't serializable. This is a more conservative version of the original rule I had, which was to import such function-pointer types as opaque pointers. That rule would've completely prevented importing function-pointer types defined in bridging headers and so simply doesn't work, so we're left trying to catch the unsupportable cases retroactively. This has the unfortunate consequence that we can't necessarily serialize the internal state of the compiler, but that was already true due to normal type uses of aggregate types from bridging headers; if we can teach the compiler to reliably serialize such types, we should be able to use the same mechanisms for function types.

This PR doesn't flip the switch to use Clang function types by default, so many of the clang-function-type-serialization FIXMEs are still in place.

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test.

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test Linux Platform
Git Sha - 54e9dcec91e38fc54e7020584e465380cd70b82e

@varungandhi-apple
Copy link
Contributor

varungandhi-apple commented Feb 6, 2020

Could you elaborate on this with an example:

This has the unfortunate consequence that we can't necessarily serialize the internal state of the compiler, but that was already true due to normal type uses of aggregate types from bridging headers; if we can teach the compiler to reliably serialize such types, we should be able to use the same mechanisms for function types.

Are bridging headers special because they are not part of a module (I think?)?

Also, it feels like this knowledge of exceptional cases should be documented somewhere alongside the code itself, but I'm not entirely sure where.

@varungandhi-apple
Copy link
Contributor

Can you add a couple of test cases with -use-clang-function-types?

  1. serialization + llvm-bcanalyzer
  2. serialization + deserialization

[I don't think that requires SIL work to land, does it?]

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

Are bridging headers special because they are not part of a module (I think?)?

Right, in two different ways. They're not part of a module, so we have no way of figuring out where they're from; and that's supposed to be okay because you aren't supposed to use declarations from bridging headers in module interfaces, which is currently enforced by simply not allowing modules to have bridging headers except for the final executable (which then doesn't have an interface).

The general restriction should be enforced in the type-checker, and we want to do that as part of the implementation-only imports checking, but we haven't yet. I suspect it will look a lot like this check.

@rjmccall rjmccall force-pushed the ciliainig-teyepee-sieeriiieaileiizieaitieieoiniie branch from 54e9dce to a242ae3 Compare February 6, 2020 07:32
@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

Can you add a couple of test cases with -use-clang-function-types?

  1. serialization + llvm-bcanalyzer
  2. serialization + deserialization

[I don't think that requires SIL work to land, does it?]

I'll flesh out the serialization tests some more tomorrow. It shouldn't need SIL work.

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test OS X Platform
Git Sha - 54e9dcec91e38fc54e7020584e465380cd70b82e

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test Linux Platform
Git Sha - 54e9dcec91e38fc54e7020584e465380cd70b82e

@rjmccall rjmccall force-pushed the ciliainig-teyepee-sieeriiieaileiizieaitieieoiniie branch from a242ae3 to f344d0a Compare February 6, 2020 18:35
@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test Linux Platform
Git Sha - a242ae3dadba92f2e14d65103da180407e302ec6

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test OS X Platform
Git Sha - a242ae3dadba92f2e14d65103da180407e302ec6

@rjmccall rjmccall force-pushed the ciliainig-teyepee-sieeriiieaileiizieaitieieoiniie branch from f344d0a to cf8be9e Compare February 6, 2020 21:29
@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test.

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test Linux Platform
Git Sha - f344d0a40b7fb97d6e7651356582ffb69ee83394

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2020

Build failed
Swift Test OS X Platform
Git Sha - f344d0a40b7fb97d6e7651356582ffb69ee83394

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 6, 2020

@swift-ci Please test OS X

@varungandhi-apple
Copy link
Contributor

I've mostly looked at everything except Serializability.cpp -- I'll go over that tomorrow or later today.

As part of this, we have to change the type export rules to
prevent `@convention(c)` function types from being used in
exported interfaces if they aren't serializable.  This is a
more conservative version of the original rule I had, which
was to import such function-pointer types as opaque pointers.
That rule would've completely prevented importing function-pointer
types defined in bridging headers and so simply doesn't work,
so we're left trying to catch the unsupportable cases
retroactively.  This has the unfortunate consequence that we
can't necessarily serialize the internal state of the compiler,
but that was already true due to normal type uses of aggregate
types from bridging headers; if we can teach the compiler to
reliably serialize such types, we should be able to use the
same mechanisms for function types.

This PR doesn't flip the switch to use Clang function types
by default, so many of the clang-function-type-serialization
FIXMEs are still in place.
@rjmccall rjmccall force-pushed the ciliainig-teyepee-sieeriiieaileiizieaitieieoiniie branch from cf8be9e to faee21b Compare February 7, 2020 03:09
@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 7, 2020

@swift-ci Please smoke test

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 7, 2020

@swift-ci Please smoke test OS X

@rjmccall
Copy link
Contributor Author

rjmccall commented Feb 7, 2020

I've mostly looked at everything except Serializability.cpp -- I'll go over that tomorrow or later today.

I'm going to merge, but if you have further suggestions, I'd be happy to apply them in a follow-up.

@rjmccall rjmccall merged commit b469389 into swiftlang:master Feb 7, 2020
@rjmccall rjmccall deleted the ciliainig-teyepee-sieeriiieaileiizieaitieieoiniie branch February 7, 2020 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants