Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conversion between struct types from different packages breaks #310

Closed
dsnet opened this issue Apr 9, 2021 · 6 comments
Closed

conversion between struct types from different packages breaks #310

dsnet opened this issue Apr 9, 2021 · 6 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@dsnet
Copy link

dsnet commented Apr 9, 2021

Reproduction:

$ git clone https://go.googlesource.com/protobuf
$ cd protobuf/cmd/protoc-gen-go
$ garble build
DkrLcCpi.go:1: cannot use m.ProtoMethods() (type *struct { jumseyKf.HPJylc36; Ua060WY0 uint64; LEKZ3V0R func(struct { jumseyKf.HPJylc36; S620HKVo Uay7rngJ.S620HKVo; Ua060WY0 uint8 }) struct { jumseyKf.HPJylc36; LEKZ3V0R int }; JNbzrZiu func(struct { jumseyKf.HPJylc36; S620HKVo Uay7rngJ.S620HKVo; L4MzpBON []byte; Ua060WY0 uint8 }) (struct { jumseyKf.HPJylc36; L4MzpBON []byte }, error); J5afwwtO func(struct { jumseyKf.HPJylc36; S620HKVo Uay7rngJ.S620HKVo; L4MzpBON []byte; Ua060WY0 uint8; EePY1yG1 interface { FindExtensionByName(Uay7rngJ.DRlssGtN) (Uay7rngJ.MsEDzXI9, error); FindExtensionByNumber(Uay7rngJ.DRlssGtN, Dv5Rwe3w.Y3l8zppN) (Uay7rngJ.MsEDzXI9, error) } }) (struct { jumseyKf.HPJylc36; Ua060WY0 uint8 }, error); ZLLLzJFW func(struct { jumseyKf.HPJylc36; NTu9CQ_i Uay7rngJ.S620HKVo; OkxSwWgX Uay7rngJ.S620HKVo }) struct { jumseyKf.HPJylc36; Ua060WY0 uint8 }; T6X4Da5i func(struct { jumseyKf.HPJylc36; S620HKVo Uay7rngJ.S620HKVo }) (struct { jumseyKf.HPJylc36 }, error) }) as type *struct { jumseyKf.HPJylc36; Pn4K9ozd uint64; Zovpk5js func(struct { jumseyKf.HPJylc36; ASrqYrjP Uay7rngJ.S620HKVo; Pn4K9ozd uint8 }) struct { jumseyKf.HPJylc36; Zovpk5js int }; N3mROQGv func(struct { jumseyKf.HPJylc36; ASrqYrjP Uay7rngJ.S620HKVo; DUmLpA_6 []byte; Pn4K9ozd uint8 }) (struct { jumseyKf.HPJylc36; DUmLpA_6 []byte }, error); CufZlXhn func(struct { jumseyKf.HPJylc36; ASrqYrjP Uay7rngJ.S620HKVo; DUmLpA_6 []byte; Pn4K9ozd uint8; L6pqAGXN interface { FindExtensionByName(Uay7rngJ.DRlssGtN) (Uay7rngJ.MsEDzXI9, error); FindExtensionByNumber(Uay7rngJ.DRlssGtN, Dv5Rwe3w.Y3l8zppN) (Uay7rngJ.MsEDzXI9, error) } }) (struct { jumseyKf.HPJylc36; Pn4K9ozd uint8 }, error); ZegXs_lp func(struct { jumseyKf.HPJylc36; XjQ_oB6v Uay7rngJ.S620HKVo; YWo0sUBi Uay7rngJ.S620HKVo }) struct { jumseyKf.HPJylc36; Pn4K9ozd uint8 }; QCHUzV4r func(struct { jumseyKf.HPJylc36; ASrqYrjP Uay7rngJ.S620HKVo }) (struct { jumseyKf.HPJylc36 }, error) } in return argument

This occurs on protobuf@v1.26.0, but I doubt the exact version matters.

@mvdan
Copy link
Member

mvdan commented Apr 9, 2021

Tricky one. We hash field names depending on the package defining the type. Conversions from T1 to T2 will then break, if the two types are defined in different packages, as fields with the same name will end up having different names after obfuscation.

It was obvious we'd have this issue with exported method names, but I never thought about this edge case with exported field names.

@mvdan
Copy link
Member

mvdan commented Apr 9, 2021

Simpler repro:

$ cat f.go
package main

import (
	"fmt"
	"go/token"
)

type MyPosition struct {
	Filename string
	Offset   int
	Line     int
	Column   int
}

func main() {
	pos := token.Position{Filename: "foo.go"}
	fmt.Printf("%T\n", pos)

	myPos := MyPosition(pos)
	fmt.Printf("%T\n", myPos)
}
$ go run f.go
token.Position
main.MyPosition
$ garble build f.go
# command-line-arguments
qFYM5dnF.go:1: cannot convert pos (type token.Position) to type MyPosition

This example is even worse, because token.Position is not obfuscated yet MyPosition is.

@mvdan mvdan added this to the v0.3.0 milestone Apr 9, 2021
@mvdan mvdan added the bug Something isn't working label Apr 9, 2021
@capnspacehook
Copy link
Member

We could treat this like we treat reflected types, obfuscate exported struct fields by default unless the parent struct type is converted to/from another type...

@mvdan
Copy link
Member

mvdan commented Apr 10, 2021

We could, but it wouldn't catch all cases. A package A can convert from exported types B1.T1 and B2.T2, for example. The protobuf case is an example of what we wouldn't catch naively, because one of the types is declared in a dependency.

I think a first step here could be to use a global salt for exported field names, similar to what is proposed for methods in #3. Then the only remaining issue would be converting from a non-obfuscated type to an obfuscated type, but that's a bit of an open question in itself.

@dsnet
Copy link
Author

dsnet commented Apr 12, 2021

Could the field name be a hash of information from the field and the type itself? If the entropy there is too low, perhaps you can include the other fields in the struct since they need to be identical in order for a conversion to work, right?

@mvdan
Copy link
Member

mvdan commented Apr 13, 2021

you can include the other fields in the struct since they need to be identical in order for a conversion to work

That's a good point :) I'll give that a try.

@mvdan mvdan self-assigned this Apr 13, 2021
@mvdan mvdan changed the title compile error on interface method with complex unnamed type conversion between struct types from different packages breaks Apr 17, 2021
mvdan added a commit to mvdan/garble-fork that referenced this issue Apr 25, 2021
Packages P1 and P2 can define identical struct types T1 and T2, and one
can convert from type T1 to T2 or vice versa.

The spec defines two identical struct types as:

	Two struct types are identical if they have the same sequence of
	fields, and if corresponding fields have the same names, and
	identical types, and identical tags. Non-exported field names
	from different packages are always different.

Unfortunately, garble broke this: since we obfuscated field names
differently depending on the package, cross-package conversions like the
case above would result in typechecking errors.

To fix this, implement Joe Tsai's idea: hash struct field names with the
string representation of the entire struct. This way, identical struct
types will have their field names obfuscated in the same way in all
packages across a build.

Note that we had to refactor "reverse" a bit to start using transformer,
since now it needs to keep track of struct types as well.

This failure was affecting the build of google.golang.org/protobuf,
since it makes regular use of cross-package struct conversions.

Note that the protobuf module still fails to build, but for other
reasons. The package that used to fail now succeeds, so the build gets a
bit further than before. burrowers#240 tracks adding relevant third-party Go
modules to CI, so we'll track the other remaining failures there.

Fixes burrowers#310.
@lu4p lu4p closed this as completed in c9b0b07 Apr 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

3 participants