-
Notifications
You must be signed in to change notification settings - Fork 416
Replace SignerFactory with global protobuf handlers for keys #734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
f70bd48 to
0e66d19
Compare
|
Judging by Travis, it looks like this PR might be responsible for the slow integration tests I reported in #733. I'm not sure where the performance issue is yet though - I'll investigate shortly. |
0e66d19 to
db67304
Compare
db67304 to
b353352
Compare
|
It turned out not to be a performance issue, but rather a sequencing pass failure that wasn't being (visibly) reported (see #738). I've now fixed that. |
b660ac7 to
1dd0803
Compare
codingllama
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sending the comments I have so far. Github is warning me my page is out of date, so I'll refresh and resume the review.
cmd/createtree/main.go
Outdated
| maxRootDuration time.Duration | ||
| privateKeyType, pemKeyPath, pemKeyPass, pkcs11ConfigPath string | ||
| } | ||
| var keyHandlers = make(map[string]func() (proto.Message, error)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI: I found it a bit odd that keyHandlers was declared here, but initialized elsewhere. There's more at a further comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's because one of the handlers comes from a file that is conditionally built (if "pkcs11" build tag is used), so it can't be populated right here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Were we going to move keyHandlers to their own file or package?
cmd/createtree/main_test.go
Outdated
| wantTree: &nonDefaultTree, | ||
| }, | ||
| { | ||
| desc: "defaultOptsOnly", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this an error test because the private key won't be set? Maybe rename to "mandatoryOptsNotSet", or add a comment? At a glance it doesn't look different from "validOpts", it only became clearer after I realized runTest "fixes" a few flags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's correct. I'll rename it.
cmd/createtree/main_test.go
Outdated
| return | ||
| } | ||
|
|
||
| if diff := pretty.Compare(tree, test.wantTree); diff != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use proto.Equal() when comparing protos, pretty.Compare doesn't know about internal proto fields, so it may diff in cases where protos are meant to be equal.
The suggested idiom is:
if !proto.Equal(pb1, pb2) {
diff := pretty.Compare(pb1, pb2)
t.Errorf("%v: post-Foo diff (-got +want):\n%v", test.desc, diff)
}(Yep, it's a bit verbose, I know :/ )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's not actually new code (the indentation has just changed), but I'll fix it up nonetheless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
cmd/createtree/pem.go
Outdated
| ) | ||
|
|
||
| func init() { | ||
| keyHandlers["PEMKeyFile"] = func() (proto.Message, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer keyHandlers to be declared and initialized in the same file. I would also prefer all flags to be defined together (at main.go), but since this is in the main package we're not breaking the rule too much.
Suggestion: for the sake of isolation / cleaner interface between createtree and key logic, we could do something like:
At createtree/keys/keys.go:
package keys
type handler func() (proto.Message, error)
func SetHandler(name string, fn handler) { ... }
func Handler(name string) (handler, bool) { ... }
func NewPEMKeyHandler(path, pass) handler { ... }At createtree/main.go:
func main() {
keys.RegisterHandler("PEMKeyFile", keys.NewPEMKeyHandler(*path, *pass))
// ...
}WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
keyHandlers can't be initialized in the same file, because the PKCS#11 handler needs to be behind a build tag. It needs to use the global registry / auto-registration pattern. I can move this registry out of main.go and into its own package though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we figured out a way for everything except the RegisterHander function to be in it's own package?
| } | ||
| wantTree.PrivateKey = keyProto | ||
|
|
||
| runTest(t, []*testCase{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we have runTest and testCase moved to a separate, test-library like file? I'd be ok with a testonly package inside createtree, for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'll then lose access to the flags and won't be able to call createTree(). I can put them in their own file, but not their own package.
|
|
||
| runTest(t, []*testCase{ | ||
| { | ||
| desc: "empty pemKeyPath", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is tested in an integration-like style, where we exercise it via createtree, I'd be tempted to use real handlers at main_test.go and move those test cases there. Or, possibly, just test the handlers directly here. If you take the comment on package separation, it'll force you down one of those roads regardless (which why I like it).
| // defer flagsaver.Save().Restore() | ||
| // // Test code that changes flags | ||
| // } // flags are reset to their original values here. | ||
| package flagsaver |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
flagsaver isn't a command we run, so it shouldn't be placed inside cmd/. It definitely deserves its own package, though. Maybe util/flagsaver?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
| import "flag" | ||
|
|
||
| // Stash holds flag values so that they can be restored at the end of a test. | ||
| type Stash struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe type Stash map[string]string ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would preclude extending this type in the future and expose the implementation detail that it stores a map of flag names to values. The only reason I can see to do this is to allow the user to examine stashed flag values - do you see that being a use case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It works fine as a struct, but since you don't have other fields an alias could have worked too. It's fine as it is.
cmd/flagsaver/flagsaver_test.go
Outdated
| // Only a subset of the possible flag types are currently tested. | ||
| func TestRestore(t *testing.T) { | ||
| tests := []struct { | ||
| // Test name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: "// name is ..."
Ditto for the other comments, although I think they could all be removed.
nit2: "desc string"? That seems to be the recommended format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nits addressed, but left the comments, since I'm not sure oldValue and newValue are completely self-explanatory (perhaps I just need better names?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
cmd/flagsaver/flagsaver_test.go
Outdated
| // The value the flag should have just before being restored to oldValue. | ||
| newValue string | ||
| }{ | ||
| {"RestoreDefaultIntValue", "int_flag", "", "666"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Please use field labels. It's clearer, saves you from specifying defaults, and saves us from having to touch all structs if a new field is added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
codingllama
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the future consider breaking PRs like this into smaller steps. It's a bit hard to figure out what changed and what was moved, since Github doesn't help much to identify the latter.
crypto/keys/der.go
Outdated
| return nil, fmt.Errorf("der: unsupported private key type: %T", key2) | ||
| } | ||
|
|
||
| key3, err3 := x509.ParseECPrivateKey(keyDER) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the ordering here in any way important? Should we try EC first (assuming it is / will be the most popular one)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, these can be re-ordered.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re-ordered to try EC first.
crypto/keys/der_test.go
Outdated
| sf.AddHandler(&keyspb.PrivateKey{}, NewFromPrivateKeyProto) | ||
|
|
||
| for _, test := range []struct { | ||
| name string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: s/name/desc? Same for others (if you take it, ofc).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
crypto/keys/der_test.go
Outdated
| wantErr: true, | ||
| }, | ||
| } { | ||
| signer, err := sf.NewSigner(context.Background(), test.keyProto) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: Declare ctx outside the loop and reuse it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
|
|
||
| sf := NewSignerFactory() | ||
| sf.AddHandler(&keyspb.PrivateKey{}, NewFromPrivateKeyProto) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we test the handler for *keyspb.Specification? Ditto for the other public functions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm planning on writing tests for all of the public functions in a follow-up PR - it that ok? I haven't added any new public functions, other than NewPrivateKeyProtoFromSpec(), that don't have tests (I'll add a test for that now).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added test for NewPrivateKeyProtoFromSpec().
crypto/keys/keys_test.go
Outdated
| } | ||
|
|
||
| func TestGenerateKey(t *testing.T) { | ||
| func TestNewFromSpec(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shuffled some things around, but tests seem to have lagged behind. Could we make sure that the tests are in the matching _test.go file, compared to the one the function is defined?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done for TestNewFromSpec. I was intending to do this more thoroughly in the follow-up CL by replacing TestLoadPrivateKeyAndSign with individual tests for each of the public funcs (and put those tests in the appropriate _test.go file).
|
|
||
| func TestPkcs11(t *testing.T) { | ||
| // PKCS11Config support is tested by integration/log_integration.sh (when $WITH_PKCS11 == "true"). | ||
| t.Skip("Only integration testing is implemented for PKCS#11") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we have a simple, happy path test? If we disable integration testing for some reason this will go without coverage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be difficult because the PKCS#11 code calls out to https://github.com/letsencrypt/pkcs11key which attempts to load a dynamic library and communicate with a PKCS#11 interface. I could expose a package variable that allows the pkcs11key.New() call to be overridden during the tests, but that's about the only way I can think of making it unit testable.
At least by now having a skipped test for this, there is some (minimal) visibility of the fact that it lacks unit tests.
crypto/keys/signer_factory.go
Outdated
| Generate ProtoGenerator | ||
|
|
||
| // handlers convert a protobuf message into a crypto.Signer. | ||
| handlers map[string]ProtoHandler |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you considered making handlers global? Is there a situation where having factory-scoped handlers is useful?
A global handler registry would allow each handler-defining package to register its own handlers at init-time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's what I started with, but...
- I've known people to be opposed to globals (e.g. @daviddrysdale lamented the scattering of flags through the codebase).
- I preferred the explicitness of adding a handler, rather than it being a side-effect of importing a package, since it's quite an important decision - do you really want unencrypted keys in your storage, for instance? It also means it's easier to control which handlers are available to tests.
- Scoping it to a SignerFactory means you can start servers/signers in-process (like the integration tests do) and not be forced to share the same set of handlers.
Having said that, I do recognise that making the handlers global and having them register on package init is a more common pattern. I can put together a commit that makes that change and see how it compares if you like?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We didn't seem to shy away from global state in other places, like the hashers registry, and an init-based registry would reduce the number of +build pkcs11 files reaching for signerFactories, so it seemed worth to suggest it.
That said, your counter-arguments and the wariness about global state are certainly strong points (I found number "2" above particularly convincing). I'm happy going forward the way it is. Thanks for the detailed response, Rob.
crypto/keys/signer_factory.go
Outdated
| func (f SignerFactory) AddHandler(keyProto proto.Message, handler ProtoHandler) { | ||
| keyProtoType := proto.MessageName(keyProto) | ||
|
|
||
| if _, alreadyExists := f.handlers[keyProtoType]; alreadyExists { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/alreadyExists/ok/, so we don't deviate from the idiom?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't entirely convinced that "ok" was an appropriate name here, as it's semantically the opposite - if it already exists, that's not really ok. Originally I was returning an error here, which meant it was very much not ok. Perhaps that doesn't matter though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK ;)
crypto/signatures.go
Outdated
| ) | ||
|
|
||
| // SignatureAlgorithm returns the algorithm used for this public key. | ||
| // Only ECDSA and RSA keys are supported. Other key types will return sigpb.DigitallySigned_UNKNOWN. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/sigpb.DigitallySigned_UNKNOWN/sigpb.DigitallySigned_ANONYMOUS/
In a somewhat related comment, is anonymous ever valid / allowed? Should we differentiate between anonymous and unknown? (This is possibly out of scope.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is intended to mirror the DigitallySigned struct in RFC5246, which is why there isn't an "unknown" value. There are cases where anonymous is valid (see that RFC), but never in Trillian. Arguably, this should return an error rather than sigpb.DigitallySigned_ANONYMOUS.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed typo.
| req *trillian.CreateTreeRequest | ||
| createErr error | ||
| commitErr, wantCommit bool | ||
| wantErr string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Asserting on error strings is likely to make tests brittle. Ideally, we'd use something more resilient to small changes, like error codes, but I that won't help if the codes are all the same. OTOH, it does make it easier to reason that the error is what you think it is, and not some problem.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, and never usually check the error string but, in this case, there are a lot of different errors that could occur. I really want to know that the test is failing for the reason I think it is, rather than because the test is misconfigured, for example. Error codes would be a better solution except, as you suggest, I don't think they'd actually be different for most of these test cases (they're almost all cases of invalid arguments). Maybe that would be good enough for distinguishing the intended type of error from unintended errors though?
|
If you like, I think I could break this PR in two, with the first just doing some moving around of code? |
|
No need to break it now, I'll review the fixes commit-by-commit from now on. Just give me nudge when you finish replying to the comments. |
e23c9e8 to
98bf15b
Compare
cmd/createtree/pem.go
Outdated
| }, nil | ||
| } | ||
|
|
||
| keyHandlers["PrivateKey"] = func() (proto.Message, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider making each of these anonymous functions real functions for ease on the eye and ease of testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
82cdaf1 to
0cd23c3
Compare
|
I've just completed some changes @gdbelvin requested offline, which swaps over to a global registry of key handlers with auto-registration on import and ultimately deletes |
b57fa08 to
7905ec8
Compare
cmd/createtree/main.go
Outdated
| } | ||
| var keyHandlers = make(map[string]func() (proto.Message, error)) | ||
|
|
||
| func createTree(ctx context.Context, opts *createOpts) (*trillian.Tree, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great change of replacing opts with the relevant flags, but for the clarity of this already large PR, could I suggest that it be a separate PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, it'll have to merge first though, since having the PKCS#11 flag behind a build tag is dependent on createOpts behind removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in #788.
cmd/createtree/main.go
Outdated
| maxRootDuration time.Duration | ||
| privateKeyType, pemKeyPath, pemKeyPass, pkcs11ConfigPath string | ||
| } | ||
| var keyHandlers = make(map[string]func() (proto.Message, error)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Were we going to move keyHandlers to their own file or package?
cmd/createtree/main.go
Outdated
| } | ||
| pemSigner, err := keys.NewFromPrivatePEMFile( | ||
| opts.pemKeyPath, opts.pemKeyPass) | ||
| func newPK(format string) (*any.Any, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function should probably live in it's own file or package along with the keyHandlers
Also consider a clearer function name, say NewPublicKeyProto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
cmd/createtree/main.go
Outdated
| pemKeyPath: *pemKeyPath, | ||
| pemKeyPass: *pemKeyPassword, | ||
| pkcs11ConfigPath: *pkcs11ConfigPath, | ||
| func keyFormats() []string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A function comment would be great
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done (though it's now called RegisteredTypes() in the createtree/keys package).
| pemKeyTree.PrivateKey, err = ptypes.MarshalAny(&keyspb.PEMKeyFile{ | ||
| Path: pemPath, | ||
| Password: pemPassword, | ||
| runTest(t, []*testCase{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you'd like to run the test in a sub function, consider using a proper SubTest
https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks
for _, tc := range []struct{}{}{
t.Run("A=1", func(t *testing.T) { ... })
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure this is so much a subtest as just delegating the actual test logic to a helper. However, I wonder if we should be using subtests for all of our tabular tests? It'd save having to prepend the test case name to every failure message, and defer would work nicely (running at the end of each test case, rather than at the end of the entire test).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The convention seems to prefer "regular" table-driven, but it may be because t.Run is a relatively recent addition, iirc. Maybe we should take this question to stylists? Meanwhile, I'd vouch for the simplest approach (no t.Run unless you really need it).
Edit: spelling
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I consulted a stylist, and they too leaned away from using subtests, but mostly because they are quite new and rarely used so far. I've decided to use them, but inside of runTest(), where it was already using an anonymous function to isolate each test case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #788 for the use of subtests.
| // the privateKeyFormat flag to "test". It also registers a "test" key handler. | ||
| // Finally, it calls the test's setFlags func (if provided) to allow it to | ||
| // change flags specific to the test, then performs the test. | ||
| func runTest(t *testing.T, tests []*testCase) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a slightly more descriptive name would be great
| // the privateKeyFormat flag to "test". It also registers a "test" key handler. | ||
| // Finally, it calls the test's setFlags func (if provided) to allow it to | ||
| // change flags specific to the test, then performs the test. | ||
| func runTest(t *testing.T, tests []*testCase) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a slightly more descriptive name would be great - and the test should probably only run one test case at a time
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are only run one at a time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah you mean that it should take a single testCase, rather than an array.
cmd/createtree/pem.go
Outdated
| ) | ||
|
|
||
| func init() { | ||
| keyHandlers["PEMKeyFile"] = func() (proto.Message, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we figured out a way for everything except the RegisterHander function to be in it's own package?
1ba8a68 to
4ddfb8f
Compare
codingllama
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Travis looks unhappy, but other than that LGTM.
Is this one of the cases where we Travis won't pass because of the google/trillian <-> google/ct-go cycle?
| pemKeyTree.PrivateKey, err = ptypes.MarshalAny(&keyspb.PEMKeyFile{ | ||
| Path: pemPath, | ||
| Password: pemPassword, | ||
| runTest(t, []*testCase{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The convention seems to prefer "regular" table-driven, but it may be because t.Run is a relatively recent addition, iirc. Maybe we should take this question to stylists? Meanwhile, I'd vouch for the simplest approach (no t.Run unless you really need it).
Edit: spelling
crypto/keys/der/der_test.go
Outdated
|
|
||
| func TestFromProto(t *testing.T) { | ||
| // ECDSA private key in DER format. | ||
| keyDER, err := base64.StdEncoding.DecodeString("MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgS81mfpvtTmaINn+gtrYXn4XpxxgE655GLSKsA3hhjHmhRANCAASwBWDdgHS04V/cN0LZgc8vZaK4I1HWLLCoaOO27Z0B1aS1aqBE7g1Oo8ldSCBJAvee866kcHhZkVniPdCG2ZZG") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: Move the DER string to a const?
crypto/keys/der/der_test.go
Outdated
| } | ||
|
|
||
| if err := testonly.CheckKeyMatchesSpec(key, test.keySpec); err != nil { | ||
| t.Errorf("%v: NewProtoFromSpec() => %v", test.desc, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/NewProtoFromSpec/CheckKeyMatchesSpec/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
| func TestPrivateKeyProtoHandler(t *testing.T) { | ||
| func TestProtoHandler(t *testing.T) { | ||
| // ECDSA private key in DER format. | ||
| keyDER, err := base64.StdEncoding.DecodeString("MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgS81mfpvtTmaINn+gtrYXn4XpxxgE655GLSKsA3hhjHmhRANCAASwBWDdgHS04V/cN0LZgc8vZaK4I1HWLLCoaOO27Z0B1aS1aqBE7g1Oo8ldSCBJAvee866kcHhZkVniPdCG2ZZG") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto (move to const suggestion)
crypto/keys/generate_test.go
Outdated
| } | ||
|
|
||
| if err := testonly.CheckKeyMatchesSpec(key, test.keySpec); err != nil { | ||
| t.Errorf("%v: NewFromSpec(): %v", test.desc, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/NewFromSpec/CheckKeyMatchesSpec/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
server/trillian_log_server/main.go
Outdated
| _ "github.com/go-sql-driver/mysql" // Load MySQL driver | ||
|
|
||
| // Register key ProtoHandlers | ||
| _ "github.com/google/trillian/crypto/keys/der/proto" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: I've been splitting imports in the following blocks: golang, external libs, aliases and _ imports. I've found it more readable than mixing aliases and _ imports on the "external" block.
Eg:
import (
"context"
"flag"
"github.com/golang/glog"
"github.com/golang/protobuf/proto"
"github.com/google/trillian"
"github.com/google/trillian/cmd"
mysqlq "github.com/google/trillian/quota/mysql"
_ "github.com/google/trillian/merkle/objhasher" // Load hashers
_ "github.com/google/trillian/merkle/rfc6962" // Load hashers
)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
|
Yes, this breaks ct-go so Travis will never be happy. PR google/certificate-transparency-go#50 will fix ct-go once this merges. |
|
What's the ETA for merging this PR? |
9d85686 to
ac4a6a3
Compare
6ac53cb to
57387ea
Compare
|
This has now been rebased. The number of modified files has reduced from 63 to 46, and the number of changes in those files has reduced. |
57387ea to
8d5f5e5
Compare
.travis.yml
Outdated
| - GOFLAGS=-race WITH_ETCD=true | ||
| - GOFLAGS= WITH_PKCS11=true | ||
| - GOFLAGS=-race WITH_PKCS11=true | ||
| - GOFLAGS="" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Use single quotes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
.travis.yml
Outdated
| - GOFLAGS="" WITH_ETCD=true | ||
| - GOFLAGS="-race" WITH_ETCD=true | ||
| - GOFLAGS="--tags pkcs11" WITH_PKCS11=true | ||
| - GOFLAGS=""-race --tags pkcs11" WITH_PKCS11=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There seems to be an extra double quote here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
server/sequencer_manager_test.go
Outdated
| } | ||
| } | ||
|
|
||
| func fakeKeyProtoHandler(pb proto.Message, signer crypto.Signer, err error) (proto.Message, keys.ProtoHandler) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe rename pb to wantPB (or something to that effect)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
gdbelvin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome. This is looking great!
Previously, when adding support for a new way of accessing keys (e.g.
PKCS#11), there were two options:
1) Modify DefaultSignerFactory.
- Anyone using DefaultSignerFactory would be able to use this, but
they'd also have any new dependencies imposed upon them.
2) Create a new SignerFactory implementation.
- Requires running a fork of Trillian.
- This new implementation would likely duplicate some code from
DefaultSignerFactory, if it wanted to support some of the same ways
of accessing keys.
An improved way of doing this is to instead have a global registry of
functions that each accept a particular kind of protobuf message and
return the corresponding private key. Functions can be added to this
registry to provide support for new methods of accessing keys, or removed
to disable particular methods. Disabled methods could be excluded from the
build entirely using build tags, sparing users who do not use a
particular method from having to install unnecessary dependencies.
Also splits cmd/createtree/main_test.go into two files: - tester.go, which contains shared helper functions. - main_test.go, which contains non-key-specific tests.
1. golang packages 2. third-party packages 3. packages with aliases 4. _ imports
dc1a860 to
31f8169
Compare
Previously, when adding support for a new way of accessing keys (e.g. PKCS#11), there were two options:
An improved way of doing this is to instead have a global registry of functions that each accept a particular kind of protobuf message and return the corresponding private key. Functions can be added to this registry to provide support for new methods of accessing keys, or removed to disable particular methods. Disabled methods could be excluded from the build entirely using build tags, sparing users who do not use a particular method from having to install unnecessary dependencies.
TODO: In a follow-up PR, I intend to improve test coverage of the pre-existing crypto/keys/... packages.