Skip to content

Conversation

@RJPercival
Copy link
Contributor

@RJPercival RJPercival commented Jul 12, 2017

Previously, when adding support for a new way of accessing keys (e.g. PKCS#11), there were two options:

  1. Modify DefaultSignerFactory.
    • Anyone using DefaultSignerFactory would be able to use this, but they'd also have any new dependencies imposed upon them.
  2. Create a new SignerFactory implementation.
    • Requires running a fork of Trillian.
    • This new implementation would likely duplicate some code from DefaultSignerFactory, if it wanted to support some of the same ways of accessing keys.

An improved way of doing this is to instead have a global registry of functions that each accept a particular kind of protobuf message and return the corresponding private key. Functions can be added to this registry to provide support for new methods of accessing keys, or removed to disable particular methods. Disabled methods could be excluded from the build entirely using build tags, sparing users who do not use a particular method from having to install unnecessary dependencies.

TODO: In a follow-up PR, I intend to improve test coverage of the pre-existing crypto/keys/... packages.

@RJPercival RJPercival self-assigned this Jul 12, 2017
@RJPercival RJPercival requested a review from gdbelvin July 12, 2017 11:43
@RJPercival RJPercival force-pushed the signerfactory_replace branch 2 times, most recently from f70bd48 to 0e66d19 Compare July 12, 2017 12:01
@RJPercival
Copy link
Contributor Author

Judging by Travis, it looks like this PR might be responsible for the slow integration tests I reported in #733. I'm not sure where the performance issue is yet though - I'll investigate shortly.

@RJPercival
Copy link
Contributor Author

It turned out not to be a performance issue, but rather a sequencing pass failure that wasn't being (visibly) reported (see #738). I've now fixed that.

@RJPercival RJPercival requested a review from codingllama July 12, 2017 17:00
@RJPercival RJPercival force-pushed the signerfactory_replace branch 2 times, most recently from b660ac7 to 1dd0803 Compare July 13, 2017 11:34
Copy link
Contributor

@codingllama codingllama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sending the comments I have so far. Github is warning me my page is out of date, so I'll refresh and resume the review.

maxRootDuration time.Duration
privateKeyType, pemKeyPath, pemKeyPass, pkcs11ConfigPath string
}
var keyHandlers = make(map[string]func() (proto.Message, error))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: I found it a bit odd that keyHandlers was declared here, but initialized elsewhere. There's more at a further comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because one of the handlers comes from a file that is conditionally built (if "pkcs11" build tag is used), so it can't be populated right here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were we going to move keyHandlers to their own file or package?

wantTree: &nonDefaultTree,
},
{
desc: "defaultOptsOnly",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an error test because the private key won't be set? Maybe rename to "mandatoryOptsNotSet", or add a comment? At a glance it doesn't look different from "validOpts", it only became clearer after I realized runTest "fixes" a few flags.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. I'll rename it.

return
}

if diff := pretty.Compare(tree, test.wantTree); diff != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use proto.Equal() when comparing protos, pretty.Compare doesn't know about internal proto fields, so it may diff in cases where protos are meant to be equal.

The suggested idiom is:

if !proto.Equal(pb1, pb2) {
    diff := pretty.Compare(pb1, pb2)
    t.Errorf("%v: post-Foo diff (-got +want):\n%v", test.desc, diff)
}

(Yep, it's a bit verbose, I know :/ )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not actually new code (the indentation has just changed), but I'll fix it up nonetheless.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

)

func init() {
keyHandlers["PEMKeyFile"] = func() (proto.Message, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer keyHandlers to be declared and initialized in the same file. I would also prefer all flags to be defined together (at main.go), but since this is in the main package we're not breaking the rule too much.

Suggestion: for the sake of isolation / cleaner interface between createtree and key logic, we could do something like:

At createtree/keys/keys.go:

package keys

type handler func() (proto.Message, error)

func SetHandler(name string, fn handler) { ... }

func Handler(name string) (handler, bool) { ... }

func NewPEMKeyHandler(path, pass) handler { ... }

At createtree/main.go:

func main() {
    keys.RegisterHandler("PEMKeyFile", keys.NewPEMKeyHandler(*path, *pass))
    // ...
}

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keyHandlers can't be initialized in the same file, because the PKCS#11 handler needs to be behind a build tag. It needs to use the global registry / auto-registration pattern. I can move this registry out of main.go and into its own package though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we figured out a way for everything except the RegisterHander function to be in it's own package?

}
wantTree.PrivateKey = keyProto

runTest(t, []*testCase{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have runTest and testCase moved to a separate, test-library like file? I'd be ok with a testonly package inside createtree, for example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll then lose access to the flags and won't be able to call createTree(). I can put them in their own file, but not their own package.


runTest(t, []*testCase{
{
desc: "empty pemKeyPath",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is tested in an integration-like style, where we exercise it via createtree, I'd be tempted to use real handlers at main_test.go and move those test cases there. Or, possibly, just test the handlers directly here. If you take the comment on package separation, it'll force you down one of those roads regardless (which why I like it).

// defer flagsaver.Save().Restore()
// // Test code that changes flags
// } // flags are reset to their original values here.
package flagsaver
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagsaver isn't a command we run, so it shouldn't be placed inside cmd/. It definitely deserves its own package, though. Maybe util/flagsaver?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

import "flag"

// Stash holds flag values so that they can be restored at the end of a test.
type Stash struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe type Stash map[string]string ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would preclude extending this type in the future and expose the implementation detail that it stores a map of flag names to values. The only reason I can see to do this is to allow the user to examine stashed flag values - do you see that being a use case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works fine as a struct, but since you don't have other fields an alias could have worked too. It's fine as it is.

// Only a subset of the possible flag types are currently tested.
func TestRestore(t *testing.T) {
tests := []struct {
// Test name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "// name is ..."

Ditto for the other comments, although I think they could all be removed.

nit2: "desc string"? That seems to be the recommended format.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits addressed, but left the comments, since I'm not sure oldValue and newValue are completely self-explanatory (perhaps I just need better names?).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

// The value the flag should have just before being restored to oldValue.
newValue string
}{
{"RestoreDefaultIntValue", "int_flag", "", "666"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please use field labels. It's clearer, saves you from specifying defaults, and saves us from having to touch all structs if a new field is added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

@codingllama codingllama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future consider breaking PRs like this into smaller steps. It's a bit hard to figure out what changed and what was moved, since Github doesn't help much to identify the latter.

return nil, fmt.Errorf("der: unsupported private key type: %T", key2)
}

key3, err3 := x509.ParseECPrivateKey(keyDER)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering here in any way important? Should we try EC first (assuming it is / will be the most popular one)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, these can be re-ordered.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-ordered to try EC first.

sf.AddHandler(&keyspb.PrivateKey{}, NewFromPrivateKeyProto)

for _, test := range []struct {
name string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: s/name/desc? Same for others (if you take it, ofc).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

wantErr: true,
},
} {
signer, err := sf.NewSigner(context.Background(), test.keyProto)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Declare ctx outside the loop and reuse it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


sf := NewSignerFactory()
sf.AddHandler(&keyspb.PrivateKey{}, NewFromPrivateKeyProto)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we test the handler for *keyspb.Specification? Ditto for the other public functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm planning on writing tests for all of the public functions in a follow-up PR - it that ok? I haven't added any new public functions, other than NewPrivateKeyProtoFromSpec(), that don't have tests (I'll add a test for that now).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test for NewPrivateKeyProtoFromSpec().

}

func TestGenerateKey(t *testing.T) {
func TestNewFromSpec(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shuffled some things around, but tests seem to have lagged behind. Could we make sure that the tests are in the matching _test.go file, compared to the one the function is defined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done for TestNewFromSpec. I was intending to do this more thoroughly in the follow-up CL by replacing TestLoadPrivateKeyAndSign with individual tests for each of the public funcs (and put those tests in the appropriate _test.go file).


func TestPkcs11(t *testing.T) {
// PKCS11Config support is tested by integration/log_integration.sh (when $WITH_PKCS11 == "true").
t.Skip("Only integration testing is implemented for PKCS#11")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have a simple, happy path test? If we disable integration testing for some reason this will go without coverage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be difficult because the PKCS#11 code calls out to https://github.com/letsencrypt/pkcs11key which attempts to load a dynamic library and communicate with a PKCS#11 interface. I could expose a package variable that allows the pkcs11key.New() call to be overridden during the tests, but that's about the only way I can think of making it unit testable.

At least by now having a skipped test for this, there is some (minimal) visibility of the fact that it lacks unit tests.

Generate ProtoGenerator

// handlers convert a protobuf message into a crypto.Signer.
handlers map[string]ProtoHandler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered making handlers global? Is there a situation where having factory-scoped handlers is useful?

A global handler registry would allow each handler-defining package to register its own handlers at init-time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's what I started with, but...

  1. I've known people to be opposed to globals (e.g. @daviddrysdale lamented the scattering of flags through the codebase).
  2. I preferred the explicitness of adding a handler, rather than it being a side-effect of importing a package, since it's quite an important decision - do you really want unencrypted keys in your storage, for instance? It also means it's easier to control which handlers are available to tests.
  3. Scoping it to a SignerFactory means you can start servers/signers in-process (like the integration tests do) and not be forced to share the same set of handlers.

Having said that, I do recognise that making the handlers global and having them register on package init is a more common pattern. I can put together a commit that makes that change and see how it compares if you like?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn't seem to shy away from global state in other places, like the hashers registry, and an init-based registry would reduce the number of +build pkcs11 files reaching for signerFactories, so it seemed worth to suggest it.

That said, your counter-arguments and the wariness about global state are certainly strong points (I found number "2" above particularly convincing). I'm happy going forward the way it is. Thanks for the detailed response, Rob.

func (f SignerFactory) AddHandler(keyProto proto.Message, handler ProtoHandler) {
keyProtoType := proto.MessageName(keyProto)

if _, alreadyExists := f.handlers[keyProtoType]; alreadyExists {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/alreadyExists/ok/, so we don't deviate from the idiom?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't entirely convinced that "ok" was an appropriate name here, as it's semantically the opposite - if it already exists, that's not really ok. Originally I was returning an error here, which meant it was very much not ok. Perhaps that doesn't matter though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK ;)

)

// SignatureAlgorithm returns the algorithm used for this public key.
// Only ECDSA and RSA keys are supported. Other key types will return sigpb.DigitallySigned_UNKNOWN.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/sigpb.DigitallySigned_UNKNOWN/sigpb.DigitallySigned_ANONYMOUS/

In a somewhat related comment, is anonymous ever valid / allowed? Should we differentiate between anonymous and unknown? (This is possibly out of scope.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is intended to mirror the DigitallySigned struct in RFC5246, which is why there isn't an "unknown" value. There are cases where anonymous is valid (see that RFC), but never in Trillian. Arguably, this should return an error rather than sigpb.DigitallySigned_ANONYMOUS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed typo.

req *trillian.CreateTreeRequest
createErr error
commitErr, wantCommit bool
wantErr string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asserting on error strings is likely to make tests brittle. Ideally, we'd use something more resilient to small changes, like error codes, but I that won't help if the codes are all the same. OTOH, it does make it easier to reason that the error is what you think it is, and not some problem.

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, and never usually check the error string but, in this case, there are a lot of different errors that could occur. I really want to know that the test is failing for the reason I think it is, rather than because the test is misconfigured, for example. Error codes would be a better solution except, as you suggest, I don't think they'd actually be different for most of these test cases (they're almost all cases of invalid arguments). Maybe that would be good enough for distinguishing the intended type of error from unintended errors though?

@RJPercival
Copy link
Contributor Author

If you like, I think I could break this PR in two, with the first just doing some moving around of code?

@codingllama
Copy link
Contributor

No need to break it now, I'll review the fixes commit-by-commit from now on. Just give me nudge when you finish replying to the comments.

@RJPercival RJPercival force-pushed the signerfactory_replace branch from e23c9e8 to 98bf15b Compare July 13, 2017 18:03
}, nil
}

keyHandlers["PrivateKey"] = func() (proto.Message, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider making each of these anonymous functions real functions for ease on the eye and ease of testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@RJPercival RJPercival force-pushed the signerfactory_replace branch 6 times, most recently from 82cdaf1 to 0cd23c3 Compare July 14, 2017 19:53
@RJPercival
Copy link
Contributor Author

I've just completed some changes @gdbelvin requested offline, which swaps over to a global registry of key handlers with auto-registration on import and ultimately deletes SignerFactory. I'll take on improvement to cmd/createtree next.

@RJPercival RJPercival force-pushed the signerfactory_replace branch 3 times, most recently from b57fa08 to 7905ec8 Compare July 14, 2017 20:58
}
var keyHandlers = make(map[string]func() (proto.Message, error))

func createTree(ctx context.Context, opts *createOpts) (*trillian.Tree, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great change of replacing opts with the relevant flags, but for the clarity of this already large PR, could I suggest that it be a separate PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it'll have to merge first though, since having the PKCS#11 flag behind a build tag is dependent on createOpts behind removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #788.

maxRootDuration time.Duration
privateKeyType, pemKeyPath, pemKeyPass, pkcs11ConfigPath string
}
var keyHandlers = make(map[string]func() (proto.Message, error))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were we going to move keyHandlers to their own file or package?

}
pemSigner, err := keys.NewFromPrivatePEMFile(
opts.pemKeyPath, opts.pemKeyPass)
func newPK(format string) (*any.Any, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should probably live in it's own file or package along with the keyHandlers

Also consider a clearer function name, say NewPublicKeyProto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

pemKeyPath: *pemKeyPath,
pemKeyPass: *pemKeyPassword,
pkcs11ConfigPath: *pkcs11ConfigPath,
func keyFormats() []string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A function comment would be great

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done (though it's now called RegisteredTypes() in the createtree/keys package).

pemKeyTree.PrivateKey, err = ptypes.MarshalAny(&keyspb.PEMKeyFile{
Path: pemPath,
Password: pemPassword,
runTest(t, []*testCase{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'd like to run the test in a sub function, consider using a proper SubTest
https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks

for _, tc := range []struct{}{}{
  t.Run("A=1", func(t *testing.T) { ... })
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is so much a subtest as just delegating the actual test logic to a helper. However, I wonder if we should be using subtests for all of our tabular tests? It'd save having to prepend the test case name to every failure message, and defer would work nicely (running at the end of each test case, rather than at the end of the entire test).

Copy link
Contributor

@codingllama codingllama Jul 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The convention seems to prefer "regular" table-driven, but it may be because t.Run is a relatively recent addition, iirc. Maybe we should take this question to stylists? Meanwhile, I'd vouch for the simplest approach (no t.Run unless you really need it).

Edit: spelling

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I consulted a stylist, and they too leaned away from using subtests, but mostly because they are quite new and rarely used so far. I've decided to use them, but inside of runTest(), where it was already using an anonymous function to isolate each test case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #788 for the use of subtests.

// the privateKeyFormat flag to "test". It also registers a "test" key handler.
// Finally, it calls the test's setFlags func (if provided) to allow it to
// change flags specific to the test, then performs the test.
func runTest(t *testing.T, tests []*testCase) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a slightly more descriptive name would be great

// the privateKeyFormat flag to "test". It also registers a "test" key handler.
// Finally, it calls the test's setFlags func (if provided) to allow it to
// change flags specific to the test, then performs the test.
func runTest(t *testing.T, tests []*testCase) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a slightly more descriptive name would be great - and the test should probably only run one test case at a time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are only run one at a time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah you mean that it should take a single testCase, rather than an array.

)

func init() {
keyHandlers["PEMKeyFile"] = func() (proto.Message, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we figured out a way for everything except the RegisterHander function to be in it's own package?

Copy link
Contributor

@codingllama codingllama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Travis looks unhappy, but other than that LGTM.

Is this one of the cases where we Travis won't pass because of the google/trillian <-> google/ct-go cycle?

pemKeyTree.PrivateKey, err = ptypes.MarshalAny(&keyspb.PEMKeyFile{
Path: pemPath,
Password: pemPassword,
runTest(t, []*testCase{
Copy link
Contributor

@codingllama codingllama Jul 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The convention seems to prefer "regular" table-driven, but it may be because t.Run is a relatively recent addition, iirc. Maybe we should take this question to stylists? Meanwhile, I'd vouch for the simplest approach (no t.Run unless you really need it).

Edit: spelling


func TestFromProto(t *testing.T) {
// ECDSA private key in DER format.
keyDER, err := base64.StdEncoding.DecodeString("MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgS81mfpvtTmaINn+gtrYXn4XpxxgE655GLSKsA3hhjHmhRANCAASwBWDdgHS04V/cN0LZgc8vZaK4I1HWLLCoaOO27Z0B1aS1aqBE7g1Oo8ldSCBJAvee866kcHhZkVniPdCG2ZZG")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Move the DER string to a const?

}

if err := testonly.CheckKeyMatchesSpec(key, test.keySpec); err != nil {
t.Errorf("%v: NewProtoFromSpec() => %v", test.desc, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/NewProtoFromSpec/CheckKeyMatchesSpec/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

func TestPrivateKeyProtoHandler(t *testing.T) {
func TestProtoHandler(t *testing.T) {
// ECDSA private key in DER format.
keyDER, err := base64.StdEncoding.DecodeString("MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgS81mfpvtTmaINn+gtrYXn4XpxxgE655GLSKsA3hhjHmhRANCAASwBWDdgHS04V/cN0LZgc8vZaK4I1HWLLCoaOO27Z0B1aS1aqBE7g1Oo8ldSCBJAvee866kcHhZkVniPdCG2ZZG")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto (move to const suggestion)

}

if err := testonly.CheckKeyMatchesSpec(key, test.keySpec); err != nil {
t.Errorf("%v: NewFromSpec(): %v", test.desc, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/NewFromSpec/CheckKeyMatchesSpec/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

_ "github.com/go-sql-driver/mysql" // Load MySQL driver

// Register key ProtoHandlers
_ "github.com/google/trillian/crypto/keys/der/proto"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: I've been splitting imports in the following blocks: golang, external libs, aliases and _ imports. I've found it more readable than mixing aliases and _ imports on the "external" block.

Eg:

import (
    "context"
    "flag"

    "github.com/golang/glog"
    "github.com/golang/protobuf/proto"
    "github.com/google/trillian"
    "github.com/google/trillian/cmd"

    mysqlq "github.com/google/trillian/quota/mysql"

    _ "github.com/google/trillian/merkle/objhasher" // Load hashers
    _ "github.com/google/trillian/merkle/rfc6962"   // Load hashers
)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@RJPercival
Copy link
Contributor Author

Yes, this breaks ct-go so Travis will never be happy. PR google/certificate-transparency-go#50 will fix ct-go once this merges.

@AMarcedone
Copy link
Contributor

What's the ETA for merging this PR?

@RJPercival
Copy link
Contributor Author

Apologies for the delay getting this merged, I was on holiday for a week and have had competing priorities. I've now broken this PR up as requested by @gdbelvin (see PRs #791, #792, #793, #794). Once those PRs are merged, I'll rebase this and hopefully it'll be of a more easily reviewable size.

@RJPercival RJPercival force-pushed the signerfactory_replace branch 2 times, most recently from 6ac53cb to 57387ea Compare August 15, 2017 23:45
@RJPercival RJPercival changed the title Replace SignerFactory interface with extensible struct Replace SignerFactory with global protobuf handlers for keys Aug 15, 2017
@RJPercival
Copy link
Contributor Author

This has now been rebased. The number of modified files has reduced from 63 to 46, and the number of changes in those files has reduced.

@RJPercival RJPercival force-pushed the signerfactory_replace branch from 57387ea to 8d5f5e5 Compare August 15, 2017 23:53
.travis.yml Outdated
- GOFLAGS=-race WITH_ETCD=true
- GOFLAGS= WITH_PKCS11=true
- GOFLAGS=-race WITH_PKCS11=true
- GOFLAGS=""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Use single quotes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

.travis.yml Outdated
- GOFLAGS="" WITH_ETCD=true
- GOFLAGS="-race" WITH_ETCD=true
- GOFLAGS="--tags pkcs11" WITH_PKCS11=true
- GOFLAGS=""-race --tags pkcs11" WITH_PKCS11=true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be an extra double quote here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
}

func fakeKeyProtoHandler(pb proto.Message, signer crypto.Signer, err error) (proto.Message, keys.ProtoHandler) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename pb to wantPB (or something to that effect)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

@gdbelvin gdbelvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. This is looking great!

Rob Percival added 3 commits August 16, 2017 15:35
Previously, when adding support for a new way of accessing keys (e.g.
PKCS#11), there were two options:
1) Modify DefaultSignerFactory.
  - Anyone using DefaultSignerFactory would be able to use this, but
    they'd also have any new dependencies imposed upon them.
2) Create a new SignerFactory implementation.
  - Requires running a fork of Trillian.
  - This new implementation would likely duplicate some code from
    DefaultSignerFactory, if it wanted to support some of the same ways
    of accessing keys.

An improved way of doing this is to instead have a global registry of
functions that each accept a particular kind of protobuf message and
return the corresponding private key. Functions can be added to this
registry to provide support for new methods of accessing keys, or removed
to disable particular methods. Disabled methods could be excluded from the
build entirely using build tags, sparing users who do not use a
particular method from having to install unnecessary dependencies.
Also splits cmd/createtree/main_test.go into two files:
- tester.go, which contains shared helper functions.
- main_test.go, which contains non-key-specific tests.
1. golang packages
2. third-party packages
3. packages with aliases
4. _ imports
@RJPercival RJPercival force-pushed the signerfactory_replace branch from dc1a860 to 31f8169 Compare August 16, 2017 14:35
@RJPercival RJPercival merged commit 533c7a4 into google:master Aug 16, 2017
@RJPercival RJPercival deleted the signerfactory_replace branch August 16, 2017 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants