Introduce Event Bus (WIP)

[mattwynne]

With this change, clients both within the Cucumber codebase and built by our users, can consume events about various things that happen as your tests run.

For example:

• when a test case finishes
• when a step definition is matched

This should help us remove some of the ugly coupling between different parts of the code, and allow for more flexiblity in the API for users who want to build formatters and other tools.

Feedback please!

The intention is that this change is backwards compatible, with the idea that it will become the mandatory API for writing formatters from version 3.0

[ghost]

Matt, I subscribed myself to your PR. Please ping me, when you're finished.

[ghost]

You might want to change Events::Bus#initialize to take an EventResolver which taskes the logic from here https://github.com/cucumber/cucumber-ruby/pull/903/files#diff-0ad7c123705bb65d670877bee7ea693fR36. The EventResolver could then be different for cucumber-ruby and aruba. What do you think @mattwynne?

[mattwynne]

Hi @dg-ratiodata this is now live in Cucumber 2.1

I'm unclear how it would help to be able to inject different strategies of EventResolver. Can you give me an example of what you would do with that feature?

[ghost]

Sorry for my late response. I was on vacation. I see two aspects here:

1. Separate concerns

   Now the event bus does two things: Receives and Forwards events + Resolves name of event to class. I would love to see this split up in more specific classes

   • EventBus -> Receives and Forwards events
   • EventNameResolver -> Resolves name of event to class

2. Pass information into event bus

   I had that idea because passing the resolver instead of the constant is not much more complicated and separates functionality much better.

[ghost]

@mattwynne Is that really an event or a name for a hook? Normally I would say, that TestStepHasRun would be a name for an event.

[ghost]

One thing I thought about lately at aruba:

How would you distinguish a hook from an event? From a user perspective you could do more or less the same with both of them. At least for aruba I tend to get rid of hooks, because I find it hard to explain when to use what. Matt and Steve, what do you think about that?

[mattwynne]

In Cucumber, the distinction is that a hook is part of the test-case, and can fail the scenario. An error in and event listener would be handled differently. (Not sure how at the moment!)

[aslakhellesoy]

In Cucumber, the distinction is that a hook is part of the test-case, and can fail the scenario. An error in and event listener would be handled differently.

Why is that @mattwynne? Common practice in event APIs is that an error raised in a listener bubbles up to the caller. (See Node.js example further down).

I think an error raised in an event listener should bubble up to the event emitter and fail the scenario. In fact, I think making events and hoods the same thing would be a Good Thing. I don't see a reason to distinguish between them.

var EventEmitter = require("events").EventEmitter;
var e = new EventEmitter();

e.on('stuff', function () {
  throw new Error('oops');
});

try {
  e.emit('stuff');
} catch(err) {
  console.log('caught error thrown by event listener');
}

[mattwynne]

Well, philosophically we see event listeners as being ‘out of the loop’- for example if someone writes a buggy formatter, that’s not a problem with your tests. That’s not to say that Cucumber won’t fail and report the error, but I wouldn’t see it as counting as a test failure.

[aslakhellesoy]

I understand that philosophy, but I think it's an unneeded distinction that only provides additional complexity.

Why not let a failing listener fail the test, and if users complain they need more fine grained distinction, deal with that later.

[mattwynne]

It actually makes things simpler if we assume we can run the tests, publish these result events but not care about what happened at the other end of the pipe. That's how it works now, and I'd prefer to keep it that way until, as you say, someone complains.

[aslakhellesoy]

Further up I think you said you're not sure how to handle an error raised from a listener. Does that mean you don't know what happens, or does it mean you want to change how it currently behaves, but you don't know how?

[brasmusson]

"Hook" seems to be used for very different things in this discussion. Going back to the question from @dg-ratiodata "Is that [AfterTestStep] really an event or a name for a hook? Normally I would say, that TestStepHasRun would be a name for an event."

after_test_step is an hook-method, and formatters that implement it will have it called at the appropriate times during the execution of Test Cases. The event AfterTestStep is an alternative to after_test_step as a mean to notify both formatters and other interested parties.

Then there are the Before/After/etc hooks, that are also Test Steps, which are both executed and reported to formatters as Test Steps, using the (hook-)methods and events.

To me it seems natural that event in the Event Bus is handled similar to the formatter (hook-)methods with respect to errors. However in my mind "event" is something fundamentally asynchronous, so errors during the processing of an event, would need to be passed back as a separate event.

[aslakhellesoy]

@brasmusson asynchronoys events implies threads (for ruby at least) and that complicates things even more. Even node.js (an inherently async platform) implements core api events synchronoysly. See my code above.

I wonder, what is the recommended use case for events (as opposed to hooks and formatters)? Right now the different ways to do the same thing can only confuse users.

I don't like menus with too many similar options.

[brasmusson]

To me it looks like that events and the event bus are basically the replacement for the formatter API, adding the step_match notification to the formatter API's (in v2.0) notifications before/after_test_case, before/after_test_step. And the events and the event bus can also be used internally - I think the --fail-fast option is one that have been mentioned as a feature that could leverage on an event bus.

[mattwynne]

Events are how you get notified about anything that's happening during the test run. They are read-only. They are designed for writing formatters, and other plug-ins that will want to know things about the features being run, but do not need to interact with the run.

Our intention is that this will be the way to write formatters from 3.0 onwards.

This simplifies things for us internally and for users, as we can be clear about which parts of the API are for output (side-effect free) and which parts of the API allow you to interact with Cucumber.

I heard you about Node Aslak, and maybe we've chosen the wrong terminology here. The model in my mind is more pub-sub than synchronous events. I certainly want the option to be able to fire those events asynchronously in the future without breaking people's expectations about the API.

[ghost]

Thanks guys for the clarification. :-) Will take the discussion into account when I decide about how to go on with hooks in aruba. Today there really is only a single hook which is relevant and used: before(:command) and I think it was meant to change the command/the environment of the command.

Back to #903 (comment) @mattwynne What about that comment? I can live with taking over the current implentation though I'm not that happy about all the code taken from ActiveSupport. I think it has really nothing to do with the event bus, but is only use to resolve the name of the event and should better be moved to some kind of a resolver-class.

[mattwynne]

Back to #903 (comment) @mattwynne What about that comment? I can live with taking over the current implentation though I'm not that happy about all the code taken from ActiveSupport. I think it has really nothing to do with the event bus, but is only use to resolve the name of the event and should better be moved to some kind of resolver-class.

—

Makes sense - want to send a PR?

[ghost]

Will do. But will take a bit... Need to prepare a going-live at the beginning of oct. Hope to find some time inbetween.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.