Publishing Events - should they be unique?

[tatejones]

When publishing events should the event name and key be unique for the workflow? I noticed the testing method WaitForEventSubscription will fall through (not wait) if the same event name and key is published multiple times.

Ideally I would like to use the same event name and key (workflow id) and allow the steps to determine if the workflow will proceed sequentially.

Could someone please clarify the process flow of events and subscriptions? Are subscriptions the act of a node queuing event(s) to be process?

Thanks in advance.

[tatejones]

From the code the order of events with same name and key are read in no order. Hence why I am seeing events queued firing in non-deterministic order. The lamba function on the waitfor could be useful, but I am wondering if this behaviour has been implemented like this by "design" or a limitation that could be resolved? Happy to assist.

[danielgerlag]

are you using the overloaded method that takes an effective date?

the effective date can be used to pre or post date events when publishing them and subscribe to past or future events when using WaitFor

[tatejones]

Correct. However when there are multiple events published (same event name & key) before the trigger occurs, the events received are non-deterministic. There needs to be an "order by" to ensure the events are correctly handled in the order published. With a 10 second poll (default), even a single user can cause data integrity issues with the event order. Unfortunately I can't rely on the event name, key and date (static) to determine order when the projection (select) returns more than one event.

The providers just require an "order by" on the event projections (before the select) to ensure correct processing order.

[danielgerlag]

Ah ok. Would it be possible to keep this domain logic out of the persistence providers and in the core library?

[tatejones]

That would a better design.Some options. 1. Perform the sorting in the core. Simple to implement, however could place memory and cpu pressure if the load of events are high. Possible if the asynchronous query is not polled for a period of time. Eg. Outage 2. Change contract with the provider to only return the oldest event (one object). Force the provider to do it. Deprecate the old api. 3. Change the contract to return an iqueryable (delay materialising) to force the ordering and number of rows to be controlled by the core. The execution will be performed within the providers persistent layer. Eg. DB server and not the client. Deprecate the old api calls. Option 3 would be my preference. On the road tate@bluedog.com.au

…

On 25 Jun 2018, at 11:56 pm, Daniel Gerlag ***@***.***> wrote: Ah ok. Would it be possible to keep this domain logic out of the persistence providers and in the core library? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[danielgerlag]

I absolutely love this level of engagement 😃 👍
Regarding option 3, what are your thoughts on this article?

http://www.codetunnel.io/should-you-return-iqueryablet-from-your-repositories/

[tatejones]

I do agree with the article if the client (workflow-core) wasn't in our control. However in this situation the client is quite specific (in: event name, key, date out: event obj) and the provider must adhere to the required contract. The main difference here is the provider is being called by the workflow-core (using inversion of control pattern). The article is discussing the opposite situation of creating an API to be called by a consumer we have no control over (bridge pattern). Your provider API is a plugin framework.

With plugin frameworks you can easily unit test a plugin implementation to ensure the correct behaviour is expected. eg. functional (input and outputs) and non-functional (speed, load and performance).

In summary I would still recommend using IQueryable (option 3) in this specific situation.

[danielgerlag]

good points, I still have concerns about the actual query execution happening without the repository method on the call stack, and also it could make for some confusing logging should the query raise an exception. Another concern would be that every persistence provider be easily able to return an IQueryable - not all DB libraries have support for Linq

[tatejones]

I am not 100% sure what you mean by the call stack in this context...I assuming the following 1. If the call stack was to disappears whilst iterating over the result (eg. Db disconnection), I believe there would be more serious problems to deal with across the entire platform. A simple ToList() can also fail when it is materialising. 2. Unhandled exceptions (eg. io exception) are 'part of parcel' when using streaming / iterative APIs. eg. EntityFramework is a great example. 3. True. If it doesn't, they will would need to move to a List and perform AsQueryable() or build a QueryProvider.

…

On 26 June 2018 at 12:53, Daniel Gerlag ***@***.***> wrote: good points, I still have concerns about the actual query execution happening without the repository method on the call stack, and also it could make for some confusing logging should the query raise an exception. Another concern would be that every persistence provider be easily able to return an IQueryable - not all DB libraries have support for Linq — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#146 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbmi9hObhsH3FwE9paBrt5vjqi1KFlwks5uAaIygaJpZM4UzBvu> .

-- Tate Jones

[bwail]

Any news on that topic? I am running into the same situation as @tatejones, I am expecting the last event occurred using the same name and key to be returned, but that's not the current behavior.

[danielgerlag]

@bwail my apologies, I lost track of this issue... I will try spend some time on it later this week.