Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmos: Support competing writers with inconsistent unfolds #108

Open
bartelink opened this issue Mar 8, 2019 · 0 comments
Open

Cosmos: Support competing writers with inconsistent unfolds #108

bartelink opened this issue Mar 8, 2019 · 0 comments

Comments

@bartelink
Copy link
Collaborator

At present, if you supply an unfold function, for each write, its output:
a) travels on the wire to the stored procedure (this does not cost extra, so optimizing that is not really a concern)
b) replaces the current unfolds in the Tip document (as usual, you pay for a read with the original size and a write with the new size (not forgetting the cost of the read per active ChangeFeedProcessor) in terms of Request Charges)

Pros and cons:

  • if you stop writing an unfold, it will get removed on the next write, saving storage space, and RUs on the write, subsequent reads, and induced reads by ChangeFeedProcessor(s)
  • if you are doing blue/green deploys, old and new writers can interleave - the loser needs to rebuild their state unless the competing writer happens to also have written the same one (and then potentially remove the competitor's one iff it writes)

A small change to the JS can improve the semantics by
a) removing unfolds that are being updated and replacing with the supplied values
b) retaining any that are not being touched (perhaps subject to some retention criteria, i.e. only keep it if it's <= N events behind Tip)
c) allowing the writer to indicate a set of caseNames that should be removed regardless of retention rules

Related: there's a tip-isa-batch branch which stores events in the Tip - the competing writers scenario's efficiency would be greatly improved by this (any conflict will yield the competing events cleanly, and any unfolds that are behind will typically see both their unfold and successor events from the single point-read roundtrip)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant