Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First steps on delta protocol connection #275

Open
enikao opened this issue May 19, 2024 · 4 comments
Open

First steps on delta protocol connection #275

enikao opened this issue May 19, 2024 · 4 comments
Labels

Comments

@enikao
Copy link
Contributor

enikao commented May 19, 2024

We assume a stateful, bi-directional connection between client and repository

  1. Client connects to repository with a clientId (same as for bulk protocol)
  2. Client tells repo the subscription scope (How to specifiy subscription scope? #269)
  3. Repo sends all nodes in scope as chunk to client.
  4. Repo assures to send all changes on any node in scope based on the state from step 3.

Rationale for 3.: Most clients would be interested the nodes in scope.
The client could also request the nodes in scope via bulk protocol. However, using two different protocols always leaves a potential gap: If the client first requests the contents via bulk at time 1, and then subscribes at time 2, it might miss delta updates that happened between 1 and 2.
If it first subscribes at time 3, and then requests via bulk at time 4, it might receive delta updates that are already included in the retrieved bulk contents. It might be hard for the client to decide whether such an update can be ignored.

@enikao enikao added the delta label May 19, 2024
@dslmeinte
Copy link
Contributor

I think connections between a client and repository should probably be quite “conversational”, with messages like (client:) “subscribe to partition <P_1>”, “unsubscribe from <P_2>”, (repo:) “hey, partition <P_3> got created”, etc.

@joswarmer
Copy link
Contributor

joswarmer commented May 20, 2024

See #274 of a potential way to close the gap.

@dslmeinte
Copy link
Contributor

See #274 of a potential way to close the gap.

Yeah, but I think messages to the repo {c|sh}ould be quite intentional, so not “if I request all deltas from time onwards I'll effectively have changed scope”.

@enikao
Copy link
Contributor Author

enikao commented Jul 19, 2024

Point 3. above could be alleviated by #68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants