-
Notifications
You must be signed in to change notification settings - Fork 903
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce Offset Index #1376
Comments
It would be great for stagare usecase, like reading a batch of sequential entries woth random access pattern (opposite to tailing reads) |
A new wire protocol rpc would great as well |
FEATURE REQUEST
We create continuous streams of growing data, eg ledgers with entities. Proposed API's:
Returns the byte offset of the last byte of the last entry committed to the ledger.
Reads all bytes between startOffset and endOffset (inclusive), returned per stored entry.
Reads all bytes from startOffset to current confirmed end of ledger, returned per stored entry.
Currently using a different solution to tackle this use case, with less durability/scalability/etc.
While not needed for our use-case, to be API complete, uncommitted API's might be good to have.
While not immediately needed for our use-case and we can tackle this with other polling mechanisms, it would be useful if we can open read binary from a ledger. |
FEATURE REQUEST
Currently in a ledger, we indexed entries by entry id. It would be good to have an index by offsets. This allows supporting APIs like:
nice-to-have
entry(/request) oriented api is not very good friendly to resource-usage when do prefetching or batching reads. offset oriented api is much better for estimating resource usage.
The text was updated successfully, but these errors were encountered: