-
Notifications
You must be signed in to change notification settings - Fork 903
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BP-32: Advisory (optimistic) write close #1343
Comments
@reddycharan do you have any proposal about http://bookkeeper.apache.org/community/bookkeeper_proposals/ |
@sijie will do |
reddycharan
changed the title
Advisory (optimistic) write close
BP-32 Advisory (optimistic) write close
Apr 27, 2018
reddycharan
changed the title
BP-32 Advisory (optimistic) write close
BP-32: Advisory (optimistic) write close
Apr 27, 2018
@reddycharan are you still working on this? |
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
FEATURE REQUEST
With #570, entrylog per ledger feature will be introduced. With this feature there will be dedicated entrylog for each ledger. < sub-task6 > of issue-570 implements EntryLogManagerForEntryLogPerLedger.
Since there is going to be entrylog per ledger, with the current Bookie implementation there is no way to know when the entrylog is writeclosed and the entrylog for the active ledger can be rotated. So in < sub-task6 > of issue-57, as a first step of handling this, expireAfterAccess (each entrylog should be automatically rotated from the current active entrylog list once a fixed duration has elapsed after the last access of entrylog for addEntry) will be implemented and also the maximum number of entrylogs that can be active at a given time (this is for limiting number of entrylogs/filedescriptors open at a given time).
The above mentioned approaches are preliminary ways of handling but it would be ideal to have explicit call to EntryLogger when the write for this ledger is done so that it can rotate the entrylog as soon as it is done writing. This will minimize the number of entrylogs/file descriptors that are open/active and also it will make progress in leastUnflushedLogId, so that GarbageCollectorThread can consider these entrylogs for garbage collection. So as part of next step, following can be done
have explicit write close request. This write close request should be sent to the current ensemble when the write handle is closed. This should be just optimistic write close operation and callback of this operation should be just logger, saying if it is succeeded / partially succeeded / failed. This should be done asynchronously and the write handle close operation should not be blocked for this response.
In the case of ledger recover open, readrequest (ReadEntryProcessorV3) with fence flag, can take care of calling appropriate methods in ledgerdescriptor / entry logger to rotate the entrylog for the ledger
in the case of auto-replication case, LedgerFragmentReplicator already uses bookieclient for addEntry (bkc.getBookieClient().addEntry), the same bookieClient instance can be used to call explicit writeClose method in bookieclient.
in the case of any write failure in bookie for an entry, then before sending error response to the client do entrylog rotation
This is should-have feature for entrylogperledger
The text was updated successfully, but these errors were encountered: