ISSUE-2220: Exception while replaying journals, shutting down

[sijie]

Original Issue: apache#2220

---

We're running Apache Pulsar with bookkeeper on a Kubernetes cluster and we have bookies (2/3) failing to restart with the following exception:
09:49:25.511 [main] ERROR org.apache.bookkeeper.bookie.Bookie - Exception while replaying journals, shutting down
java.io.IOException: org.apache.bookkeeper.bookie.BookieException$OperationRejectedException
at org.apache.bookkeeper.bookie.Bookie$4.process(Bookie.java:895) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.bookie.Journal.scanJournal(Journal.java:821) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.bookie.Journal.replay(Journal.java:866) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.bookie.Bookie.readJournal(Bookie.java:901) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.bookie.Bookie.start(Bookie.java:922) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.proto.BookieServer.start(BookieServer.java:141) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.server.service.BookieService.doStart(BookieService.java:58) ~[org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:78) ~[org.apache.bookkeeper-bookkeeper-common-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$2(LifecycleComponentStack.java:113) ~[org.apache.bookkeeper-bookkeeper-common-4.9.2.jar:4.9.2]
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:407) [com.google.guava-guava-25.1-jre.jar:?]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:113) [org.apache.bookkeeper-bookkeeper-common-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.common.component.ComponentStarter.startComponent(ComponentStarter.java:80) [org.apache.bookkeeper-bookkeeper-common-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.server.Main.doMain(Main.java:229) [org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.server.Main.main(Main.java:203) [org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
at org.apache.bookkeeper.proto.BookieServer.main(BookieServer.java:314) [org.apache.bookkeeper-bookkeeper-server-4.9.2.jar:4.9.2]
Caused by: org.apache.bookkeeper.bookie.BookieException$OperationRejectedException

The exception is re-thrown so not sure of root cause but OperationRejectedException is thrown from only one place in version 4.9.