[fix][broker] Fix index generator is not rollback after entries are failed added.

[gaozhangmin]

Motivation

Line#800 beforeAddEntry invokes interceptor.interceptWithNumberOfMessages(brokerEntryMetadata, numberOfMessages); to increase the indexGenerator. But it doesn't rollback if entries added failed in LINE#804-813.

This issue will cause the wrong index property set in managed ledger properties map. which would be recovery when managed ledger is initialized.

pulsar/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java

Lines 799 to 816 in 4ed8a87

	protected synchronized void internalAsyncAddEntry(OpAddEntry addOperation) {
	if (!beforeAddEntry(addOperation)) {
	return;
	}
	final State state = STATE_UPDATER.get(this);
	if (state.isFenced()) {
	addOperation.failed(new ManagedLedgerFencedException());
	return;
	} else if (state == State.Terminated) {
	addOperation.failed(new ManagedLedgerTerminatedException("Managed ledger was already terminated"));
	return;
	} else if (state == State.Closed) {
	addOperation.failed(new ManagedLedgerAlreadyClosedException("Managed ledger was already closed"));
	return;
	} else if (state == State.WriteFailed) {
	addOperation.failed(new ManagedLedgerAlreadyClosedException("Waiting to recover from failure"));
	return;
	}

)

Modifications

Decrease the index generator in AppendIndexMetadataInterceptor by

public void decreaseWithNumberOfMessages(int numberOfMessages) {
        indexGenerator.addAndGet(-numberOfMessages);
    }

Verifying this change

org.apache.pulsar.broker.intercept.MangedLedgerInterceptorImplTest#testAddEntryFailed

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

gaozhangmin#11

[github-actions]

@gaozhangmin Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

[eolivelli]

@poorbarcode @BewareMyPower

I support this patch, so +1 to it
but
this change is a "API change", as we are adding a new method to the [ManagedLedgerInterceptor] (https://github.com/apache/pulsar/pull/19727/files) API

We should have discussed about this change on the mailing, a PIP is not needed, but we cannot do such changes without discussing them with the community.

[BewareMyPower]

We should have discussed about this change on the mailing

It makes sense to me.

I'm also wondering whether it's accepted to add a default method to an interface for release branches. The initial purpose to prevent API changes to release branches is to ensure users can upgrade X.Y.Z to X.Y.(Z+1) without modifying any existing code. For example, if any method signature of BrokerEntryMetadataInterceptor is modified, all existing plugins (protocol handlers, connectors, etc.) that use an interceptor could be affected. I'm not sure whether adding a default method could affect the compatibility.

Since @gaozhangmin added the labels to cherry-pick this PR to release branches, could you drive the discussion in the mail list?

[gaozhangmin]

https://lists.apache.org/thread/m0tpp7f0ftf6cg0cpq25l23lx6gzgwl9 @eolivelli @BewareMyPower

[michaeljmarshall]

As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label