Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve] Allow the content be empty when deserialize in MetadataCache #23131

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zymap
Copy link
Member

@zymap zymap commented Aug 7, 2024


Motivation

When using admin to create topic, it will set the content to byte[0] then put it into the value. Sometimes we need to update the value then we will received the deserailization error.
So we need to check the value type to make it as optional empty when the value is null.

Fixes #xyz

Main Issue: #xyz

PIP: #xyz

Motivation

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

---

### Motivation

When using admin to create topic, it will set the content to byte[0]
then put it into the value. Sometimes we need to update the value
then we will received the deserailization error.
So we need to check the value type to make it as optional empty
when the value is null.
Copy link

github-actions bot commented Aug 7, 2024

@zymap Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@zymap zymap changed the title [Fix] Allow the content be empty when deserialize in MetadataCache [improve] Allow the content be empty when deserialize in MetadataCache Aug 7, 2024
@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Aug 7, 2024
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@horizonzy horizonzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

} catch (IOException e) {
return FutureUtils.exception(e);
}
currentValue = Optional.of(clone);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of making the change above, Optional.of could be changed to Optional.ofNullable to achieve the same outcome.

Comment on lines +111 to +113
if (res.getValue().length == 0) {
return FutureUtils.value(Optional.of(new CacheGetResult<>(null, res.getStat())));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There would be less code duplication if this change would be handled in a way where obj is set to null when res.getValue().length == 0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example T obj = res.getValue().length > 0 ? serde.deserialize(path, res.getValue(), res.getStat()) : null;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BookieServiceInfo handled the case of res.getValue.length==0 in deserialize

if (bookieServiceInfo == null || bookieServiceInfo.length == 0) {
return BookieServiceInfoUtils.buildLegacyBookieServiceInfo(bookieId);
}

That's why the test TestAutoRecoveryAlongWithBookieServers#testAutoRecoveryAlongWithBookieServers failed.

Maybe we need to deserialize first and add the res.getValue.length==0 check in the exception section?

Copy link
Member Author

@zymap zymap Aug 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hangc0276 Thank you! I realized I was wrong to do this fix here because the byte[0] may be a magic value for the SerDe implementation. It should be handled in the SerDe classes.

Comment on lines +111 to +113
if (res.getValue().length == 0) {
return FutureUtils.value(Optional.of(new CacheGetResult<>(null, res.getStat())));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BookieServiceInfo handled the case of res.getValue.length==0 in deserialize

if (bookieServiceInfo == null || bookieServiceInfo.length == 0) {
return BookieServiceInfoUtils.buildLegacyBookieServiceInfo(bookieId);
}

That's why the test TestAutoRecoveryAlongWithBookieServers#testAutoRecoveryAlongWithBookieServers failed.

Maybe we need to deserialize first and add the res.getValue.length==0 check in the exception section?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants