Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[schema][client][improve] Add decode InputStream for Schema #16659

Merged
merged 14 commits into from
Aug 2, 2022

Conversation

tjiuming
Copy link
Contributor

Motivation

Add decode(InputStream) for Schema, for the purpose of preventing allocate heap memory directly.

Modifications

  1. Add new methods in Schema.java with default implementation.
  2. implement new methods for AbstractStructSchema and AutoConsumeSchema
  3. replace decode(byte[]) to decode(InputStream) in MessageImpl#decodeBySchema

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (yes)
  • The schema: (yes)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jul 18, 2022
Copy link
Member

@nodece nodece left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codelipenghui codelipenghui added this to the 2.11.0 milestone Jul 21, 2022
@codelipenghui codelipenghui added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/schema labels Jul 21, 2022
@codelipenghui codelipenghui modified the milestones: 2.11.0, 2.12.0 Jul 26, 2022
Copy link
Contributor

@congbobo184 congbobo184 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tjiuming hi, I don't understand, ByteBufInputStream has already use the decode(input) why we also need ReadonlyByteBufInputStream

@codelipenghui codelipenghui removed this from the 2.11.0 milestone Jul 27, 2022
@codelipenghui codelipenghui added this to the 2.12.0 milestone Jul 27, 2022
@tjiuming
Copy link
Contributor Author

I've discussed with @congbobo184, that introducing decode(InputStream) to Schema may lead to the incorrect usage of users, so decode(ByteBuffer) is enough.

@tjiuming
Copy link
Contributor Author

@congbobo184 PTAL

@nodece
Copy link
Member

nodece commented Jul 31, 2022

@tjiuming Don't run /pulsarbot run-failure-checks again, the CI has been passed.

@tjiuming tjiuming requested a review from congbobo184 August 1, 2022 09:04
@tjiuming
Copy link
Contributor Author

tjiuming commented Aug 2, 2022

/pulsarbot run-failure-checks

@Technoboy- Technoboy- modified the milestones: 2.12.0, 2.11.0 Aug 2, 2022
@Technoboy- Technoboy- merged commit 33dd4f2 into apache:master Aug 2, 2022
@tjiuming tjiuming deleted the dev/schema_improve branch August 3, 2022 08:01
Gleiphir2769 pushed a commit to Gleiphir2769/pulsar that referenced this pull request Aug 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/schema doc-not-needed Your PR changes do not impact docs type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants