Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] unify topic-level policies enable judgment conditions #19501

Merged
merged 2 commits into from
Jan 22, 2024

Conversation

aloyszhang
Copy link
Contributor

@aloyszhang aloyszhang commented Feb 13, 2023

Motivation

Broker may throw the following exception when deleting topic

ERROR org.apache.pulsar.broker.admin.v2.PersistentTopics - [null] Failed to delete topic persistent://test/test/t-partition-17
java.lang.UnsupportedOperationException: Topic policies service is disabled.
        at org.apache.pulsar.broker.service.TopicPoliciesService$TopicPoliciesServiceDisabled.deleteTopicPoliciesAsync(TopicPoliciesService.java:146) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at org.apache.pulsar.broker.service.BrokerService.deleteTopicPolicies(BrokerService.java:3345) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at org.apache.pulsar.broker.service.AbstractTopic.deleteTopicPolicies(AbstractTopic.java:1240) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$delete$31(PersistentTopic.java:1247) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[?:?]
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309) ~[?:?]
        at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$delete$36(PersistentTopic.java:1245) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
        at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$delete$28(PersistentTopic.java:1232) ~[pulsar-broker.jar:2.12.0-SNAPSHOT]
        at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:787) ~[?:?]
        at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.87.Final.jar:4.1.87.Final]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

This is caused by the configuration of topicLevelPoliciesEnabled and systemTopicEnabled are not consistent.

Modifications

unify topic-level policies enable judgment conditions

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc-not-needed

Matching PR in forked repository

PR in forked repository:
aloyszhang#14

@github-actions
Copy link

@aloyszhang Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we'd better require users to set system topic enabled to true if they want to enable topic level policies. So that we don't need check both the system topic and topic policies is enabled in multiple places.

@mattisonchao @Technoboy- WDYT?

@codelipenghui codelipenghui added this to the 3.0.0 milestone Feb 14, 2023
Copy link
Contributor

@315157973 315157973 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is meaningless to enable topic policy but not system topic.
Therefore, I think it would be better to do parameter verification at startup?

@aloyszhang
Copy link
Contributor Author

aloyszhang commented Feb 21, 2023

It is meaningless to enable topic policy but not system topic.
Therefore, I think it would be better to do parameter verification at startup?

Make sense to me. WDYT about this @codelipenghui ?

@Technoboy-
Copy link
Contributor

IMO, we'd better require users to set system topic enabled to true if they want to enable topic level policies. So that we don't need check both the system topic and topic policies is enabled in multiple places.

@mattisonchao @Technoboy- WDYT?

Agree

@github-actions
Copy link

github-actions bot commented Apr 9, 2023

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Apr 9, 2023
@poorbarcode
Copy link
Contributor

Since we will start the RC version of 3.0.0 on 2023-04-11, I will change the label/milestone of PR who have not been merged.

  • The PR of type feature is deferred to 3.1.0
  • The PR of type fix is deferred to 3.0.1

So drag this PR to 3.0.1

@poorbarcode poorbarcode modified the milestones: 3.0.0, 3.1.0 Apr 10, 2023
@github-actions github-actions bot removed the Stale label Apr 11, 2023
@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@codecov-commenter
Copy link

codecov-commenter commented Jan 15, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (176bdea) 73.58% compared to head (edd8578) 36.36%.
Report is 3 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #19501       +/-   ##
=============================================
- Coverage     73.58%   36.36%   -37.22%     
+ Complexity    32325    12605    -19720     
=============================================
  Files          1859     1723      -136     
  Lines        138263   134066     -4197     
  Branches      15153    14736      -417     
=============================================
- Hits         101736    48749    -52987     
- Misses        28644    78851    +50207     
+ Partials       7883     6466     -1417     
Flag Coverage Δ
inttests 24.10% <45.45%> (+<0.01%) ⬆️
systests 23.69% <8.33%> (-0.09%) ⬇️
unittests 31.98% <58.33%> (-40.87%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...n/java/org/apache/pulsar/broker/PulsarService.java 68.67% <100.00%> (-13.04%) ⬇️
...rg/apache/pulsar/broker/service/AbstractTopic.java 64.36% <100.00%> (-23.50%) ⬇️
...rg/apache/pulsar/broker/service/BrokerService.java 57.35% <100.00%> (-23.85%) ⬇️
...sar/broker/service/persistent/PersistentTopic.java 51.27% <100.00%> (-27.27%) ⬇️
...org/apache/pulsar/broker/ServiceConfiguration.java 97.78% <50.00%> (-1.61%) ⬇️
...pulsar/broker/admin/impl/PersistentTopicsBase.java 11.50% <0.00%> (-53.29%) ⬇️
.../org/apache/pulsar/broker/admin/AdminResource.java 43.24% <0.00%> (-34.33%) ⬇️

... and 1427 files with indirect coverage changes

@Technoboy- Technoboy- merged commit 153cd5e into apache:master Jan 22, 2024
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs ready-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants