Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][admin] Pretty print bookies racks-placement command output #20516

Merged
merged 1 commit into from
Jun 8, 2023

Conversation

vineeth1995
Copy link
Contributor

@vineeth1995 vineeth1995 commented Jun 6, 2023

Motivation

Pretty print the output from "bin/pulsar-admin bookies racks-placement" command so that it is in a more readable format

Modifications

Use Gson package to pretty print the output of the above command

Verifying this change

This change is already covered by existing tests, such as bookies() testcase in PulsarAdminToolTest.java

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@github-actions
Copy link

github-actions bot commented Jun 6, 2023

@vineeth1995 Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@github-actions github-actions bot added doc-label-missing doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Jun 6, 2023
Copy link
Member

@michaeljmarshall michaeljmarshall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

For what it's worth, I think it could make sense to make this a parameter, similar to other CLIs that I have used. However, we already commonly output pretty printed JSON, so I think this is valid.

Copy link
Member

@tisonkun tisonkun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments inline.

@@ -35,7 +38,8 @@ private class GetAll extends CliCommand {

@Override
void run() throws Exception {
print(getAdmin().bookies().getBookiesRackInfo());
Gson gson = new GsonBuilder().setPrettyPrinting().create();
print(gson.toJson(getAdmin().bookies().getBookiesRackInfo()));
Copy link
Member

@tisonkun tisonkun Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this call is already back by:

    <T> void print(T item) {
        try {
            if (item instanceof String) {
                System.out.println(item);
            } else {
                System.out.println(writer.writeValueAsString(item));
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

... where

private static ObjectMapper mapper = ObjectMapperFactory.create();
private static ObjectWriter writer = mapper.writerWithDefaultPrettyPrinter();

Isn't the current code pretty print? Otherwise, we may try to fix the Jackson ObjectMapper config instead of wrapping up one more layer on Gson to hack.

Copy link
Contributor Author

@vineeth1995 vineeth1995 Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @tisonkun, you are right but since BookiesRackConfiguration class extends TreeMap, it goes to the first definition of print

<K, V> void print(Map<K, V> items) { for (Map.Entry<K, V> entry : items.entrySet()) { print(entry.getKey() + " " + entry.getValue()); } }

where it is serialized to string already. So it does not do pretty print.

For example below is the output with current implementation:
default {127.0.0.1:3181=BookieInfoImpl(rack=/rack1, hostname=127.0.0.1)}

Output after the change:
{ "default": { "127.0.0.1:3181": { "rack": "/rack1", "hostname": "127.0.0.1" } }

@tisonkun
Copy link
Member

tisonkun commented Jun 8, 2023

@vineeth1995 here is a method you can make use of:

diff --git a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CliCommand.java b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CliCommand.java
index 7a6836eb74..c96b0bd436 100644
--- a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CliCommand.java
+++ b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CliCommand.java
@@ -214,13 +214,21 @@ public abstract class CliCommand {
             if (item instanceof String) {
                 System.out.println(item);
             } else {
-                System.out.println(writer.writeValueAsString(item));
+                prettyPrint(item);
             }
         } catch (Exception e) {
             throw new RuntimeException(e);
         }
     }
 
+    <T> void prettyPrint(T item) {
+        try {
+            System.out.println(writer.writeValueAsString(item));
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        }
+    }
+
     private static ObjectMapper mapper = ObjectMapperFactory.create();
     private static ObjectWriter writer = mapper.writerWithDefaultPrettyPrinter();
     private static Set<Character> sizeUnit = Sets.newHashSet('k', 'K', 'm', 'M', 'g', 'G', 't', 'T');
diff --git a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdBookies.java b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdBookies.java
index 8c0b4d80ae..27502a305a 100644
--- a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdBookies.java
+++ b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdBookies.java
@@ -23,9 +23,6 @@ import com.beust.jcommander.ParameterException;
 import com.beust.jcommander.Parameters;
 import com.google.common.base.Strings;
 import java.util.function.Supplier;
-
-import com.google.gson.Gson;
-import com.google.gson.GsonBuilder;
 import lombok.NonNull;
 import org.apache.pulsar.client.admin.PulsarAdmin;
 import org.apache.pulsar.common.policies.data.BookieInfo;
@@ -38,8 +35,7 @@ public class CmdBookies extends CmdBase {
 
         @Override
         void run() throws Exception {
-            Gson gson = new GsonBuilder().setPrettyPrinting().create();
-            System.out.println(gson.toJson(getAdmin().bookies().getBookiesRackInfo()));
+            prettyPrint(getAdmin().bookies().getBookiesRackInfo());
         }
     }

@tisonkun
Copy link
Member

tisonkun commented Jun 8, 2023

I don't know why we treat String, List and Map specifically so I don't just print all types with pretty printer.

But it should be simple to follow #20516 (comment) that we don't sometimes use Jackson and the other time use Gson. It's too casual and the hack can create more chaos.

@vineeth1995
Copy link
Contributor Author

@tisonkun I made the requested changes. Can you please approve?

@codecov-commenter
Copy link

Codecov Report

Merging #20516 (cd1b036) into master (3b862ae) will increase coverage by 36.11%.
The diff coverage is 61.84%.

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #20516       +/-   ##
=============================================
+ Coverage     36.78%   72.90%   +36.11%     
- Complexity    12059    31917    +19858     
=============================================
  Files          1690     1867      +177     
  Lines        129001   138565     +9564     
  Branches      14041    15219     +1178     
=============================================
+ Hits          47453   101017    +53564     
+ Misses        75294    29525    -45769     
- Partials       6254     8023     +1769     
Flag Coverage Δ
inttests 24.16% <14.47%> (+0.02%) ⬆️
systests 24.90% <0.00%> (-0.06%) ⬇️
unittests 72.20% <61.84%> (+40.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../apache/pulsar/broker/admin/impl/PackagesBase.java 76.19% <ø> (+23.89%) ⬆️
...rg/apache/pulsar/broker/service/BrokerService.java 80.59% <ø> (+24.26%) ⬆️
.../apache/pulsar/functions/instance/ContextImpl.java 61.00% <0.00%> (+18.41%) ⬆️
...s/runtime/kubernetes/KubernetesRuntimeFactory.java 80.21% <50.00%> (+80.21%) ⬆️
...pache/pulsar/broker/admin/impl/NamespacesBase.java 72.05% <56.86%> (+41.36%) ⬆️
...n/java/org/apache/pulsar/admin/cli/CliCommand.java 59.55% <66.66%> (+59.55%) ⬆️
...ker/authorization/PulsarAuthorizationProvider.java 71.42% <100.00%> (+39.74%) ⬆️
.../org/apache/pulsar/broker/admin/AdminResource.java 78.85% <100.00%> (+35.05%) ⬆️
...pulsar/broker/admin/impl/PersistentTopicsBase.java 63.91% <100.00%> (+53.73%) ⬆️
...n/java/org/apache/pulsar/admin/cli/CmdBookies.java 87.50% <100.00%> (+87.50%) ⬆️

... and 1426 files with indirect coverage changes

@tisonkun
Copy link
Member

tisonkun commented Jun 8, 2023

Merging...

Thanks for your contribution @vineeth1995!

@tisonkun tisonkun merged commit c39f7bb into apache:master Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/admin doc-not-needed Your PR changes do not impact docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants