Skip to content

Commit

Permalink
ISSUE apache#1799: bkctl is broken with default build options
Browse files Browse the repository at this point in the history
Descriptions of the changes in this PR:

*Motivation*

`bkctl` is designed in a modularized way for extensibility.
it loads command groups via ServiceLoader. However current build
profile doesn't leverage this extensibility. Instead it hardcodes
all the commands in one service load file. So if `bkctl` is built
without `-Dstream`, it will broke.

*Changes*

- move `ledger` related commands to `tools/ledger`
- generate the service load file by concating `commands` files from each module's resources directory.

*Results*

bkctl works for all build profiles.

- without `-Dstream`

```
$ bin/bkctl

bkctl interacts and operates Apache BookKeeper clusters

Usage:  bkctl [flags] [command group] [commands]

Commands:

    bookie      Commands on operating a single bookie
    bookies     Commands on operating a cluster of bookies
    cookie      Commands on operating cookies
    ledger      Commands on interacting with ledgers

    help        Display help information about it

Flags:

    -c, --conf
        Configuration file

    -n, --namespace
        Namespace scope to run commands (only valid for table service for now)

    -u, --service-uri
        Service Uri

    -h, --help
        Display help information

Use "bkctl [command] --help" or "bkctl help [command]" for more information
about a command
```

- with `-Dstream`

```
$ bin/bkctl

bkctl interacts and operates Apache BookKeeper clusters

Usage:  bkctl [flags] [command group] [commands]

Commands:

    bookie          Commands on operating a single bookie
    bookies         Commands on operating a cluster of bookies
    cluster         Commands on administrating bookkeeper clusters
    cookie          Commands on operating cookies
    ledger          Commands on interacting with ledgers
    namespace       Commands on operating namespaces
    table           Commands on interacting with tables
    tables          Commands on operating tables

    help            Display help information about it

Flags:

    -c, --conf
        Configuration file

    -n, --namespace
        Namespace scope to run commands (only valid for table service for now)

    -u, --service-uri
        Service Uri

    -h, --help
        Display help information

Use "bkctl [command] --help" or "bkctl help [command]" for more information
about a command
```

Master Issue: apache#1799 




Reviewers: Ivan Kelly <ivank@apache.org>, Enrico Olivelli <eolivelli@gmail.com>

This closes apache#1800 from sijie/refactor_bkctl, closes apache#1799
  • Loading branch information
sijie authored and eolivelli committed Nov 12, 2018
1 parent 398aa4c commit faeb9bb
Show file tree
Hide file tree
Showing 31 changed files with 164 additions and 44 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,8 @@ data/
# Pid files
**/*.pid

# files are generated under following directories
tools/all/src/main/resources

# Exclude versionBackup file (generated by `mvn versions:set`)
**/*.versionsBackup
10 changes: 9 additions & 1 deletion site/docs/latest/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ Once you have the BookKeeper on your local machine, either by [downloading](#dow
$ mvn package
```

Since 4.8.0, bookkeeper introduces `table service`. If you would like to build and tryout table service, you can build it with `stream` profile.

```shell
$ mvn package -Dstream
```

> You can skip tests by adding the `-DskipTests` flag when running `mvn package`.
### Useful Maven commands
Expand All @@ -60,7 +66,9 @@ Command | Action
`mvn verify` | Performs a wide variety of verification and validation tasks
`mvn apache-rat:check` | Run Maven using the [Apache Rat](http://creadur.apache.org/rat/apache-rat-plugin/) plugin
`mvn compile javadoc:aggregate` | Build Javadocs locally
`mvn package assembly:single` | Build a complete distribution using the Maven [Assembly](http://maven.apache.org/plugins/maven-assembly-plugin/) plugin
`mvn -am -pl bookkeeper-dist/server package` | Build a server distribution using the Maven [Assembly](http://maven.apache.org/plugins/maven-assembly-plugin/) plugin

> You can enable `table service` by adding the `-Dstream` flag when running above commands.
## Package directory

Expand Down
111 changes: 72 additions & 39 deletions tools/all/pom.xml
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
<?xml version="1.0"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
Expand All @@ -27,33 +29,37 @@
<dependencies>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-tools-framework</artifactId>
<artifactId>bookkeeper-tools-ledger</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-server</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>buildtools</artifactId>
<version>${project.parent.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-server</artifactId>
<type>test-jar</type>
<version>${project.parent.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>${maven-antrun-plugin.version}</version>
<executions>
<execution>
<id>append-ledger-commands</id>
<phase>generate-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<concat destfile="${project.basedir}/src/main/resources/META-INF/services/org.apache.bookkeeper.tools.framework.CommandGroup"
overwrite="true">
<filelist dir="${project.basedir}/../ledger/src/main/resources" files="commands">
</filelist>
</concat>
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>stream</id>
Expand All @@ -70,6 +76,33 @@
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>${maven-antrun-plugin.version}</version>
<executions>
<execution>
<id>append-stream-commands</id>
<phase>generate-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<concat destfile="${project.basedir}/src/main/resources/META-INF/services/org.apache.bookkeeper.tools.framework.CommandGroup"
append="true">
<filelist dir="${project.basedir}/../stream/src/main/resources" files="commands">
</filelist>
</concat>
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
57 changes: 57 additions & 0 deletions tools/ledger/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<?xml version="1.0"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>bookkeeper-tools-parent</artifactId>
<groupId>org.apache.bookkeeper</groupId>
<version>4.9.0-SNAPSHOT</version>
</parent>
<artifactId>bookkeeper-tools-ledger</artifactId>
<name>Apache BookKeeper :: Tools :: Ledger</name>
<dependencies>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-tools-framework</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-server</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>buildtools</artifactId>
<version>${project.parent.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-server</artifactId>
<type>test-jar</type>
<version>${project.parent.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,3 @@ org.apache.bookkeeper.tools.cli.commands.BookieCommandGroup
org.apache.bookkeeper.tools.cli.commands.BookiesCommandGroup
org.apache.bookkeeper.tools.cli.commands.CookieCommandGroup
org.apache.bookkeeper.tools.cli.commands.LedgerCommandGroup
org.apache.bookkeeper.stream.cli.ClusterCommandGroup
org.apache.bookkeeper.stream.cli.NamespaceCommandGroup
org.apache.bookkeeper.stream.cli.TableAdminCommandGroup
org.apache.bookkeeper.stream.cli.TableCommandGroup
1 change: 1 addition & 0 deletions tools/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
<packaging>pom</packaging>
<modules>
<module>framework</module>
<module>ledger</module>
<module>all</module>
</modules>
<profiles>
Expand Down
22 changes: 22 additions & 0 deletions tools/stream/src/main/resources/commands
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

org.apache.bookkeeper.stream.cli.ClusterCommandGroup
org.apache.bookkeeper.stream.cli.NamespaceCommandGroup
org.apache.bookkeeper.stream.cli.TableAdminCommandGroup
org.apache.bookkeeper.stream.cli.TableCommandGroup

0 comments on commit faeb9bb

Please sign in to comment.