Skip to content

Commit 78bae9e

Browse files
brfrn169brfrn169
authored
HBASE-11062 hbtop (#476)
Signed-off-by: Stack <stack@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org> Signed-off-by: Josh Elser <elserj@apache.org> Signed-off-by: Andrew Purtell <apurtell@apache.org>
1 parent 51b6ce3 commit 78bae9e

File tree

89 files changed

+9815
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

89 files changed

+9815
-1
lines changed

‎bin/hbase

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ if [ $# = 0 ]; then
120120
echo " rowcounter Run RowCounter tool"
121121
echo " cellcounter Run CellCounter tool"
122122
echo " pre-upgrade Run Pre-Upgrade validator tool"
123+
echo " hbtop Run HBTop tool"
123124
echo " CLASSNAME Run the class named CLASSNAME"
124125
exit 1
125126
fi
@@ -223,7 +224,7 @@ if [ "${INTERNAL_CLASSPATH}" != "true" ]; then
223224
done
224225

225226
# If command can use our shaded client, use it
226-
declare -a commands_in_client_jar=("classpath" "version")
227+
declare -a commands_in_client_jar=("classpath" "version" "hbtop")
227228
for c in "${commands_in_client_jar[@]}"; do
228229
if [ "${COMMAND}" = "${c}" ]; then
229230
if [ -n "${HADOOP_IN_PATH}" ] && [ -f "${HADOOP_IN_PATH}" ]; then
@@ -656,6 +657,24 @@ elif [ "$COMMAND" = "pre-upgrade" ] ; then
656657
CLASS='org.apache.hadoop.hbase.tool.PreUpgradeValidator'
657658
elif [ "$COMMAND" = "completebulkload" ] ; then
658659
CLASS='org.apache.hadoop.hbase.tool.BulkLoadHFilesTool'
660+
elif [ "$COMMAND" = "hbtop" ] ; then
661+
CLASS='org.apache.hadoop.hbase.hbtop.HBTop'
662+
if [ -n "${shaded_jar}" ] ; then
663+
for f in "${HBASE_HOME}"/lib/hbase-hbtop*.jar; do
664+
if [ -f "${f}" ]; then
665+
CLASSPATH="${CLASSPATH}:${f}"
666+
break
667+
fi
668+
done
669+
for f in "${HBASE_HOME}"/lib/commons-lang3*.jar; do
670+
if [ -f "${f}" ]; then
671+
CLASSPATH="${CLASSPATH}:${f}"
672+
break
673+
fi
674+
done
675+
fi
676+
677+
HBASE_OPTS="${HBASE_OPTS} -Dlog4j.configuration=file:${HBASE_HOME}/conf/log4j-hbtop.properties"
659678
else
660679
CLASS=$COMMAND
661680
fi

‎conf/log4j-hbtop.properties

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
log4j.rootLogger=WARN,console
18+
log4j.threshold=WARN
19+
20+
# console
21+
log4j.appender.console=org.apache.log4j.ConsoleAppender
22+
log4j.appender.console.target=System.err
23+
log4j.appender.console.layout=org.apache.log4j.PatternLayout
24+
log4j.appender.console.layout.ConversionPattern=%d{ISO8601} %-5p [%t] %c{2}: %m%n
25+
26+
# ZooKeeper will still put stuff at WARN
27+
log4j.logger.org.apache.zookeeper=ERROR

‎hbase-assembly/pom.xml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,10 @@
309309
<groupId>org.apache.hbase</groupId>
310310
<artifactId>hbase-zookeeper</artifactId>
311311
</dependency>
312+
<dependency>
313+
<groupId>org.apache.hbase</groupId>
314+
<artifactId>hbase-hbtop</artifactId>
315+
</dependency>
312316
<dependency>
313317
<groupId>jline</groupId>
314318
<artifactId>jline</artifactId>

‎hbase-hbtop/README.md

Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
<!---
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# hbtop
21+
22+
## Overview
23+
24+
`hbtop` is a real-time monitoring tool for HBase like Unix's top command.
25+
It can display summary information as well as metrics per Region/Namespace/Table/RegionServer.
26+
In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see.
27+
Also, with the drill-down feature, you can find hot regions easily in a top-down manner.
28+
29+
## Usage
30+
31+
You can run hbtop with the following command:
32+
33+
```
34+
$ hbase hbtop
35+
```
36+
37+
In this case, the values of `hbase.client.zookeeper.quorum` and `zookeeper.znode.parent` in `hbase-site.xml` in the classpath or the default values of them are used to connect.
38+
39+
Or, you can specify your own zookeeper quorum and znode parent as follows:
40+
41+
```
42+
$ hbase hbtop -Dhbase.client.zookeeper.quorum=<zookeeper quorum> -Dzookeeper.znode.parent=<znode parent>
43+
```
44+
45+
![Top screen](img/top_screen.gif "Top screen")
46+
47+
The top screen consists of a summary part and of a metrics part.
48+
In the summary part, you can see `HBase Version`, `Cluster ID`, `The number of region servers`, `Region count`, `Average Cluster Load` and `Aggregated Request/s`.
49+
In the metrics part, you can see metrics per Region/Namespace/Table/RegionServer depending on the selected mode.
50+
The top screen is refreshed in a certain period – 3 seconds by default.
51+
52+
### Scrolling metric records
53+
54+
You can scroll the metric records in the metrics part.
55+
56+
![Scrolling metric records](img/scrolling_metric_records.gif "Scrolling metric records")
57+
58+
### Command line arguments
59+
60+
| Argument | Description |
61+
|---|---|
62+
| -d,--delay &lt;arg&gt; | The refresh delay (in seconds); default is 3 seconds |
63+
| -h,--help | Print usage; for help while the tool is running press `h` key |
64+
| -m,--mode &lt;arg&gt; | The mode; `n` (Namespace)&#124;`t` (Table)&#124;r (Region)&#124;`s` (RegionServer), default is `r` (Region) |
65+
66+
### Modes
67+
68+
There are the following 4 modes in hbtop:
69+
70+
| Mode | Description |
71+
|---|---|
72+
| Region | Showing metric records per region |
73+
| Namespace | Showing metric records per namespace |
74+
| Table | Showing metric records per table |
75+
| RegionServer | Showing metric records per region server |
76+
77+
#### Region mode
78+
79+
In Region mode, the default sort field is `#REQ/S`.
80+
81+
The fields in this mode are as follows:
82+
83+
| Field | Description | Displayed by default |
84+
|---|---|---|
85+
| RNAME | Region Name | false |
86+
| NAMESPACE | Namespace Name | true |
87+
| TABLE | Table Name | true |
88+
| SCODE | Start Code | false |
89+
| REPID | Replica ID | false |
90+
| REGION | Encoded Region Name | true |
91+
| RS | Short Region Server Name | true |
92+
| LRS | Long Region Server Name | false |
93+
| #REQ/S | Request Count per second | true |
94+
| #READ/S | Read Request Count per second | true |
95+
| #FREAD/S | Filtered Read Request Count per second | true |
96+
| #WRITE/S | Write Request Count per second | true |
97+
| SF | StoreFile Size | true |
98+
| USF | Uncompressed StoreFile Size | false |
99+
| #SF | Number of StoreFiles | true |
100+
| MEMSTORE | MemStore Size | true |
101+
| LOCALITY | Block Locality | true |
102+
| SKEY | Start Key | false |
103+
| #COMPingCELL | Compacting Cell Count | false |
104+
| #COMPedCELL | Compacted Cell Count | false |
105+
| %COMP | Compaction Progress | false |
106+
| LASTMCOMP | Last Major Compaction Time | false |
107+
108+
#### Namespace mode
109+
110+
In Namespace mode, the default sort field is `#REQ/S`.
111+
112+
The fields in this mode are as follows:
113+
114+
| Field | Description | Displayed by default |
115+
|---|---|---|
116+
| NAMESPACE | Namespace Name | true |
117+
| #REGION | Region Count | true |
118+
| #REQ/S | Request Count per second | true |
119+
| #READ/S | Read Request Count per second | true |
120+
| #FREAD/S | Filtered Read Request Count per second | true |
121+
| #WRITE/S | Write Request Count per second | true |
122+
| SF | StoreFile Size | true |
123+
| USF | Uncompressed StoreFile Size | false |
124+
| #SF | Number of StoreFiles | true |
125+
| MEMSTORE | MemStore Size | true |
126+
127+
#### Table mode
128+
129+
In Table mode, the default sort field is `#REQ/S`.
130+
131+
The fields in this mode are as follows:
132+
133+
| Field | Description | Displayed by default |
134+
|---|---|---|
135+
| NAMESPACE | Namespace Name | true |
136+
| TABLE | Table Name | true |
137+
| #REGION | Region Count | true |
138+
| #REQ/S | Request Count per second | true |
139+
| #READ/S | Read Request Count per second | true |
140+
| #FREAD/S | Filtered Read Request Count per second | true |
141+
| #WRITE/S | Write Request Count per second | true |
142+
| SF | StoreFile Size | true |
143+
| USF | Uncompressed StoreFile Size | false |
144+
| #SF | Number of StoreFiles | true |
145+
| MEMSTORE | MemStore Size | true |
146+
147+
#### RegionServer mode
148+
149+
In RegionServer mode, the default sort field is `#REQ/S`.
150+
151+
The fields in this mode are as follows:
152+
153+
| Field | Description | Displayed by default |
154+
|---|---|---|
155+
| RS | Short Region Server Name | true |
156+
| LRS | Long Region Server Name | false |
157+
| #REGION | Region Count | true |
158+
| #REQ/S | Request Count per second | true |
159+
| #READ/S | Read Request Count per second | true |
160+
| #FREAD/S | Filtered Read Request Count per second | true |
161+
| #WRITE/S | Write Request Count per second | true |
162+
| SF | StoreFile Size | true |
163+
| USF | Uncompressed StoreFile Size | false |
164+
| #SF | Number of StoreFiles | true |
165+
| MEMSTORE | MemStore Size | true |
166+
| UHEAP | Used Heap Size | true |
167+
| MHEAP | Max Heap Size | true |
168+
169+
### Changing mode
170+
171+
You can change mode by pressing `m` key in the top screen.
172+
173+
![Changing mode](img/changing_mode.gif "Changing mode")
174+
175+
### Changing the refresh delay
176+
177+
You can change the refresh by pressing `d` key in the top screen.
178+
179+
![Changing the refresh delay](img/changing_refresh_delay.gif "Changing the refresh delay")
180+
181+
### Changing the displayed fields
182+
183+
You can move to the field screen by pressing `f` key in the top screen. In the fields screen, you can change the displayed fields by choosing a field and pressing `d` key or `space` key.
184+
185+
![Changing the displayed fields](img/changing_displayed_fields.gif "Changing the displayed fields")
186+
187+
### Changing the sort field
188+
189+
You can move to the fields screen by pressing `f` key in the top screen. In the field screen, you can change the sort field by choosing a field and pressing `s`. Also, you can change the sort order (ascending or descending) by pressing `R` key.
190+
191+
![Changing the sort field](img/changing_sort_field.gif "Changing the sort field")
192+
193+
194+
### Changing the order of the fields
195+
196+
You can move to the fields screen by pressing `f` key in the top screen. In the field screen, you can change the order of the fields.
197+
198+
![Changing the order of the fields](img/changing_order_of_fields.gif "Changing the sort field")
199+
200+
### Filters
201+
202+
You can filter the metric records with the filter feature. We can add filters by pressing `o` key for ignoring case or `O` key for case sensitive.
203+
204+
![Adding filters](img/adding_filters.gif "Adding filters")
205+
206+
The syntax is as follows:
207+
```
208+
<Field><Operator><Value>
209+
```
210+
211+
For example, we can add filters like the following:
212+
```
213+
NAMESPACE==default
214+
REQ/S>1000
215+
```
216+
217+
The operators we can specify are as follows:
218+
219+
| Operator | Description |
220+
|---|---|
221+
| = | Partial match |
222+
| == | Exact match |
223+
| > | Greater than |
224+
| >= | Greater than or equal to |
225+
| < | Less than |
226+
| <= | Less than and equal to |
227+
228+
You can see the current filters by pressing `^o` key and clear them by pressing `=` key.
229+
230+
![Showing and clearing filters](img/showing_and_clearing_filters.gif "Showing and clearing filters")
231+
232+
### Drilling down
233+
234+
You can drill down the metric record by choosing a metric record that you want to drill down and pressing `i` key in the top screen. With this feature, you can find hot regions easily in a top-down manner.
235+
236+
![Drilling down](img/driling_down.gif "Drilling down")
237+
238+
### Help screen
239+
240+
You can see the help screen by pressing `h` key in the top screen.
241+
242+
![Help screen](img/help_screen.gif "Help screen")
243+
244+
## Others
245+
246+
### How hbtop gets the metrics data
247+
248+
hbtop gets the metrics from ClusterMetrics which is returned as the result of a call to Admin#getClusterMetrics() on the current HMaster. To add metrics to hbtop, they will need to be exposed via ClusterMetrics.

‎hbase-hbtop/img/adding_filters.gif

2.64 MB
Loading

‎hbase-hbtop/img/changing_displayed_fields.gif

2.5 MB
Loading

‎hbase-hbtop/img/changing_mode.gif

3.19 MB
Loading

‎hbase-hbtop/img/changing_order_of_fields.gif

2.75 MB
Loading

‎hbase-hbtop/img/changing_refresh_delay.gif

2.19 MB
Loading

‎hbase-hbtop/img/changing_sort_field.gif

3.28 MB
Loading

‎hbase-hbtop/img/driling_down.gif

3.82 MB
Loading

‎hbase-hbtop/img/help_screen.gif

909 KB
Loading

‎hbase-hbtop/img/scrolling_metric_records.gif

6.27 MB
Loading

‎hbase-hbtop/img/showing_and_clearing_filters.gif

1.31 MB
Loading

‎hbase-hbtop/img/top_screen.gif

2.05 MB
Loading

0 commit comments

Comments
 (0)