HBASE-20060 Add details of off heap memstore into book. (#334)

Author: openinx

hbase-common/src/main/resources/hbase-default.xml

@@ -696,7 +696,30 @@ possible configurations would overwhelm and obscure the important.
       Enables the MemStore-Local Allocation Buffer,
       a feature which works to prevent heap fragmentation under
       heavy write loads. This can reduce the frequency of stop-the-world
-      GC pauses on large heaps.</description>
+      GC pauses on large heaps.
+    </description>
+  </property>
+  <property>
+    <name>hbase.hregion.memstore.mslab.chunksize</name>
+    <value>2097152</value>
+    <description>The maximum byte size of a chunk in the MemStoreLAB. Unit: bytes</description>
+  </property>
+  <property>
+    <name>hbase.regionserver.offheap.global.memstore.size</name>
+    <value>0</value>
+    <description>The amount of off-heap memory all MemStores in a RegionServer may use.
+      A value of 0 means that no off-heap memory will be used and all chunks in MSLAB
+      will be HeapByteBuffer, otherwise the non-zero value means how many megabyte of
+      off-heap memory will be used for chunks in MSLAB and all chunks in MSLAB will be
+      DirectByteBuffer. Unit: megabytes.
+    </description>
+  </property>
+  <property>
+    <name>hbase.hregion.memstore.mslab.max.allocation</name>
+    <value>262144</value>
+    <description>The maximal size of one allocation in the MemStoreLAB, if the desired byte
+      size exceed this threshold then it will be just allocated from JVM heap rather than MemStoreLAB.
+    </description>
   </property>
   <property>
     <name>hbase.hregion.max.filesize</name>

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java

@@ -160,19 +160,23 @@ Cell getNextRow(final Cell cell) {
         getNextRow(cell, this.snapshot.getCellSet()));
   }
 
-  @Override public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent) {
+  @Override
+  public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent) {
   }
 
-  @Override protected boolean preUpdate(MutableSegment currentActive, Cell cell,
+  @Override
+  protected boolean preUpdate(MutableSegment currentActive, Cell cell,
       MemStoreSizing memstoreSizing) {
     return true;
   }
 
-  @Override protected void postUpdate(MutableSegment currentActive) {
+  @Override
+  protected void postUpdate(MutableSegment currentActive) {
     return;
   }
 
-  @Override protected boolean sizeAddedPreOperation() {
+  @Override
+  protected boolean sizeAddedPreOperation() {
     return false;
   }
 
@@ -186,7 +190,8 @@ public long preFlushSeqIDEstimation() {
     return HConstants.NO_SEQNUM;
   }
 
-  @Override public boolean isSloppy() {
+  @Override
+  public boolean isSloppy() {
     return false;
   }
 

src/main/asciidoc/_chapters/offheap_read_write.adoc

@@ -183,4 +183,23 @@ Please also see: link:https://issues.apache.org/jira/browse/HBASE-22483[HBASE-22
 [[regionserver.offheap.writepath]]
 == Offheap write-path
 
-TODO
+In HBase 2.0.0, link:https://issues.apache.org/jira/browse/HBASE-15179[HBASE-15179] made the HBase write path to work off-heap. By default, the MemStores use
+MSLAB to avoid memory fragmentation. It creates bigger fixed sized chunks and memstore cell's data will get copied into these chunks. These chunks can be pooled
+also and from 2.0.0 the MSLAB (MemStore-Local Allocation Buffer) pool is by default ON. Write off-heaping makes use of the MSLAB pool. It creates MSLAB chunks
+as Direct ByteBuffers and pools them. HBase defaults to using no off-heap memory for MSLAB which means that cells are copied to heap chunk in MSLAB by default
+rather than off-heap chunk.
+
+`hbase.regionserver.offheap.global.memstore.size` is the configuration key which controls the amount of off-heap data whose value is the number of megabytes
+of off-heap memory that should be by MSLAB (e.g. `25` would result in 25MB of off-heap). Be sure to increase `HBASE_OFFHEAPSIZE` which will set the JVM's
+MaxDirectMemorySize property. Its default value is 0, means MSLAB use heap chunks.
+
+`hbase.hregion.memstore.mslab.chunksize` controls the size of each off-heap chunk, defaulting to `2097152` (2MB).
+
+When a Cell is added to a MemStore, the bytes for that Cell are copied into these off-heap buffers (if set the `hbase.regionserver.offheap.global.memstore.size` to non-zero)
+and a Cell POJO will refer to this memory area. This can greatly reduce the on-heap occupancy of the MemStores and reduce the total heap utilization for RegionServers
+in a write-heavy workload. On-heap and off-heap memory utiliazation are tracked at multiple levels to implement low level and high level memory management.
+The decision to flush a MemStore considers both the on-heap and off-heap usage of that MemStore. At the Region level, the sum of the on-heap and off-heap usages and
+compares them against the region flush size (128MB, by default). Globally, on-heap size occupancy of all memstores are tracked as well as off-heap size. When any of
+these sizes breaches the lower mark (`hbase.regionserver.global.memstore.size.lower.limit`) or the maximum size `hbase.regionserver.global.memstore.size`), all
+regions are selected for forced flushes.
+