1. Eskimo Introduction

A state of the art Big Data Infrastructure and Management Web Console to build, manage and operate Big Data 2.0 Analytics clusters

Eskimo is in a certain way the Operating System of your Big Data Cluster:

• A plug and play, working out of the Box, Big Data Analytics platform fulfilling enterprise environment requirements.

• A state of the art Big Data 2.0 platform

  • based on Docker, Marathon, Mesos and Systemd

  • packaging Gluster, Spark, Kafka, Flink, Nifi and ElasticSearch

  • with all the administration and management consoles such as Cerebro, Kibana, Zeppelin, Kafka-Manager, Grafana and Prometheus.

• An Administration Application aimed at drastically simplifying the deployment, administration and operation of your Big Data Cluster

• A Data Science Laboratory and Production environment where Data Analytics is both

  • developed and

  • operated in production

Eskimo is as well:

• a collection of ready to use docker containers packaging fine-tuned and highly customized plug and play services with all the nuts and bolds required to make them work well together.

• a framework for developing, building and deploying Big Data and NoSQL services based on Docker, SystemD and Marathon.

Eskimo Packages all the components above and takes care of all the nuts and bolts required to make them work together smoothly and without the user or administrator needing to take care of anything.

1.1. Eskimo CE Project Development Status

GitHub Action CI Integration Build Status

SonarCloud Quality Checks Status

2. Information

Reach http://www.eskimo.sh for more information on Eskimo or look at the documentation in the folder doc.

3. Building eskimo

3.1. Requirements

Eskimo uses plain old Apache Maven for building (https://maven.apache.org/) as well as JDK 11+ (https://openjdk.java.net/) for both building and execution.

Minimum requirements are as follows:

• JDK 11 or greater

• Apache maven 3.5 or greater.

Every other dependency will be downloaded during the maven build process.

Theoretically, Eskimo can be built on any Operating System providing a command line and supporting Java 11+ (Note : previous versions of JDK down to 8 may be supported without any guarantee) and Maven 3+.
However, the eskimo source tree makes an extensive use of symbolic links wich - unfortunately - are not supported on Microsoft Windows.
For this reason, building eskimo on windows is absolutely possible but it is highly discouraged to develop eskimo on Windows.

3.2. Building using maven

In order to build eskimo, simply run the following command in this very folder (the one containing this readme.adoc file):

Build Eskimo

mvn clean install

As a result of the build process, Eskimo is available as an extractable archive in eskimo-version-bin.zip or eskimo-version-bin.tar.gz in the folder target.

3.3. Specific hints and troubleshooting information regarding eskimo building

You might want to read carefully the following additional information related to building eskimo.

3.3.1. Put maven and Java in PATH

Of course, for the above command to work, you need to have java and mvn in your path.

Use for instance the following commands on Linux:

Put maven and java in PATH on Linux

export JAVA_HOME=/usr/local/lib/jdk-11
export MAVEN_HOME=/usr/local/lib/apache-maven-3.5.3
export PATH=$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH

(You might want to put above commands in your /etc/profile or /etc/bash.bashrc)

And for instance the following commands on Windows

Put maven and java in PATH on Windows

set JAVA_HOME=C:\programs\jdk-11
set MAVEN_HOME=C:\programs\apache-maven-3.6.1
set PATH=%MAVEN_HOME%\bin;%JAVA_HOME%\bin;%PATH%

3.3.2. Install maven dependency trilead-ssh2

Eskimo used the library trilead-ssh2 version build-217-jenkins-27 which is not available in standard maven repositories.
This library is bundled with the eskimo sources in the folder lib.
One can install it in his own local maven repository using the following command:

Install trilead-ssh2 library

cd libs
bash install_libs.sh

3.4. Eskimo Build result

The build results at the end of the maven build process is located in the folder `target.
The results are two archive files:

• A zip archive : eskimo-version-bin.zip

• A tarball archive : eskimo-version-bin.tar.gz

You can find pre-built packages of Eskimo on https://www.eskimo.sh.

The archive contains the full software packages with all dependencies and command line executables required to start it as well as the full documentation, etc.

4. Running Eskimo

This section gives instructions about running eskimo.

Eskimo itself can run on any Operating System supporting a JVM (Linux, Windows, Mac OSX) but cluster nodes are only supported running Linux.

4.1. Prerequisites

4.1.1. System requirements

In order to run eskimo, one needs to have

• At least 15Gb of disk storage space on the machine running Eskimo

• At least one linux machine available on the network (can be the same machine than the one running Eskimo) with either a debian-based (Ubuntu, Debian, etc.), red-hat-based (CentOS, Fedora, etc.) or OpenSUSE Operating System.

(See The Eskimo User Guide for more information on Eskimo cluster requirements.)

Eskimo is reached using a web browser (see startup logs). Supported web browsers are:

• Microsoft Edge 12 or greater

• Mozilla FireFox 36 or greater

• Google Chrome 41 or greater

Note: there may be other browsers / versions supported (Safari, Opera but they are not certified to work with Eskimo)

4.1.2. Available JVM with Java in PATH

In order to run eskimo, one needs to have java in the path.

Use for instance the following commands on Linux:

Put java in PATH on Linux

export JAVA_HOME=/usr/local/lib/jdk-11
export PATH=$JAVA_HOME/bin:$PATH

(You might want to put above commands in your /etc/profile or /etc/bash.bashrc)

And for instance the following commands on Windows

Put java in PATH on Windows

set JAVA_HOME=C:\programs\jdk-11
set PATH=%JAVA_HOME%\bin;%PATH%

4.2. Extract archive.

After building eskimo using maven, the zip and tarball archives are located in the folder target.

One of these archives needs to be extracted on the local filesystem.

Then in the folder bin under the newly extracted eskimo binary distribution folder, one can find two scripts:

• a script eskimo.bat to execute eskimo on Windows

• a script eskimo.sh to execute eskimo on Linux.

4.3. Access eskimo

With eskimo properly started using the above scripts, one can reach eskimo using http://machine_ip:9191.
The default port number is 9191. This can be changed in configuration file eskimo.properties.

The default login / password credentials are admin / password.

4.4. Typical startup issues

Several issues can happen upon first eskimo startup.
This section describes common issues and ways to resolved them.

4.4.1. eskimo-users.json cannot be written

If you meet an error as the following on startup:

Impossible to write eskimo-users.json

Caused by: ch.niceideas.common.utils.FileException: ./eskimo-users.json (Unauthorized access)
        at ch.niceideas.common.utils.FileUtils.writeFile(FileUtils.java:154)
        at ch.niceideas.eskimo.security.JSONBackedUserDetailsManager.<init>(JSONBackedUserDetailsManager.java:81)
        at ch.niceideas.eskimo.configurations.WebSecurityConfiguration.userDetailsService(WebSecurityConfiguration.java:127)
        ... 50 more
Caused by: java.io.FileNotFoundException: ./eskimo-users.json (Unauthorized access)
        at java.base/java.io.FileOutputStream.open0(Native Method)
        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:276)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:220)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:170)
        at java.base/java.io.FileWriter.<init>(FileWriter.java:90)
        at ch.niceideas.common.utils.FileUtils.writeFile(FileUtils.java:149)
        ... 52 more

Eskimo uses a local file to define users and access credentials. Upon first startup, if that file doesn’t exist already, it is created by eskimo (with the default credentials above) at the path pointed to by the property security.userJsonFile in eskimo.properties.

If you experience the error above or something alike, change that propery to point to a location where the first version of the file can successfully be created.

5. Eskimo source distribution layout

The directory structure of the eskimo source distribution is as follows:

• doc contains the source documentation in asciidoc format

• libs contains dependencies not available in standard maven repositories

• packages_dev contains the docker images development framework and packages

• packages_distrib is the destination folder in which downloaded or built images are placed

• services_setup contains the services installation framework and packages

• src contains the source files to build eskimo

• test_lab contains various tools to build VMs aimed at testing eskimo

6. Test Laboratory

The folder test_lab folder in the Eskimo CE project root folder contains a Vagrant framework used to create virtual machines to test Eskimo.

The supported virtual machines are declared in the file VagrantFile in the variable nodes:

Vagrant Virtual Machines declaration

# Define cluster nodes
nodes = [
  { :hostname => 'deb-node1',  :box => 'debian/buster64',
                               :ip => '192.168.10.11', :ram => 8000 },
  { :hostname => 'deb-node2',  :box => 'ubuntu/xenial64',
                               :ip => '192.168.10.12', :ram => 6000 },
  { :hostname => 'cent-node1', :box => 'centos/7',
                               :ip => '192.168.10.13', :ram => 6000},
  { :hostname => 'cent-node2', :box => 'fedora/29-cloud-base',
                               :ip => '192.168.10.14', :ram => 6000 }
]

The defined VMs use different Operating Systems to test Eskimo’s installation on different OSes.
One can edit this variable to declare one’s own VM’s as required.

An individual VM is then started with vargrant using VirtualBox as follows:

Start deb-node1 with VirtualBox

$ cd ./test_lab/vagrant/
$ vagrant up deb-node1

Vagrant can also start VM’s using libvirt / QEMU / kvm if the required libraries and provider are properly installed:

Start deb-node1 with libvirt

$ cd ./test_lab/vagrant/
$ vagrant up deb-node1 --provider=libvirt

7. Further information

Look at the following files for more information

• The Eskimo User Guide

• readme.adoc in the folder packages_dev for information about the docker packages building framework

• readme.adoc in the folder services_setup for information about the services installation and operation framework.

Appendix A: Copyright and License

Eskimo is Copyright 2019 - 2022 eskimo.sh - All rights reserved.
Author : http://www.eskimo.sh

Eskimo is available under a dual licensing model : commercial and GNU AGPL.
If you did not acquire a commercial licence for Eskimo, you can still use it and consider it free software under the terms of the GNU Affero Public License. You can redistribute it and/or modify it under the terms of the GNU Affero Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
Compliance to each and every aspect of the GNU Affero Public License is mandatory for users who did no acquire a commercial license.

Eskimo is distributed as a free software under GNU AGPL in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero Public License for more details.

You should have received a copy of the GNU Affero Public License along with Eskimo. If not, see https://www.gnu.org/licenses/ or write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA, 02110-1301 USA.

You can be released from the requirements of the license by purchasing a commercial license. Buying such a commercial license is mandatory as soon as :

• you develop activities involving Eskimo without disclosing the source code of your own product, software, platform, use cases or scripts.

• you deploy eskimo as part of a commercial product, platform or software.

For more information, please contact eskimo.sh at https://www.eskimo.sh

The above copyright notice and this licensing notice shall be included in all copies or substantial portions of the Software.