Skip to content

Commit

Permalink
Major Revision - Python3 - See updated Readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Scott Vintinner committed Dec 24, 2016
1 parent 36be2a4 commit a519310
Show file tree
Hide file tree
Showing 102 changed files with 8,582 additions and 1,355 deletions.
4 changes: 2 additions & 2 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2014 M. Scott Vintinner
Copyright (c) 2016 M. Scott Vintinner

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand All @@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
192 changes: 133 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,142 +1,216 @@
SysAdminBoard
=======================

SysAdminBoard is a simple dashboard system written in Python, HTML and Javascript and served on a simple [CherryPy Webserver](http://cherrypy.org/) (included).
SysAdminBoard is a simple dashboard system written in Python, HTML and Javascript and served on a simple [CherryPy Webserver](http://cherrypy.org/) (included). It was originally written to reformat snmp data for the [Panic Statusboard iPad App](http://www.panic.com/Statusboard/), but has since become a fully stand-alone project that can grab data from a variety of sources and render charts and graphs in a web browser.

It was originally written for the [Panic StatusBoard iPad App](http://www.panic.com/statusboard/), but I had some stability problems that forced me to restart the iPad at least once a day. To work around this problem, I replaced our iPads with a couple ASUS Chromeboxes and modified the code to run as a simple web page. I've added new HTML pages to replicate the clock, twitter, weather and graph functionality of the iPad app.

This project is not intended to be a ready-to-deploy solution. However, if you are comfortable with Python, you should be able to get this working without much effort.

Here is a photo our installation:
Here is a older photo our installation. We use some small ASUS chromeboxes along with the [Chrome Kiosk extension](https://chrome.google.com/webstore/detail/kiosk/afhcomalholahplbjhnmahkoekoijban?hl=en) to display our dashboards.

![Network Services Room](readme-images/location1.jpg)

##Features
###VMware vSphere ESX Host Monitoring
This module will talk to a VMware vSphere vCenter server using VMware APIs to get data about ESX hosts. Items are sorted based on top CPU usage over a 30 minute period.

![ESX Host Gadget](readme-images/host.png)

This code will talk to a VMware vSphere vCenter server using VMware APIs to get data about ESX hosts. Items are sorted based on top CPU usage over a 30 minute period.
###VMware vSphere VM Monitoring
This module will talk to a VMware vSphere vCenter server using VMware APIs to get data about the top ESX VMs. Items are sorted based on top CPU usage over a 30 minute period.

![VMware VM Gadget](readme-images/vm.png)

This code will talk to a VMware vSphere vCenter server using VMware APIs to get data about the top ESX VMs. Items are sorted based on top CPU usage over a 30 minute period.
###Network Bandwidth Monitoring

This module demonstrates how to grab network bandwidth data. In the first image, it is pulling in snmp data from a Palo Alto Firewall. In the second image, it is pulling snmp data some VLANs on a Cisco Catalyst 6880 switch.

![Network Bandwidth Monitoring Gadget](readme-images/network.png)
![Network Bandwidth Monitoring Gadget #2](readme-images/network2.png)

###SNMP Network Monitoring

This module demonstrates how to grab SNMP table data. In this case, it is pulling in ISDN values from a Cisco Voice Gateway to show the number of active phone calls.

![SNMP Network Monitoring Gadget](readme-images/snmp.png)

This code generates JSON data only that is consumed by the Statusboard iPad app's built in graph function.

(New) I've written an HTML page that will also display graph data using [Chartjs](http://chartjs.org/) (see the VNX Storage Pool IOPS screenshot below).

###SNMP Temperature Gadget
![SNMP Temperature Gadget](readme-images/temp.png)

This code talks to a couple different APC devices to pull in temperature, humidity, voltage and runtime data.
###EMC VNX Monitoring
![EMC VNX Monitoring Gadget](readme-images/vnx2.png)

This code talks to an EMC VNX Reporting and Monitoring web server to pull down performance data.
This module talks to a couple different APC devices to pull in temperature, humidity, voltage and runtime data.

![SNMP Temperature Gadget](readme-images/temp.png)

###Exchange Monitoring
![Exchange Monitoring Gadget](readme-images/exch.png)

This code monitors a Microsoft Exchange server to display SMTP message totals for the day along with RPC and latency information (per CAS server). Note that this code requires my [pyPerfmon](https://github.com/flakshack/pyPerfmon) app running on each Exchange server to be monitored.
This module monitors a Microsoft Exchange server to display SMTP message totals for the day along with RPC and latency information (per CAS server). Note that this code requires my [pyPerfmon](https://github.com/flakshack/pyPerfmon) app running on each Exchange server to be monitored.

![Exchange Monitoring Gadget](readme-images/exch.png)

###Tintri Monitoring

This module monitors a Tintri hybrid storage device using REST API calls.

![Tintri Monitoring Gadget](readme-images/tintri.png)

This code monitors a Tintri hybrid storage device using REST API calls.

###Rubrik Monitoring

This module monitors a Rubrik backup system using REST API calls.

![Rubrik Monitoring Gadget](readme-images/rubrik.png)

This code monitors a Rubrik backup system using REST API calls.
###Nutanix Monitoring

This module monitors a Nutanix hyperconverged system using REST API calls. The first image shows cluster monitoring.

![Nutanix Monitoring Gadget](readme-images/nutanix.png)

The second Nutanix gadget shows per-VM storage monitoring. It shows the top X VMs in the cluster sorted by IOPS based over a 30 minute window.

![Nutanix VM Monitoring Gadget](readme-images/storage.png)


###Weather

This is a simple javascript-only weather gadget that pulls data from [OpenWeatherMap.org](http://openweathermap.org). If you use this gadget, please sign up for a free APPID on that site and edit the source file to include your code.

![Weather Gadget](readme-images/weather.png)

This is a simple javascript-only weather gadget that pulls data from [OpenWeatherMap.org](http://openweathermap.org). If you use this gadget, please sign up for a free APPID on that site and edit the source file to include your code.


###Clock

This is a great javascript-only clock from [YXKFW](http://www.yxkfw.com/?p=15718).

![Clock Gadget](readme-images/clock.png)

This is a great javascript-only clock from [YXKFW](http://www.yxkfw.com/?p=15718).
###Twitter

![Twitter Gadget](readme-images/twitter.png)


## Code Layout
Individual python files are designed to be run independently for testing. You can run any of the python files directly and it will output data in JSON format. (Personally, I recommend loading it up in the [PyCharm](http://www.jetbrains.com/pycharm/) debugger).
This project is basically a copy of what I run in production. You will need to edit the data module files to provide details related to your environment, for example server ip addresses and SNMP OIDs. You will want to download a copy of the free [PyCharm Community Edition](https://www.jetbrains.com/pycharm/) so you can step through modules as necessary to get them working in your environment.

You will need to edit the files to provide your server ip addresses or SNMP OIDs. You should edit the credentials.py file to store usernames and passwords. Although the python files are hidden behind the web server, the credentials are being stored in plain text, so be sure that you are using restricted accounts. For example, a read-only VMware vSphere account is all we need.
Each data module in the system is a single python file designed to be run independently for testing. You can run any of the python files directly and it will output data in JSON format. When launched via the main webserver module, these data modules provide JSON data to HTML/javascript front-end pages.

The static HTML pages are loaded by the Statusboard iPad App which then uses AJAX to retrieve the JSON data.
You should edit the credentials.py file to store usernames and passwords. Although the python files are hidden behind the web server, the credentials are being stored in plain text, so be sure that you are using locked down accounts with read-only privileges. For example, a read-only VMware vSphere account is all we need.

The main function here is the webserver.py. This launches the CherryPy webserver and loads each data generator into a separate thread. To enable/disable a module, find the MODULES section and call create a SysAdminBoardModule object, specifying the module filename (without the .py). For example: SysAdminBoardModule('vmware_host') will load the vmware_host.py file, setup the webserver URLs and the process callback thread.
The main module is webserver.py. This launches the CherryPy webserver and loads each data module into a separate thread. To enable/disable a module, find the **MODULES** section and use the SysAdminBoardModule function, specifying the module filename (without the .py).

If you browse to the webserver, it will now display a list of loaded modules with links to display the output appropriately (HTML and AJAX). Note that the webserver loads on port 8080 by default unless you make the iptables changes below to redirect from port 80.
For example this function:
```
SysAdminBoardModule('vmware_host')
```
will load the vmware_host.py data module, setup the webserver to serve the front-end HTML page (/vmware_host) and serve the json data (/vmware_host/ajax). By default, the webserver only has the sample module enabled.

If you browse to the webserver, it will display a list of loaded modules with links to display the output appropriately (HTML and AJAX). Note that the webserver loads on port 8080 by default unless you make the iptables changes below to redirect from port 80.

## Simple Linux Configuration
Here are some directions for a base CentOS Linux server install.
Here are some directions for a base CentOS 6 Linux server install. (Note that these directions were updated and simplified on 12/21/2016 to account for the upgrade to Python3).

Install Python3 (note that gcc and python34-devel are required for pysnmp)

Install setuptools and pip
```
curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py | python -
yum install epel-release
yum install python34 python-pip python34-devel gcc
```

Create a virtual environment in /opt/sysadminboard-ve to store the required python modules (so they don't intermingle with the normal OS python modules.)

```
pip install -U pip
pip install virtualenv
cd /opt
virtualenv -p python3 sysadminboard-ve
```

Install required Python modules for the statusboard code.
Now that the virtual environment is available, use this command anytime you want to test or run pip

```
pip install CherryPy
pip install -U pysphere
pip install mechanize
pip install mysql-connector-python
pip install importlib
pip install routes
source /opt/sysadminboard-ve/bin/activate
```

Install pysnmp
Confirm which python you're using

```
yum groupinstall "Development Tools"
yum install python-devel
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
python ez_setup.py
easy_install pysnmp
which python
```

Setup Service

There is a simple init.d script in the source init.d directory. Copy the sysadminboard file to /etc/init.d/sysadminboard on server. Copy all of my files into /opt/sysadminboard, create a user and group called sbpython, change ownership of all files to sbpython. (The Webserver process will run as sbpython).
Install modules using pip (make sure you're in the virtualenv first using the source command above.)

```
pip install CherryPy
pip install routes
pip install pyvmomi
pip install mysql-connector
pip install requests
pip install pysnmp
```

Copy all files to /opt/sysadminboard. Note that the default log_settings.json file is set for debugging on Mac OS X. Replace it with the CentOS version in log_settings_samples.

Create sbpython account and assign owner rights to the static folder. The webserver will run as this user account, so it needs rights to this folder. All other files will be owned by root. Be sure to mark the credentials file so other accounts cannot access it.

```
adduser sbpython
chown -R sbpython:sbpython /opt/sysadminboard/static
chmod 700 /opt/sysadminboard/credentials.py
```

There are sample syslog and logrotate files that you can copy to your system to facilitate logging.

```
cp /opt/sysadminboard/centos_install/rsyslog.d/sysadminboard.conf /etc/rsyslog.d
service rsyslog restart
cp /opt/sysadminboard/centos_install/logrotate.d/sysadminboard /etc/logrotate.d
```

Setup Service
There is a simple init.d script in the source init.d directory. Copy the sysadminboard file to /etc/init.d/ on server

```
cp /opt/sysadminboard/centos_install/init.d/sysadminboard /etc/init.d
chmod +x /etc/init.d/sysadminboard
chkconfig sysadminboard on
chown -R sbpython:sbpython /opt/sysadminboard
```
chkconfig sysadminboard on
```

You can run the following commands now to stop or start the service.

With the init file in place, you can run the following commands (and it will load on startup):
```
service sysadminboard start
service sysadminboard stop
service sysadminboard status
service sysadminboard restart
```
service sysadminboard status
```

Note that you use the deactivate command to exit a virtual environment. The source...activate and deactivate commands are not needed when starting the service, only when testing.

```
deactivate
```


Add these rules to your firewall to redirect from port 8080 to port 80:

```
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
# Redirect port 80 to port 8080
 iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
```

## HTML-only version

Check out the dashboard.html file as an example of running the site without an iPad.





##Major Change Log
2016-12-21
* Changed python support to Python3. Python2.x no longer supported.
* Changed vmware access method from pysphere to pyvmomi (official VMware sdk)
* Added logging support and improved error handling
* Added Nutanix gadgets
* Resolved issues with Rubrik API behavior
* Updated readme.md to include new build directions (including virtualenv for Python3).
* Now including copies of all the modules and dashboards that I'm running in production.
* Retired the VNX modules.

2016-09-21
* New Rubrik gadget

Expand Down
12 changes: 7 additions & 5 deletions init.d/sysadminboard → centos_install/init.d/sysadminboard
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
#!/bin/bash
# chkconfig: 2345 90 90
# description: sysadminboard cherrypy web server
# description: sysadminboard monitoring services
### BEGIN INIT INFO
# Provides: sysadminboard CherryPy WebServer
# Provides: SysAdminBoard Monitoring Services
# Required-Start: network
# Required-Stop: network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: Start the program
### END INIT INFO

# Place this file in /etc/init.d, chmod +x, and chkconfig sysadminboard on

### Fill in these bits:
START_CMD="su -s /bin/bash -c 'python -O /opt/sysadminboard/webserver.py' sbpython"
NAME="SysAdminBoardPython"
START_CMD="su -s /bin/bash -c 'cd /opt/sysadminboard && source /opt/sysadminboard-ve/bin/activate && python -O /opt/sysadminboard/webserver.py' sbpython"
NAME="sysadminboard"
PGREP_STRING="/opt/sysadminboard/webserver.py"
PID_FILE="/var/run/sysadminboard/sysadminboard.pid"
PID_FILE="/var/run/sysadminboard.pid"
USER="sbpython"

### No further muckin' about needed!
Expand Down
10 changes: 10 additions & 0 deletions centos_install/logrotate.d/sysadminboard
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# For CentOS systems, place this file in /etc/logrotate.d

/var/log/sysadminboard {
missingok
compress
delaycompress
daily
rotate 5
copytruncate
}
3 changes: 3 additions & 0 deletions centos_install/rsyslog.d/sysadminboard.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# For CentOS systems, place this file in /etc/rsyslog.d
if $programname == 'sysadminboard' then /var/log/sysadminboard
& ~
24 changes: 12 additions & 12 deletions credentials.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,29 +8,29 @@
__author__ = 'scott@flakshack.com (Scott Vintinner)'


# EMC VNX Reporter
VNX_REPORTER_USERNAME = "someuser"
VNX_REPORTER_PASSWORD = "somepassword"


# VMware
VMWARE_VCENTER_USERNAME = "somedomain\\someuser"
VMWARE_VCENTER_PASSWORD = "somepassword"
VMWARE_VCENTER_USERNAME = "domain\\username"
VMWARE_VCENTER_PASSWORD = "yourpassword"


# SNMP Community String (Read-Only)
SNMP_COMMUNITY = "public"


# Tintri
TINTRI_USER = "someuser"
TINTRI_PASSWORD = "somepassword"
TINTRI_USER = "youraccount"
TINTRI_PASSWORD = "yourpassword"


# Workdesk MySQL
WORKDESK_USER = 'someuser'
WORKDESK_PASSWORD = 'somepassword'
WORKDESK_USER = 'youraccount'
WORKDESK_PASSWORD = 'yourpassword'

# Rubrik
RUBRIK_USER = 'someuser'
RUBRIK_PASSWORD = 'somepassword'
RUBRIK_USER = 'youraccount'
RUBRIK_PASSWORD = 'yourpassword'

# Nutanix
NUTANIX_USER = 'youraccount'
NUTANIX_PASSWORD = 'yourpassword'
Loading

0 comments on commit a519310

Please sign in to comment.