Merge branch 'develop'

netenglabs · Jan 10, 2024 · 8a69536 · 8a69536
2 parents 49cfd2c + 03dc164
commit 8a69536
Show file tree

Hide file tree

Showing 58 changed files with 4,409 additions and 3,410 deletions.
diff --git a/build/requirements.txt b/build/requirements.txt
diff --git a/docs/config_file.md b/docs/config_file.md
@@ -43,8 +43,8 @@ coalescer:
 | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |:-------------------------------- | ------------------- |
 | data-directory               | The directory where the poller stores the data collected from the network                                                                                                                                                                  | -                                | yes                 |
 | temp-directory               | Where SuzieQ stores temporary data                                                                                                                                                                                                         | /tmp/.suzieq/                    | no                  |
-| rest.rest_certfile           | certificate for the REST server                                                                                                                                                                                                            | -                                | no                  |
-| rest.rest_keyfile            | keyfile for the REST server                                                                                                                                                                                                                | -                                | no                  |
+| rest.rest-certfile           | certificate for the REST server                                                                                                                                                                                                            | -                                | no                  |
+| rest.rest-keyfile            | keyfile for the REST server                                                                                                                                                                                                                | -                                | no                  |
 | rest.API_KEY                 | API key for the REST server                                                                                                                                                                                                                | -                                | yes (if using rest) |
 | rest.address                 | IP address of the REST server.                                                                                                                                                                                                             | 127.0.0.1                        | no                  |
 | rest.port                    | port of the REST server                                                                                                                                                                                                                    | 80                               | no                  |

diff --git a/docs/gui.md b/docs/gui.md
@@ -15,3 +15,121 @@ Some additional arguments are allowed to set up the GUI:
 
 - `-p --port`: defines in which port the GUI is going to run. By default is `8501`
 - `-c --config`: SuzieQ configuration file to use. Check the [Configuration](config_file.md) page to check the default configuration file location.
+
+## Start a GUI behind an nginx reverse proxy
+
+This section is an step by step guide on how to set up an [nginx](http://nginx.org/en/docs/) reverse proxy for the SuzieQ GUI.
+
+### Requirements
+
+- nginx: [nginx installation guide](http://nginx.org/en/docs/install.html)
+- SuzieQ GUI running
+
+### Configure and start nginx
+
+Move into the nginx config directory
+
+``` shell
+cd /etc/nginx
+```
+
+!!! info
+        Some of the commands below may need `sudo`
+
+Fill the values into the following template and put it in a file under `/etc/nginx/sites-available/suzieq.conf`
+
+```
+server {
+        listen <PROXY_SERVER_PORT> default_server;
+        listen [::]:<PROXY_SERVER_PORT> default_server;
+        server_name <SERVER_NAME>;
+        location /<PROXY_PATH> {
+                proxy_pass <SUZIEQ_GUI_URL>;
+        }
+        # streamlit redirects config
+        location /<PROXY_PATH>/streamlit-components-demo {
+                proxy_pass <SUZIEQ_GUI_URL>/;
+        }
+        location ^~ /<PROXY_PATH>/static {
+                proxy_pass <SUZIEQ_GUI_URL>/static/;
+        }
+        location ^~ /<PROXY_PATH>/healthz {
+                proxy_pass <SUZIEQ_GUI_URL>/healthz;
+        }
+        location ^~ /<PROXY_PATH>/vendor {
+                proxy_pass <SUZIEQ_GUI_URL>/vendor;
+        }
+        location ^~ /<PROXY_PATH>/component {
+                proxy_pass <SUZIEQ_GUI_URL>/component;
+        }
+
+        location /<PROXY_PATH>/stream {
+                proxy_pass <SUZIEQ_GUI_URL>/stream;
+                proxy_http_version 1.1;
+                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header Host $host;
+                proxy_set_header Upgrade $http_upgrade;
+                proxy_set_header Connection "upgrade";
+                proxy_read_timeout 86400;
+        }
+}
+```
+
+The example below shows a configuration file shows how to create a reverse proxy
+to serve a SuzieQ GUI running on `http:localhost:8501` to be mapped to `http:localhost:80/suzieq`
+
+```
+server {
+        listen 80 default_server;
+        listen [::]:80 default_server;
+        server_name localhost;
+        location /suzieq {
+                proxy_pass http://127.0.0.1:8501;
+        }
+        # streamlit redirects config
+        location /suzieq/streamlit-components-demo {
+                proxy_pass http://127.0.0.1:8501/;
+        }
+        location ^~ /suzieq/static {
+                proxy_pass http://127.0.0.1:8501/static/;
+        }
+        location ^~ /suzieq/healthz {
+                proxy_pass http://127.0.0.1:8501/healthz;
+        }
+        location ^~ /suzieq/vendor {
+                proxy_pass http://127.0.0.1:8501/vendor;
+        }
+        location ^~ /suzieq/component {
+                proxy_pass http://127.0.0.1:8501/component;
+        }
+
+        location /suzieq/stream {
+                proxy_pass http://127.0.0.1:8501/stream;
+                proxy_http_version 1.1;
+                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header Host $host;
+                proxy_set_header Upgrade $http_upgrade;
+                proxy_set_header Connection "upgrade";
+                proxy_read_timeout 86400;
+        }
+}
+```
+
+!!! warning
+        These examples, do **NOT** provide any security.
+        Please check the [nginx documentation](http://nginx.org/en/docs/) to know how add security to your nginx server
+
+
+Now copy the configuration from `/etc/nginx/sites-available` to `/etc/nginx/sites-enabled`.
+It is recommended to use a symbolic link.
+
+``` shell
+ln -s /etc/nginx/sites-available/suzieq.conf /etc/nginx/sites-enabled/suzieq.conf
+```
+
+Now restart the nginx service
+``` shell
+sudo systemctl restart nginx
+```
+
+You can open you browser and connect to your configured nginx to see the SuzieQ GUI
diff --git a/docs/poller.md b/docs/poller.md
@@ -20,9 +20,13 @@ The inventory file format is covered in the [inventory page](./inventory.md).
 
 To monitor the status of the poller, you can look at the log files created (by default) in the `/tmp` directory. All the aspects related to the creation/update of the inventory are logged into `sq-poller-controller.log`, while the each worker logs into `sq-poller-<x>.log` where `x` is the worker id.
 
+!!! warning
+    If you want to start the poller process as background task, remember to redirect the stdout to `/dev/null` otherwise the poller might crash when it tries to write something on the terminal.<br>
+    `sq-poller -I inventory.yml >/dev/null &`
+
 ## Polling Modes
 
-SuzieQ poller can run in either **continuous mode** or **snapshot mode**. In continuous mode, the poller runs forever, gathering data at the specified period from the supplied inventory file. Alternately, it can run in snapshot mode where it gathers the data just once and stops once it has gathered data from all the nodes in the inventory file. If we cannot gather data from a node, we do not persist in gathering data in the snapshot mode. 
+SuzieQ poller can run in either **continuous mode** or **snapshot mode**. In continuous mode, the poller runs forever, gathering data at the specified period from the supplied inventory file. Alternately, it can run in snapshot mode where it gathers the data just once and stops once it has gathered data from all the nodes in the inventory file. If we cannot gather data from a node, we do not persist in gathering data in the snapshot mode.
 
 The default mode is the continuous mode. To use the snapshot mode, add the option `--run-once=update`.
 
@@ -92,7 +96,7 @@ have production versions of that code.
 
 ## Polling Period
 
-When polling in continuous mode, SuzieQ uses the default period specified in the suzieq-cfg.yml [configuration file](./config_file.md) (you can change the default location via the `-c` option when launching the poller). 
+When polling in continuous mode, SuzieQ uses the default period specified in the suzieq-cfg.yml [configuration file](./config_file.md) (you can change the default location via the `-c` option when launching the poller).
 
 Independent of this, you can change the polling period of any individual service by modifying (or adding) the `period:<time in secs>` key-value pair to the service configuration file (located under lib/python\<python version\>/site-packages/suzieq/config) wherever SuzieQ is installed.
 
@@ -128,7 +132,7 @@ jpietsch> sqpoller show status=fail namespace=dual-evpn
      354  dual-evpn  server104  ospfNbr       1         []        []       []      []        []                    0 2020-06-17 05:14:40.980
 ```
 
-In this case the errors are because we aren't running any of those services (mlag, evpn etc.) on those nodes (server101, edge01 etc.).  
+In this case the errors are because we aren't running any of those services (mlag, evpn etc.) on those nodes (server101, edge01 etc.).
 
 ## Database and Data Persistence
 

diff --git a/docs/rate-limiting-AAA.md b/docs/rate-limiting-AAA.md
@@ -20,8 +20,8 @@ rest:
 # The certificates listed below are provided purely to get started, In any
 # secure deployment, these must be generated specifically for the site and
 # these lines uncommented and containing the location of the site-specific file.
-# rest_certfile: /suzieq/cert.pem
-# rest_keyfile: /suzieq/key.pem
+# rest-certfile: /suzieq/cert.pem
+# rest-keyfile: /suzieq/key.pem
 #
   API_KEY: 496157e6e869ef7f3d6ecb24a6f6d847b224ee4f
   logging-level: WARNING

diff --git a/docs/release-notes.md b/docs/release-notes.md
@@ -1,5 +1,19 @@
 # Release Notes
 
+## 0.22.0 (Jan 10, 2023)
+
+This is largely a bugfix release, with some improvements in the output of some commands, and dependent library updates.
+
+* Path is improved to handle cases where the source or destination can belong to a SVI without the specific IP address being in the ARPND table
+* Other minor impovements in path logic
+* Updating automatic device detect to handle newer SONiC versions
+* Added support for count of errDisabled ports in ```interface summarize```
+* Added support for displaying serial number instead of architecture in ```device show```
+* Fixed bug in crash when port is specified in device config section of the poller
+* Fixing bug in version extraction of SONiC versions
+* Bunch of GUI fixes
+* Documentation updates
+
 ## 0.21.0 (June 6, 2023)
 
 This release fixes a number of important bugs across various platforms. The main new feature is the display of a status string for sqPoller show to help identify problems more easily.

diff --git a/docs/rest-server.md b/docs/rest-server.md
@@ -52,12 +52,12 @@ openssl rand -hex 20
 
 ## Configure SSL Key and Certificate
 
-It is possible to secure connections to the REST server via SSL. You can specify the full path of your SSL key and certificate via the ```rest_keyfile``` and ```rest_certfile``` variables under the `rest` section of the suzieq config file, typically at `~/.suzieq/suzieq-cfg.yml`. Here is an example of a config file with these two parameters defined:
+It is possible to secure connections to the REST server via SSL. You can specify the full path of your SSL key and certificate via the ```rest-keyfile``` and ```rest-certfile``` variables under the `rest` section of the suzieq config file, typically at `~/.suzieq/suzieq-cfg.yml`. Here is an example of a config file with these two parameters defined:
 
 ```yaml
 rest:
-    rest_certfile: /home/suzieq/cert.pem
-    rest_keyfile: /home/suzieq/cert.pem
+    rest-certfile: /home/suzieq/cert.pem
+    rest-keyfile: /home/suzieq/cert.pem
 ```
 
 For more details about the configuration file, check the [configuration](config_file.md) file documentation.