Alert

The Alert application reads data from TDEngine, calculating according to predefined rules to generate alerts, and pushes alerts to downstream applications like AlertManager.

Install

From Binary

Precompiled binaries is available at taosdata website, please download and unpack it by below shell command.

$ tar -xzf tdengine-alert-$version-$OS-$ARCH.tar.gz 

If you have no TDengine server or client installed, please execute below command to install the required driver library:

$ ./install_driver.sh

From Source Code

Two prerequisites are required to install from source.

1. TDEngine server or client must be installed.
2. Latest Go language must be installed.

When these two prerequisites are ready, please follow steps below to build the application:

$ mkdir taosdata
$ cd taosdata
$ git clone https://github.com/taosdata/tdengine.git
$ cd tdengine/alert/cmd/alert
$ go build

If go build fails because some of the dependency packages cannot be downloaded, please follow steps in goproxy.io to configure GOPROXY and try go build again.

Configure

The configuration file format of Alert application is standard json, below is its default content, please revise according to actual scenario.

{
  "port": 8100,
  "database": "file:alert.db",
  "tdengine": "root:taosdata@/tcp(127.0.0.1:0)/",
  "log": {
    "level": "production",
    "path": "alert.log"
  },
  "receivers": {
    "alertManager": "http://127.0.0.1:9093/api/v1/alerts",
    "console": true
  }
}

The use of each configuration item is:

• port: This is the http service port which enables other application to manage rules by restful API.
• database: rules are stored in a sqlite database, this is the path of the database file (if the file does not exist, the alert application creates it automatically).
• tdengine: connection string of TDEngine server, note in most cases the database information should be put in a rule, thus it should NOT be included here.
• log > level: log level, could be production or debug.
• log > path: log output file path.
• receivers > alertManager: the alert application pushes alerts to AlertManager at this URL.
• receivers > console: print out alerts to console (stdout) or not.

When the configruation file is ready, the alert application can be started with below command (alert.cfg is the path of the configuration file):

$ ./alert -cfg alert.cfg

Prepare an alert rule

From technical aspect, an alert could be defined as: query and filter recent data from TDEngine, and calculating out a boolean value from these data according to a formula, and trigger an alert if the boolean value last for a certain duration.

This is a rule example in json format:

{
  "name": "rule1",
  "sql": "select sum(col1) as sumCol1 from test.meters where ts > now - 1h group by areaid",
  "expr": "sumCol1 > 10",
  "for": "10m",
  "period": "1m",
  "labels": {
    "ruleName": "rule1"
  },
  "annotations": {
    "summary": "sum of rule {{$labels.ruleName}} of area {{$values.areaid}} is {{$values.sumCol1}}"
  }
}

The fields of the rule is explained below:

• name: the name of the rule, must be unique.
• sql: this is the sql statement used to query data from TDEngine, columns of the query result are used in later processing, so please give the column an alias if aggregation functions are used.
• expr: an expression whose result is a boolean value, arithmatic and logical calculations can be included in the expression, and builtin functions (see below) are also supported. Alerts are only triggered when the expression evaluates to true.
• for: this item is a duration which default value is zero second. when expr evaluates to true and last at least this duration, an alert is triggered.
• period: the interval for the alert application to check the rule, default is 1 minute.
• labels: a label list, labels are used to generate alert information. note if the sql statement includes a group by clause, the group by columns are inserted into this list automatically.
• annotations: the template of alert information which is in go template syntax, labels can be referenced by $labels.<label name> and columns of the query result can be referenced by $values.<column name>.

Operators

Operators which can be used in the expr field of a rule are list below, () can be to change precedence if default does not meet requirement.

Operator	Unary/Binary	Precedence	Effect
~	Unary	6	Bitwise Not
!	Unary	6	Logical Not
+	Unary	6	Positive Sign
-	Unary	6	Negative Sign
*	Binary	5	Multiplication
/	Binary	5	Division
%	Binary	5	Modulus
<<	Binary	5	Bitwise Left Shift
>>	Binary	5	Bitwise Right Shift
&	Binary	5	Bitwise And
+	Binary	4	Addition
-	Binary	4	Subtraction
|	Binary	4	Bitwise Or
^	Binary	4	Bitwise Xor
==	Binary	3	Equal
!=	Binary	3	Not Equal
<	Binary	3	Less Than
<=	Binary	3	Less Than or Equal
>	Binary	3	Great Than
>=	Binary	3	Great Than or Equal
&&	Binary	2	Logical And
||	Binary	1	Logical Or

Built-in Functions

Built-in function can be used in the expr field of a rule.

• min: returns the minimum one of its arguments, for example: min(1, 2, 3) returns 1.
• max: returns the maximum one of its arguments, for example: max(1, 2, 3) returns 3.
• sum: returns the sum of its arguments, for example: sum(1, 2, 3) returns 6.
• avg: returns the average of its arguments, for example: avg(1, 2, 3) returns 2.
• sqrt: returns the square root of its argument, for example: sqrt(9) returns 3.
• ceil: returns the minimum integer which greater or equal to its argument, for example: ceil(9.1) returns 10.
• floor: returns the maximum integer which lesser or equal to its argument, for example: floor(9.9) returns 9.
• round: round its argument to nearest integer, for examples: round(9.9) returns 10 and round(9.1) returns 9.
• log: returns the natural logarithm of its argument, for example: log(10) returns 2.302585.
• log10: returns base 10 logarithm of its argument, for example: log10(10) return 1.
• abs: returns the absolute value of its argument, for example: abs(-1) returns 1.
• if: if the first argument is true returns its second argument, and returns its third argument otherwise, for examples: if(true, 10, 100) returns 10 and if(false, 10, 100) returns 100.

Rule Management

• Add / Update

  • API address: http://<server>:<port>/api/update-rule
  • Method: POST
  • Body: the rule
  • Example：curl -d '@rule.json' http://localhost:8100/api/update-rule

• Delete

  • API address: http://<server>:<port>/api/delete-rule?name=<rule name>
  • Method：DELETE
  • Example：curl -X DELETE http://localhost:8100/api/delete-rule?name=rule1

• Enable / Disable

  • API address: http://<server>:<port>/api/enable-rule?name=<rule name>&enable=[true | false]
  • Method POST
  • Example：curl -X POST http://localhost:8100/api/enable-rule?name=rule1&enable=true

• Retrieve rule list

  • API address: http://<server>:<port>/api/list-rule
  • Method: GET
  • Example：curl http://localhost:8100/api/list-rule