APISENSE COLLECTOR

Note

It is recommended to check the RESTSense Specification as well as the RESTSense Agent for a better understanding of the RESTSense project. You may also find useful to check the Demo project to see a real example of RESTSense in action.

Description

the APISense collector is the component responsible for gathering data from the monitored system, which must itegrate the RESTSense Agent package. The following diagram shows a basic overview of the interaction between the collector and the agent.

flowchart LR
    subgraph Monitored System
        A[RESTSense\nAgent]
    end
    U((User)) --> C
    subgraph RESTSense Collector
        A --> B[gRPC Server]
        C[REST API]
    end
    B --> D[(NoSQL DB)]
    C --> D

Loading

As shown in the diagram above, the RESTSense collector provide two different interfaces: Grpc and REST. The Grpc interface is used by the RESTSense agent to send data to the collector, while the REST interface is used by the user to query the data stored in the collector's database.

gRPC - Integration with RESTSense Agent

The gRPC interface is provided by the collector to allow the RESTSense agent to send telemetry of the monitored system. Internally, Grpc connection use the Protocol Buffers (Protobuf) to serialize the data sent by the agent.

Protocol Buffers (Protobuf)

RESTSense makes use of OpenTelemetry as the standard for telemetry data. The collector relies on OpenTelemetry Protobuf definitions to establish the communication with the agent.

REST API

The REST API is provided by the collector to allow the user to query the data stored in the collector's database. REST is the standard for web APIs, using JSON for data serialization. The collector provides a set of REST endpoints which can be called through HTTP requests to retrieve data from RESTSense Collector MongoDB database.

The REST Service is implemented following an API-First approach where the API definition represents the interface provided to the external world. To achieve that, RESTSense collector relies on OAS Tools v3 framework, which leverages OpenAPI features to generate the REST API implementation.

Swagger

Since the REST API is generated from the OpenAPI definition, the collector provides a Swagger UI to allow the user to explore the API and test the endpoints. The Swagger UI is available at /docs. The following endpoints are available:

• Traces API:
  • /api/v1/traces - Retrieves a list of traces
  • /api/v1/traces/{traceId} - Retrieves a trace by its ID
• Metrics API:
  • /api/v1/metrics - Retrieves a list of metrics
  • /api/v1/metrics/{name} - Retrieves a metric by name
  • /api/v1/metrics/{name}/data - Retrieves a metric data
• Resources API:
  • /api/v1/resources - Retrieves a list of resources

Data Model

The RESTSense collector implements the RESTSense Specification v1.0 at the data layer. Hence, the data model of the collector is composed of three main entities: traces, metrics and resources.

The data layer is implemented using MongoDB as the NoSQL database, which data is retrieved through the native mongoDB driver wrapped in Mongoose schemas that must be compliant with the specification.

Deployment

The RESTSense Collector may be deployed locally or inside a Docker container. The following sections describe how to deploy the collector in both environments.

Configuration

The project is configured through environment variables. The following table describes the available variables:

Variable	Description	Default Value
MONGO_URL	MongoDB connection URL	mongodb://localhost:27017/restsense
GRPC_PORT	gRPC port	4317
HTTP_PORT	HTTP port	8080 (local) or 80 (docker)

Local (dev) environment

To deploy the collector locally, you must have Node.js installed in your machine and a MongoDB instance running (whether locally or a remote server). Make dure to configure the MONGO_URL environment variable to point to your MongoDB instance.

To start the collector, run the following command:

npm run start

Docker Compose

The project counts with a Dockerfile to build a Docker image of the collector and deploy it along a MongoDB instance using Docker Compose. You can use the following docker-compose.yml file to deploy the collector and the database:

Note

This docker-compose file will assume that the collector Dockerfile is in the current directory and will try to build the image from it.

name: restsense-backend
version: "3.8"
services:
    restsense-collector:
        build: .
        image: restsense/collector
        container_name: restsense-collector
        environment:
            MONGO_URL: mongodb://restsense-db:27017/restsense
        ports:
            - "8000:80"
            - "4317:4317"
        depends_on:
            - restsense-db

    restsense-db:
        image: mongo
        container_name: restsense-db

Copy the content above to a docker-compose.yml file and run the following command to start the containers:

docker-compose up --build -d

The REST API will be available at http://localhost:8000 and the gRPC server at localhost:4317.

License

This project is licensed under the GNU GPLv3 license. See the LICENSE file for more details.