-
Notifications
You must be signed in to change notification settings - Fork 4
Integrate rust logging #423
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
e999e09
f9f434e
ccc6c9e
2264a51
ee37dd8
e39cf34
db073e0
eb5efa6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Logging | ||
|
|
||
| This is a document describing internal consideration when it comes to logging. | ||
| To see the documentation about the logging feature as present in the driver, see [logging docs](../logging.md). | ||
|
|
||
| ## Default behavior | ||
|
|
||
| In the DSx driver, there was no option to configure the logging levels. Since the logging goes through | ||
| napi layer, I decided to introduce a client configuration option that limits the logs going through | ||
| napi layer - with the performance as the main goal here. While I didn't test the exact performance impact here | ||
| (which can be done in the future - I would recommend it, if you the developer of this driver plan to change this feature), | ||
| I expect it to be quite high, as the rust driver generates quite a lot of logs for a single DB query. | ||
|
|
||
| However, to avoid excessive log noise for most users, the default behavior is to pass only `warning` | ||
| and above (default: when no log level is provided by the user). Users who need finer-grained diagnostics | ||
| can set `logLevel` to `trace` or `debug` explicitly. This differs from the DSx driver where all logs | ||
| were always emitted. | ||
|
|
||
| ## Log attaching | ||
|
|
||
| In the DSx driver, the logs were generated on the client (so you could listen to logs generated by individual clients). | ||
| However, the rust driver does not allow for distinctions between the clients that generated the log (we only have a single source | ||
| for logs, which does not allow distinguishing what client generated it). | ||
|
|
||
| Since the use-cases where someone will have 2 or more clients seem to be very rare (this is only our assumption), | ||
| we decided that it's best to keep the API compatibility of how the logs are delivered. This means, that when someone | ||
| attaches log listener to the client, this will receive events from all active clients (at the specified log level). | ||
|
|
||
| This has the benefit of keeping the old interface of logging (so logging receiver set up for DSx driver should work here), | ||
| at the cost of duplicating the logs when multiple client sessions are active (every active client will receive single instance | ||
| of a log, no matter what client triggered this action). | ||
|
|
||
| Again, this is something that can be reworked during 1.0 release, while keeping the old interface as backward compatible (but probably deprecated). | ||
|
|
||
| ## Shutdown | ||
|
|
||
| We detach logging in two cases: | ||
|
|
||
| - The client is shut down — via an explicit call to `client.shutdown()` | ||
| - The client is destroyed — garbage collected via `FinalizationRegistry` | ||
|
|
||
| While the cleanup in case of object destruction is [heuristic](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/FinalizationRegistry#notes_on_cleanup_callbacks), | ||
| it's a backup case, in case the user forgets to manually close the client. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| # Logging | ||
|
|
||
| The driver uses [events](https://nodejs.org/api/events.html) to expose logging | ||
| information, keeping it decoupled from any specific logging framework. | ||
|
|
||
| The `Client` class inherits from | ||
| [EventEmitter](https://nodejs.org/api/events.html#events_class_eventemitter) | ||
| and emits `'log'` events: | ||
|
|
||
| ```js | ||
| client.on('log', (level, target, message, furtherInfo) => { | ||
| console.log(`${level} - ${target}: ${message}`); | ||
| }); | ||
| ``` | ||
|
|
||
| ## Enabling logging | ||
|
|
||
| Logging is **enabled by default**. When no `logLevel` is specified, all events | ||
| are captured (at `trace` level). | ||
|
|
||
| **WARNING** | ||
| This default behavior may be changed before 1.0 release. | ||
| We recommend explicitly setting desired logging level when using this driver. | ||
|
|
||
| To choose a specific minimum severity: | ||
|
|
||
| ```js | ||
| const { Client, types } = require('scylladb-driver-alpha'); | ||
|
|
||
| const client = new Client({ | ||
| contactPoints: ['127.0.0.1'], | ||
| logLevel: types.logLevels.info, | ||
| }); | ||
| ``` | ||
|
|
||
| To disable logging entirely, set `logLevel` to `'off'`: | ||
|
|
||
| ```js | ||
| const client = new Client({ | ||
| contactPoints: ['127.0.0.1'], | ||
| logLevel: types.logLevels.off, | ||
| }); | ||
| ``` | ||
|
|
||
| The callback is registered when `connect()` is called and unregistered on | ||
| `shutdown()`. No log events are emitted before the client connects. | ||
|
Comment on lines
+45
to
+46
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 💭 That's a bit unfortunate. We lose any logs that are emitted earlier. Can't we somehow enable logging earlier? Probably not, because they must be tied to the client?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are there any logs generated before? I don't think so, since the rust client is created only at the connect
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Entities other than
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider execution profiles, policies, etc. |
||
|
|
||
|
adespawn marked this conversation as resolved.
|
||
| ## Log levels | ||
|
|
||
| Log levels are exposed through the `types.logLevels` enum: | ||
|
|
||
| | Enum variant | Raw value | Description | | ||
| | ------------------- | ----------- | ---------------------------------------------------- | | ||
| | `logLevels.trace` | `'trace'` | Finest-grained diagnostic information (TRACE events) | | ||
| | `logLevels.debug` | `'debug'` | Fine-grained diagnostic information (DEBUG events) | | ||
| | `logLevels.info` | `'info'` | High-level informational messages | | ||
| | `logLevels.warning` | `'warning'` | Potentially harmful situations | | ||
| | `logLevels.error` | `'error'` | Error conditions | | ||
| | `logLevels.off` | `'off'` | Disables logging entirely | | ||
|
|
||
| The `logLevel` option acts as a **filter**: only events at or above the | ||
| configured severity are delivered to the listener. Filtering happens on the | ||
| native side, before crossing the FFI boundary, so suppressed events have | ||
| negligible overhead. | ||
|
|
||
| | `logLevel` value | Events delivered | | ||
| | ------------------- | ----------------------------------- | | ||
| | not set | WARN and above — **default** | | ||
| | `logLevels.off` | None | | ||
| | `logLevels.trace` | All (TRACE and above) | | ||
| | `logLevels.debug` | DEBUG and above | | ||
| | `logLevels.info` | INFO and above | | ||
| | `logLevels.warning` | WARN and above | | ||
| | `logLevels.error` | ERROR only | | ||
|
|
||
| The `trace` level is only suitable for debugging and is usually very | ||
| noisy. We recommend gathering events from `info` and above in production | ||
| environments. | ||
|
Comment on lines
+76
to
+78
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's why
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We initially decided to do WARN: #423 (comment) but after in person review, you suggested going back to trace, to keep the API compatibility: #423 (comment)
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, I see the latter comment. I wrote about using the old default, but what I mainly meant was that we shouldn't set it to OFF by default if the old default was not OFF. It's shocking that the old driver had logging ON and set to TRACE by default. This means getting flooded with trace messages, right? This must have slowed down the driver a lot. Is it even suitable for production? After you answer above questions, I think we should choose either INFO or WARN as the new default.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
It was the user who was filtering the logs, not the driver
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, so myriads of logs came to the user and they discarded it. Still a large overhead, right?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Understood. We definitely don't want such situation to occur by default. |
||
|
|
||
| ## Event arguments | ||
|
|
||
| Each `'log'` event delivers four arguments: | ||
|
|
||
| | Argument | Type | Description | | ||
| | ------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `level` | `string` | One of the level strings from the table above. | | ||
| | `target` | `string` | Identifies the source of the event. Either a class name (e.g. `"Client"`) or an internal module path (e.g. `"scylla::network::connection"`). | | ||
| | `message` | `string` | Human-readable description of the event. | | ||
| | `furtherInfo` | `string` | Additional structured context. Some events include key=value pairs from tracing spans (e.g. `peer_addr=10.0.0.1:9042`). May be an empty string. | | ||
|
adespawn marked this conversation as resolved.
|
||
|
|
||
| ### Event sources | ||
|
|
||
| Log events are emitted by both the Rust driver core and the JavaScript wrapper. | ||
| The `target` field identifies where an event originated — it contains either an | ||
| internal module path (e.g. `"scylla::network::connection"`) or a JS class name | ||
| (e.g. `"Client"`). | ||
|
|
||
| Both sources deliver events through the same `'log'` event, so a single | ||
| listener receives everything. | ||
|
|
||
| ## Multiple clients | ||
|
|
||
| Each `Client` registers its own logging callback independently. Multiple | ||
| clients can coexist, each with its own `logLevel`. | ||
|
|
||
| > **WARNING:** all clients share the same underlying Rust tracing subscriber. | ||
| > This means every client receives log events from the entire process — | ||
| > including events triggered by other `Client` instances. Keep this in mind | ||
| > when filtering or routing events. | ||
|
|
||
| ## Example | ||
|
|
||
| ```js | ||
| const { Client, types } = require('scylladb-driver-alpha'); | ||
|
|
||
| const client = new Client({ | ||
| contactPoints: ['10.0.1.101', '10.0.1.102'], | ||
| logLevel: types.logLevels.info, | ||
| }); | ||
|
|
||
| client.on('log', (level, target, message, furtherInfo) => { | ||
| const extra = furtherInfo ? ` (${furtherInfo})` : ''; | ||
| console.log(`[${level}] ${target}: ${message}${extra}`); | ||
| }); | ||
|
|
||
| await client.connect(); | ||
| // [info] Client: Connecting to cluster using 'ScyllaDB Node.js RS Driver' version ... | ||
| // [info] scylla::cluster::worker: Node added to cluster: ... | ||
| // ... | ||
|
|
||
| await client.shutdown(); | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -224,6 +224,18 @@ const { ExecutionProfile } = require("./execution-profile.js"); | |
| * to represent CQL varint data type. Defaults to true. | ||
| * | ||
| * Note, that using Integer as Varint (`useBigIntAsVarint == false`) is deprecated. | ||
| * @property {String} [logLevel] The minimum severity of log events emitted by the driver. | ||
| * | ||
| * **WARNING:** While you can configure different log levels for different clients, each client will receive | ||
| * log messages from all clients. | ||
|
wprzytula marked this conversation as resolved.
|
||
| * | ||
| * Valid values are defined in the {@link module:types~logLevels} enum (introduced in this driver). | ||
| * We recommend using the enum values (e.g. `types.logLevels.info`) rather than raw strings. | ||
| * | ||
| * When set to a value other than `'off'`, additional driver log messages (connection events, query routing, | ||
| * retries, etc.) will be emitted as `'log'` events on the {@link Client} instance. | ||
| * | ||
| * When not set, events at `warning` level and above are captured. Set to `'off'` to disable logging. | ||
| * @property {Array.<ExecutionProfile>} [profiles] The array of [execution profiles]{@link ExecutionProfile}. | ||
| * @property {Function} [promiseFactory] Function to be used to create a `Promise` from a | ||
| * callback-style function. | ||
|
|
@@ -470,6 +482,7 @@ function defaultOptions() { | |
| useBigIntAsLong: true, | ||
| useBigIntAsVarint: true, | ||
| }, | ||
| logLevel: undefined, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤔 This changes nothing, right? The key set to |
||
| }; | ||
| } | ||
|
|
||
|
|
@@ -545,6 +558,8 @@ function extend(baseOptions, userOptions) { | |
|
|
||
| validateEncodingOptions(options.encoding); | ||
|
|
||
| validateLogLevel(options.logLevel); | ||
|
|
||
| if (options.profiles && !Array.isArray(options.profiles)) { | ||
| throw new TypeError( | ||
| "profiles must be an Array of ExecutionProfile instances", | ||
|
|
@@ -713,6 +728,24 @@ function validateEncodingOptions(encodingOptions) { | |
| } | ||
| } | ||
|
|
||
| const validLogLevels = Object.values(types.logLevels); | ||
|
|
||
| /** | ||
| * Validates the logLevel option. | ||
| * @param {string} logLevel | ||
| * @private | ||
| */ | ||
| function validateLogLevel(logLevel) { | ||
| if ( | ||
| logLevel !== undefined && | ||
| (typeof logLevel !== "string" || !validLogLevels.includes(logLevel)) | ||
| ) { | ||
| throw new TypeError( | ||
| `logLevel must be one of ${validLogLevels.map((l) => `'${l}'`).join(", ")}`, | ||
| ); | ||
| } | ||
| } | ||
|
Comment on lines
+733
to
+747
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 💭 This seems buggy.
|
||
|
|
||
| function validateApplicationInfo(options) { | ||
| function validateString(key) { | ||
| const str = options[key]; | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.