Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore!: update prometheus-gc-stats dependency #5677

Merged
merged 2 commits into from
Jun 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/publish-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
with:
node-version: 18
registry-url: "https://registry.npmjs.org"
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/publish-rc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,10 @@ jobs:
- uses: actions/checkout@v3
with:
fetch-depth: 0 # Needs full depth for changelog generation
- uses: actions/setup-node@v2
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/publish-stable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-browser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: ${{matrix.node}}
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: ${{matrix.node}}
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-sim-merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-sim.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: 18
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
- uses: actions/setup-node@v3
with:
node-version: ${{matrix.node}}
check-latest: true
- name: Node.js version
id: node
run: echo "v8CppApiVersion=$(node --print "process.versions.modules")" >> $GITHUB_OUTPUT
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
[![Eth Consensus Spec v1.1.10](https://img.shields.io/badge/ETH%20consensus--spec-1.1.10-blue)](https://github.com/ethereum/consensus-specs/releases/tag/v1.1.10)
[![codecov](https://codecov.io/gh/ChainSafe/lodestar/branch/unstable/graph/badge.svg)](https://codecov.io/gh/ChainSafe/lodestar)
![ES Version](https://img.shields.io/badge/ES-2020-yellow)
![Node Version](https://img.shields.io/badge/node-18.x-green)
![Node Version](https://img.shields.io/badge/node-18.15.x-green)
[![gitpoap badge](https://public-api.gitpoap.io/v1/repo/ChainSafe/lodestar/badge)](https://www.gitpoap.io/gh/ChainSafe/lodestar)

[Lodestar](https://lodestar.chainsafe.io) is a TypeScript implementation of the [Ethereum Consensus specification](https://github.com/ethereum/consensus-specs) developed by [ChainSafe Systems](https://chainsafe.io).
Expand Down
3 changes: 1 addition & 2 deletions packages/beacon-node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@
"@chainsafe/libp2p-gossipsub": "^6.2.0",
"@chainsafe/libp2p-noise": "^11.0.4",
"@chainsafe/persistent-merkle-tree": "^0.5.0",
"@chainsafe/prometheus-gc-stats": "^1.0.0",
"@chainsafe/ssz": "^0.10.2",
"@chainsafe/threads": "^1.11.0",
"@ethersproject/abi": "^5.7.0",
Expand Down Expand Up @@ -145,7 +146,6 @@
"jwt-simple": "0.5.6",
"libp2p": "0.42.2",
"prom-client": "^14.2.0",
"prometheus-gc-stats": "^0.6.4",
"qs": "^6.11.1",
"snappyjs": "^0.7.0",
"strict-event-emitter-types": "^2.0.0",
Expand All @@ -158,7 +158,6 @@
"devDependencies": {
"@types/eventsource": "^1.1.11",
"@types/leveldown": "^4.0.3",
"@types/prometheus-gc-stats": "^0.6.2",
"@types/qs": "^6.9.7",
"@types/supertest": "^2.0.12",
"@types/tmp": "^0.2.3",
Expand Down
7 changes: 5 additions & 2 deletions packages/beacon-node/src/metrics/metrics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import {RegistryMetricCreator} from "./utils/registryMetricCreator.js";
import {createValidatorMonitor, ValidatorMonitor} from "./validatorMonitor.js";
import {collectNodeJSMetrics} from "./nodeJsMetrics.js";

export type Metrics = BeaconMetrics & LodestarMetrics & ValidatorMonitor & {register: RegistryMetricCreator};
export type Metrics = BeaconMetrics &
LodestarMetrics &
ValidatorMonitor & {register: RegistryMetricCreator; close: () => void};

export function createMetrics(
opts: MetricsOptions,
Expand All @@ -33,7 +35,7 @@ export function createMetrics(
lodestar.unhandledPromiseRejections.inc();
});

collectNodeJSMetrics(register);
const close = collectNodeJSMetrics(register);

// Merge external registries
for (const externalRegister of externalRegistries) {
Expand All @@ -47,5 +49,6 @@ export function createMetrics(
...lodestar,
...validatorMonitor,
register,
close,
};
}
8 changes: 5 additions & 3 deletions packages/beacon-node/src/metrics/nodeJsMetrics.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import {collectDefaultMetrics, Registry} from "prom-client";
import gcStats from "prometheus-gc-stats";
import {gcStats} from "@chainsafe/prometheus-gc-stats";

export function collectNodeJSMetrics(register: Registry, prefix?: string): void {
export function collectNodeJSMetrics(register: Registry, prefix?: string): () => void {
collectDefaultMetrics({
register,
prefix,
Expand All @@ -13,5 +13,7 @@ export function collectNodeJSMetrics(register: Registry, prefix?: string): void
// - nodejs_gc_runs_total: Counts the number of time GC is invoked
// - nodejs_gc_pause_seconds_total: Time spent in GC in seconds
// - nodejs_gc_reclaimed_bytes_total: The number of bytes GC has freed
gcStats(register, {prefix})();
// `close` must be called to stop the gc collection process from continuing
const close = gcStats(register, {collectionInterval: 6000, prefix});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is the interval every 6 seconds? Is it just randomly selected or is there some reason for that specific value?

return close;
}
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ const abortController = new AbortController();
// Set up metrics, nodejs and discv5-specific
const metricsRegister = workerData.metricsEnabled ? new RegistryMetricCreator() : null;
if (metricsRegister) {
collectNodeJSMetrics(metricsRegister, "network_worker_");
const closeMetrics = collectNodeJSMetrics(metricsRegister, "network_worker_");
abortController.signal.addEventListener("abort", closeMetrics, {once: true});
}

// Main event bus shared across the stack
Expand Down
4 changes: 3 additions & 1 deletion packages/beacon-node/src/network/discv5/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,10 @@ const logger = getNodeLogger(workerData.loggerOpts);
// Set up metrics, nodejs and discv5-specific
let metricsRegistry: RegistryMetricCreator | undefined;
let enrRelevanceMetric: Gauge<"status"> | undefined;
let closeMetrics: () => void | undefined;
if (workerData.metrics) {
metricsRegistry = new RegistryMetricCreator();
collectNodeJSMetrics(metricsRegistry, "discv5_worker_");
closeMetrics = collectNodeJSMetrics(metricsRegistry, "discv5_worker_");

// add enr relevance metric
enrRelevanceMetric = metricsRegistry.gauge<"status">({
Expand Down Expand Up @@ -95,6 +96,7 @@ const module: Discv5WorkerApi = {
return (await metricsRegistry?.metrics()) ?? "";
},
async close() {
closeMetrics?.();
discv5.removeListener("discovered", onDiscovered);
subject.complete();
await discv5.stop();
Expand Down
1 change: 1 addition & 0 deletions packages/beacon-node/src/node/nodejs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ export class BeaconNode {
initBeaconMetrics(metrics, anchorState);
// Since the db is instantiated before this, metrics must be injected manually afterwards
db.setMetrics(metrics.db);
signal.addEventListener("abort", metrics.close, {once: true});
}

const monitoring = opts.monitoring.endpoint
Expand Down
5 changes: 4 additions & 1 deletion packages/beacon-node/test/unit/metrics/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,8 @@ import {testLogger} from "../../utils/logger.js";
export function createMetricsTest(): Metrics {
const state = ssz.phase0.BeaconState.defaultViewDU();
const logger = testLogger();
return createMetrics({enabled: true, port: 0}, config, state, logger);
const metrics = createMetrics({enabled: true, port: 0}, config, state, logger);
// we don't need gc metrics running for tests
metrics.close();
return metrics;
}
26 changes: 6 additions & 20 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -627,6 +627,11 @@
resolved "https://registry.npmjs.org/@chainsafe/persistent-ts/-/persistent-ts-0.19.1.tgz"
integrity sha512-fUFFFFxdcpYkMAHnjm83EYL/R/smtVmEkJr3FGSI6dwPk4ue9rXjEHf7FTd3V8AbVOcTJGriN4cYf2V+HOYkjQ==

"@chainsafe/prometheus-gc-stats@^1.0.0":
version "1.0.0"
resolved "https://registry.yarnpkg.com/@chainsafe/prometheus-gc-stats/-/prometheus-gc-stats-1.0.0.tgz#9404abcf7e7a823596ecf3d71697f644568bde8c"
integrity sha512-l9aKCxQHBBBZIbCAl0y+7D4gded1cHUbIIRips68vN5zgpEuxjojUJSebuhLm+cgYdK1Wvf35gusA802bVfcdQ==

"@chainsafe/ssz@^0.10.2":
version "0.10.2"
resolved "https://registry.yarnpkg.com/@chainsafe/ssz/-/ssz-0.10.2.tgz#c782929e1bb25fec66ba72e75934b31fd087579e"
Expand Down Expand Up @@ -3774,11 +3779,6 @@
dependencies:
"@types/node" "*"

"@types/prometheus-gc-stats@^0.6.2":
version "0.6.2"
resolved "https://registry.yarnpkg.com/@types/prometheus-gc-stats/-/prometheus-gc-stats-0.6.2.tgz#b84246b13d0e7bd8bb61fa97a9ab624306b92697"
integrity sha512-HkT55AB8gPA7mrlkSVEoUgKQmnadWfioPZs0AakXRiggZzGKRCyLHe+WUpnvFashtWOJHUBQDjuR/siT9BKr/Q==

"@types/qs@^6.9.7":
version "6.9.7"
resolved "https://registry.yarnpkg.com/@types/qs/-/qs-6.9.7.tgz#63bb7d067db107cc1e457c303bc25d511febf6cb"
Expand Down Expand Up @@ -4910,7 +4910,7 @@ binary-extensions@^2.0.0:
resolved "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.2.0.tgz"
integrity sha512-jDctJ/IVQbZoJykoeHbhXpOlNBqGNcwXJKJog42E5HDPUwQTSdjCHdihjj0DlnheQ7blbT6dHOafNAiS8ooQKA==

bindings@^1.3.0, bindings@^1.3.1, bindings@^1.5.0:
bindings@^1.3.0, bindings@^1.5.0:
version "1.5.0"
resolved "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz"
integrity sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==
Expand Down Expand Up @@ -12049,11 +12049,6 @@ open@^8.4.0:
is-docker "^2.1.1"
is-wsl "^2.2.0"

optional@^0.1.3:
version "0.1.4"
resolved "https://registry.npmjs.org/optional/-/optional-0.1.4.tgz"
integrity sha512-gtvrrCfkE08wKcgXaVwQVgwEQ8vel2dc5DDBn9RLQZ3YtmtkBss6A2HY6BnJH4N/4Ku97Ri/SF8sNWE2225WJw==

optionator@^0.9.1:
version "0.9.1"
resolved "https://registry.npmjs.org/optionator/-/optionator-0.9.1.tgz"
Expand Down Expand Up @@ -12736,15 +12731,6 @@ prom-client@^14.2.0:
dependencies:
tdigest "^0.1.1"

prometheus-gc-stats@^0.6.4:
version "0.6.4"
resolved "https://registry.yarnpkg.com/prometheus-gc-stats/-/prometheus-gc-stats-0.6.4.tgz#7326216b92ef71591a535cc31b89ee3f94150fe9"
integrity sha512-HtxtDYRurj7gZS9AqjcfEAldf2e053mh+XW//OjifRxr6Y/aLx8y7ETwWesnJ9DaufvAMyqUUQJUzhB9jLc6vg==
dependencies:
optional "^0.1.3"
optionalDependencies:
gc-stats "^1.4.0"

promise-all-reject-late@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/promise-all-reject-late/-/promise-all-reject-late-1.0.1.tgz#f8ebf13483e5ca91ad809ccc2fcf25f26f8643c2"
Expand Down