Skip to content

YQ kqprun added scripts for starting prometheus and connector #14816

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

GrigoriyPA
Copy link
Collaborator

@GrigoriyPA GrigoriyPA commented Feb 19, 2025

Changelog entry

Added scripts for starting prometheus and connector in kqprun

Changelog category

  • Improvement

Description for reviewers

Copy link

github-actions bot commented Feb 19, 2025

🟢 2025-02-19 21:04:21 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Feb 19, 2025

2025-02-19 21:05:50 UTC Pre-commit check linux-x86_64-release-asan for d8d7fd7 has started.
2025-02-19 21:06:40 UTC Artifacts will be uploaded here
2025-02-19 21:08:57 UTC ya make is running...
🟢 2025-02-19 21:09:14 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-02-19 21:09:26 UTC Build successful.

Copy link

github-actions bot commented Feb 19, 2025

2025-02-19 21:12:30 UTC Pre-commit check linux-x86_64-relwithdebinfo for d8d7fd7 has started.
2025-02-19 21:14:28 UTC Artifacts will be uploaded here
2025-02-19 21:16:49 UTC ya make is running...
🟢 2025-02-19 21:16:58 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-02-19 21:17:03 UTC Build successful.

Copy link

github-actions bot commented Feb 20, 2025

2025-02-20 07:20:43 UTC Pre-commit check linux-x86_64-release-asan for 0ba5e02 has started.
2025-02-20 07:20:51 UTC Artifacts will be uploaded here
2025-02-20 07:23:11 UTC ya make is running...
🟢 2025-02-20 07:23:17 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-02-20 07:23:23 UTC Build successful.

Copy link

github-actions bot commented Feb 20, 2025

2025-02-20 07:23:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0ba5e02 has started.
2025-02-20 07:24:02 UTC Artifacts will be uploaded here
2025-02-20 07:26:25 UTC ya make is running...
🟢 2025-02-20 07:26:30 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-02-20 07:26:36 UTC Build successful.

vitalyisaev2
vitalyisaev2 previously approved these changes Feb 20, 2025
cp ${2:-"$SCRIPT_DIR/configuration/connector_config.yaml"} ./connector_config.yaml
sed -i "s/\${TARGET_PORT}/$1/g" ./connector_config.yaml

docker run --rm --name=fq-connector-go-$1 --network host -v ./connector_config.yaml:/opt/ydb/cfg/fq-connector-go.yaml ghcr.io/ydb-platform/fq-connector-go:latest
Copy link
Member

@vitalyisaev2 vitalyisaev2 Feb 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Не совсем понятно, зачем коннектору --network=host? Ведь вряд ли кто-то будет ходить в коннектор извне, кроме kqprun, который будет работать на той же машине, что и контейнер с коннектором. Да даже если надо ходить - всё равно стандартный паттерн это docker run -d -p <host_ip>:<host_port>:<container_port> <image_name>. Можешь попробовать развернуть какой-нибудь Nginx и сходить на него с другой машины. Если не заработает - это значит, что какие-то яндексовские велосипеды в области файрволла блокируют сетевую доступность, и надо просить Вадима их поправить. Поэтому ты можешь использовать просто стандартные настройки сети (т. е. вообще не указывать параметр --network), но вместо этого указать маппинг порта коннектора в тот, что тебе нужен, типа docker run -p внешний_порт:внутренний_порт .... Когда всё правильно настроено на хосте, --network=host никогда не нужен.
  2. Исходя из п. 1, тебе не нужен кастомный конфиг коннектора: не нужно его рендерить из шаблона и не нужно монтировать в контейнер. Внутри образа с коннектором уже есть дефолтный конфиг, и, как мне кажется, его здесь будет вполне достаточно.

cp ${3:-"$SCRIPT_DIR/configuration/prometheus_config.yaml"} ./prometheus_config.yaml
sed -i "s/\${TARGET_PORT}/$1/g" ./prometheus_config.yaml

docker run --rm --name=prometeus-$1 --network host -v ./prometheus_config.yaml:/etc/prometheus/prometheus.yml prom/prometheus --config.file=/etc/prometheus/prometheus.yml --web.listen-address=:$2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Всё вышесказанное относится и к контейнеру с Prometheus в том числе.

@GrigoriyPA GrigoriyPA force-pushed the YQ-kqprun-add-scripts-for-prometheus-and-connector branch from 3821b24 to 9d9707c Compare March 4, 2025 09:40
Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 09:43:33 UTC Pre-commit check linux-x86_64-release-asan for b5d8259 has started.
2025-03-04 09:43:48 UTC Artifacts will be uploaded here
2025-03-04 09:46:10 UTC ya make is running...
🟢 2025-03-04 09:46:16 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 09:46:22 UTC Build successful.

Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 09:43:39 UTC Pre-commit check linux-x86_64-relwithdebinfo for b5d8259 has started.
2025-03-04 09:43:44 UTC Artifacts will be uploaded here
2025-03-04 09:46:53 UTC ya make is running...
🟢 2025-03-04 09:47:00 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 09:47:06 UTC Build successful.

vitalyisaev2
vitalyisaev2 previously approved these changes Mar 4, 2025
Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 13:03:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for af4d116 has started.
2025-03-04 13:04:17 UTC Artifacts will be uploaded here
2025-03-04 13:06:43 UTC ya make is running...
🟢 2025-03-04 13:06:49 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 13:06:55 UTC Build successful.

Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 13:05:24 UTC Pre-commit check linux-x86_64-release-asan for af4d116 has started.
2025-03-04 13:05:38 UTC Artifacts will be uploaded here
2025-03-04 13:08:07 UTC ya make is running...
🟢 2025-03-04 13:08:13 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 13:08:20 UTC Build successful.

yumkam
yumkam previously approved these changes Mar 4, 2025
@GrigoriyPA GrigoriyPA dismissed stale reviews from yumkam and vitalyisaev2 via 8048809 March 4, 2025 13:35
@GrigoriyPA GrigoriyPA requested a review from yumkam March 4, 2025 13:35
@GrigoriyPA GrigoriyPA requested a review from vitalyisaev2 March 4, 2025 13:35
Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 13:36:28 UTC Pre-commit check linux-x86_64-release-asan for f4c5ab0 has started.
2025-03-04 13:36:58 UTC Artifacts will be uploaded here
2025-03-04 13:39:57 UTC ya make is running...
🟢 2025-03-04 13:40:03 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 13:40:10 UTC Build successful.

Copy link

github-actions bot commented Mar 4, 2025

2025-03-04 13:37:16 UTC Pre-commit check linux-x86_64-relwithdebinfo for f4c5ab0 has started.
2025-03-04 13:37:31 UTC Artifacts will be uploaded here
2025-03-04 13:39:45 UTC ya make is running...
🟢 2025-03-04 13:39:51 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-04 13:39:57 UTC Build successful.

@GrigoriyPA GrigoriyPA merged commit dd29c37 into ydb-platform:main Mar 4, 2025
12 checks passed
@GrigoriyPA GrigoriyPA deleted the YQ-kqprun-add-scripts-for-prometheus-and-connector branch March 4, 2025 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants