Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemd service cannot start #301

Closed
panboo0106 opened this issue Oct 17, 2022 · 10 comments
Closed

systemd service cannot start #301

panboo0106 opened this issue Oct 17, 2022 · 10 comments
Assignees
Labels
UNCONFIRMED Initial state. Need's triaged

Comments

@panboo0106
Copy link

Describe the issue:
systemd service cannot start

Describe the expected behavior:

# journalctl -u postgres_exporter@pg12.service
/etc/systemd/system/postgres_exporter@.service:8: Ignoring unknown escape sequences: "set -o pipefail; cat $QUERY_FILE_LIST | sed -e 's/#PG_STAT_ST...

I think double '$$' not good for me:
image

i can work when i use one '$'

Tell us about your environment:

  • pgMonitor version: current version
  • Container or non-container: no
  • Container name / image: no
  • Operating System for non-container: Rocky Linux8.5
  • PostgreSQL Version: 12.11
  • Exporter(s) in use (incl. version): 0.11.1
  • Prometheus version: no
  • AlertManager version: no
  • Grafana version: no
@panboo0106 panboo0106 added the UNCONFIRMED Initial state. Need's triaged label Oct 17, 2022
@keithf4
Copy link
Collaborator

keithf4 commented Oct 17, 2022

What version of pgMonitor are you using? We'd seen this error in the past as well when trying to use startup options containing multiple .yml files and it was fixed with this PR in version 4.6

#269

@keithf4 keithf4 self-assigned this Oct 17, 2022
@panboo0106
Copy link
Author

What version of pgMonitor are you using? We'd seen this error in the past as well when trying to use startup options containing multiple .yml files and it was fixed with this PR in version 4.6

#269
v4.7

@keithf4
Copy link
Collaborator

keithf4 commented Oct 17, 2022

Can you please share the entire contents of your service startup file? Just to make sure if the changes were applied and so we can test with what you have?

@keithf4
Copy link
Collaborator

keithf4 commented Oct 17, 2022

Also noticed that your service file is named differently than the one pgMonitor provides. How are you installing it?

@panboo0106
Copy link
Author

PGBACKREST_INFO_THROTTLE_MINUTES=10
PG_STAT_STATEMENTS_LIMIT=20
PG_STAT_STATEMENTS_THROTTLE_MINUTES=-1
OPT="--web.listen-address=0.0.0.0:9187 --extend.query-path=/etc/xdrops/postgres_exporter/12/queries.yml --disable-default-metrics --disable-settings-metrics"
DATA_SOURCE_NAME="postgresql://ccp_monitoring:xxxxxxxxxxx@localhost:5432/postgres?sslmode=disable"
# For global / cluster metric
QUERY_FILE_LIST="/etc/xdrops/postgres_exporter/12/queries_global.yml /etc/xdrops/postgres_exporter/12/queries_general.yml /etc/xdrops/postgres_exporter/12/queries_backrest.yml /etc/xdrops/postgres_exporter/12/queries_pgbouncer.yml"
[Unit]
Description=Postgres Exporter Server - Service Instance: %I
After=network-online.target
[Service]
User=ccp_monitoring
WorkingDirectory=/etc/xdrops/postgres_exporter
EnvironmentFile=/etc/xdrops/postgres_exporter/12/systemd/%i
ExecStartPre=/bin/bash -c "set -o pipefail; cat $QUERY_FILE_LIST | sed -e 's/#PG_STAT_STATEMENTS_THROTTLE_MINUTES#/${PG_STAT_STATEMENTS_THROTTLE_MINUTES}/g' -e 's/#PGBACKREST_INFO_THROTTLE_MINUTES#/${PGBACKREST_INFO_THROTTLE_MINUTES}/g' -e 's/#PG_STAT_STATEMENTS_LIMIT#/${PG_STAT_STATEMENTS_LIMIT}/g' > $(echo $OPT | sed 's/.*--extend.query-path=\([^\ ]*\.yml\).*/\1/')"
ExecStart=/etc/xdrops/postgres_exporter/postgres_exporter $OPT
ExecReload=/usr/bin/kill -HUP $MAINPID
Restart=always
[Install]
WantedBy=multi-user.target

image

image
pgbouncer backrest metrics show on grafana well

I just only change filename 、directory path and discard a ‘$`.

@keithf4
Copy link
Collaborator

keithf4 commented Oct 17, 2022

Thank you for the info. We'll look into it.

@hunleyd
Copy link
Collaborator

hunleyd commented Oct 18, 2022

So I do see the $$ in the file in my testing, but the service still starts in my testing on Rocky 8. I edited the service file to make it a single $ and rebooted the node with no issues. I think we need to make the edit @keithf4

@keithf4
Copy link
Collaborator

keithf4 commented Oct 18, 2022

@leo-minorui

So I've included this change in the PR for the PG15 update. It seems like it may have been trying to escape the user of a dollar here according to systemd rules, but may not be needed in this case?
https://www.freedesktop.org/software/systemd/man/systemd.service.html

To pass a literal dollar sign, use "$$"

However, I'm still a bit curious how this is breaking for you if we cannot reproduce this on our Rocky test system, nor any other OS's that we test (RHEL8/7, CenOS7, Ubuntu20/22). Do you have anything else customized with the shell for the user that's running the exporter? Or done anything to the bash interpreter since this is explicitly calling bash?

@keithf4
Copy link
Collaborator

keithf4 commented Oct 18, 2022

What version of systemd are you running? This was from a CentOS7 system I have running now

systemctl --version
systemd 219

@panboo0106
Copy link
Author

@leo-minorui

So I've included this change in the PR for the PG15 update. It seems like it may have been trying to escape the user of a dollar here according to systemd rules, but may not be needed in this case? https://www.freedesktop.org/software/systemd/man/systemd.service.html

To pass a literal dollar sign, use "$$"

However, I'm still a bit curious how this is breaking for you if we cannot reproduce this on our Rocky test system, nor any other OS's that we test (RHEL8/7, CenOS7, Ubuntu20/22). Do you have anything else customized with the shell for the user that's running the exporter? Or done anything to the bash interpreter since this is explicitly calling bash?

Sorry to reply late.
The version is systemd 239 (239-58.el8)

For me, the logic of command( > $(echo $OPT | sed 's/.--extend.query-path=([^\ ].yml).*/\1/')") is to combine multiple file contents into a new file named 'queries.yaml'

and in my mind , double dollar signs mean process number in bash.
like this :

echo $$ 
854301

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
UNCONFIRMED Initial state. Need's triaged
Projects
None yet
Development

No branches or pull requests

3 participants