feat: Surface plugin secret fetch errors in docker compose logs via shared log file

[j4anvii]

Type of change

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation
• CI

Mention the secrets provider

Description

This pull request introduces a new file-based logging mechanism for the plugin, allowing logs to be written to a host-mounted file with rotation support. It also updates configuration and deployment files to support this feature and improves error handling and logging throughout the codebase.

Plugin Logging Enhancements

• Added a new plugin_logger.go module that implements file-based logging with log rotation for the plugin, configurable via environment variables (PLUGIN_LOG_PATH and PLUGIN_LOG_MAX_SIZE_MB).
• Updated main.go to initialize and clean up the plugin log file writer, and to ensure errors are logged to both the file and stderr. Fatal errors now use os.Exit(1) for proper shutdown. [1] [2] [3]

Configuration and Deployment Updates

• Added new bind mount for /run/swarm-external-secrets in config.json to support log file sharing, and introduced new environment variables for log path and size configuration. [1] [2]
• Updated docker-compose.yml to add a secrets-logger sidecar service that tails the plugin log file for easier debugging.
• Modified scripts/deploy.sh to create the plugin log directory on the host with proper permissions.

Debugging and Documentation

• Updated docs/debugging.md to recommend tailing the plugin log file as the preferred debugging method, with fallback to daemon logs.

Error Handling Improvements

• Added structured error logging via a new logPluginError method in driver.go, improving visibility of plugin errors in the log file. [1] [2] [3] [4]

Commands & Configuration to test

Screen.Recording.2026-03-03.at.12.02.18.AM.mp4

commands to run to test it out (im giving my zshhistory)

5407 │ docker run --rm --privileged -v /run:/host-run alpine sh -c "mkdir -p /host-run/swarm-external-secrets && chmod 755 /host-run/swarm-external-secrets"\n\ndocker run -d --name vault-test -p 8200:8200 -e "VAULT_DEV_ROOT_TOKEN_ID=test-root-token" hashicorp/vault:latest server -dev\n
5408 │ docker exec vault-test env VAULT_ADDR="http://127.0.0.1:8200" VAULT_TOKEN="test-root-token" vault kv put secret/database/mysql password="my-super-secret-pass" username="admin"
5410 │ bash scripts/build.sh\n
5412 │ docker p
5414 │ docker plugin set swarm-external-secrets:latest SECRETS_PROVIDER="vault" VAULT_ADDR="http://127.0.0.1:8200" VAULT_AUTH_METHOD="token" VAULT_TOKEN="test-root-token" VAULT_MOUNT_PATH="secret" ENABLE_ROTATION="true" ROTATION_INTERVAL="10s" ENABLE_MONITORING="false"\ndocker plugin set swarm-external-secrets:latest gid=0 uid=0\ndocker plugin enable swarm-external-secrets:latest
5415 │ docker secret create --driver swarm-external-secrets:latest -l vault_path="database/mysql" -l vault_field="password" my_good_secret\ndocker service create --name app-good --secret my_good_secret busybox:latest sh -c "cat /run/secrets/my_good_secret && sleep 3600"
5416 │ docker secret create --driver swarm-external-secrets:latest -l vault_path="database/mysql" -l vault_field="nonexistent_field" my_bad_secret\ndocker service create --name app-bad --secret my_bad_secret busybox:latest sleep 3600
5417 │ docker service create --name secrets-logger --mount type=bind,source=/run/swarm-external-secrets,target=/run/swarm-external-secrets,readonly --mode global alpine:3.20 sh -c "while [ ! -f /run/swarm-external-secrets/plugin.log ]; do sleep 1; done; tail -F /run/swarm-external-secrets/plugin.log"
5418 │ clear
5419 │ docker service logs secrets-logger
5420 │ docker service logs secrets-logger 2>&1 | grep "level=error"\n
5421 │ history -20\n

Screenshots & Logs

n/a

Related Tickets & Documents

• Related Issue [RFE] Surface plugin secret fetch errors in docker compose logs via shared log file #54
• Closes [RFE] Surface plugin secret fetch errors in docker compose logs via shared log file #54

[j4anvii]

@sanjay7178 PTAL!

[sanjay7178]

See as par I mentioned in the github issue , it writes the logs defaults to /run/swarm-external-secrets/plugin.log , here are some changes I request

• use LOG_LEVEL , which is tends to be a integer ranges from 1-5 (means Trace, Debug, Info, Warning, Error, Fatal and Panic used logrus) , if this value is present only should log the file append by default log path /run/swarm-external-secrets/plugin.log
• make PLUGIN_LOG_PATH optional , if this set it overrides the log path user defined path .

[sanjay7178]

also fix the github CI issues , I seen the screen recording , it seems you're not using secrets provider to validate the functionality of the logging logic .

[j4anvii]

@sanjay7178

here is the new logs https://paste.opensuse.org/pastes/7b4097e0a478 (convert this in ansi view)

it uses vault secrets provider (see provider=vault)

I have ran the pipelines in j4anvii#4 and all are passing after the addition of commits bebeaea 41e5283

[Copilot]

Pull request overview

Adds a host-mounted, rotating file logger for the Docker Swarm secrets plugin so plugin errors (notably secret fetch failures) can be surfaced via a log-tail sidecar and viewed in compose/stack logs, addressing the debugging gap described in Issue #54.

Changes:

• Introduces plugin_logger.go with a rotating file writer configured via LOG_LEVEL, PLUGIN_LOG_PATH, and PLUGIN_LOG_MAX_SIZE_MB.
• Updates plugin startup/shutdown and request error paths to emit structured error logs to the shared log file.
• Updates deployment artifacts (plugin config.json mount, compose sidecar, scripts, and docs) to support the shared log directory and recommended debugging flow.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
scripts/tests/smoke-test-helper.sh	Creates the host log directory before enabling the plugin in smoke tests.
scripts/deploy.sh	Creates the host log directory as part of deployment.
scripts/demo-rotation.sh	Minor formatting/line-fix in demo script output.
providers/vault.go	Adds #nosec annotation related to TLS client key path field.
providers/openbao.go	Adds #nosec annotation related to TLS client key path field.
providers/aws.go	Adds #nosec annotation related to AWS access key identifier field.
plugin_logger.go	New rotating file logger and file logging setup logic.
main.go	Initializes plugin file logging and changes fatal paths to os.Exit.
driver.go	Adds logPluginError helper and uses it in secret fetch error paths.
docs/debugging.md	Recommends tailing the shared log file as the preferred debugging method.
docker-compose.yml	Adds a secrets-logger sidecar to tail the shared plugin log file.
config.json	Adds a bind mount for /run/swarm-external-secrets and new logging-related env vars.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sanjay7178]

@j4anvii can you please resolve the changes requested by copilot ?

[j4anvii]

@sanjay7178 done

[sanjay7178]

Can you please check the comment mentioned in config.json , needs a change to test it . Please test whether it works or not locally , don't make a commit , if it's works let me know .

[j4anvii]

@sanjay7178 yes, it works

❯ docker run --rm -v /run/swarm-external-secrets:/logs:ro alpine ls -lh /logs/

total 2M     
-rw-------    1 root     root     1003.2K Mar  4 08:51 plugin.log
-rw-------    1 root     root      999.2K Mar  4 08:50 plugin.log.1

i put this in my code

	padding := strings.Repeat("X", 999*1024)
	log.Infof("rotation-test-padding-1: %s", padding)
	log.Infof("rotation-test-padding-2: %s", padding)

[sanjay7178]

I mean custom log path ?

[j4anvii]

@sanjay7178 verified

❯ docker plugin set swarm-external-secrets:latest \
  SECRETS_PROVIDER="vault" \
  VAULT_ADDR="http://127.0.0.1:8200" \
  VAULT_AUTH_METHOD="token" \
  VAULT_TOKEN="test-root-token" \
  VAULT_MOUNT_PATH="secret" \
  ENABLE_ROTATION="true" \
  ROTATION_INTERVAL="10s" \
  ENABLE_MONITORING="false" \
  LOG_LEVEL="3" \
  PLUGIN_LOG_PATH="/run/swarm-external-secrets/custom.log"

docker plugin set swarm-external-secrets:latest gid=0 uid=0
docker plugin enable swarm-external-secrets:latest

swarm-external-secrets:latest
❯ docker secret create --driver swarm-external-secrets:latest \
  -l vault_path="database/mysql" -l vault_field="nonexistent_field" my_bad_secret
docker service create --name app-bad --secret my_bad_secret \
  busybox:latest sleep 3600

yb4nk7fx409aoq48xv430rwjm
at9xd8kos615qb0ivpssbv5ly
overall progress: 0 out of 1 tasks 
1/1: ready     
^COperation continuing in background.
Use `docker service ps at9xd8kos615qb0ivpssbv5ly` to check progress.
❯ docker run --rm -v /run/swarm-external-secrets:/logs:ro alpine ls -la /logs/

total 12
drwxr-x---    2 root     root            60 Mar  5 06:47 .
drwxr-xr-x    1 root     root          4096 Mar  5 06:47 ..
-rw-------    1 root     root          4471 Mar  5 06:47 custom.log

Its custom.log instead of plugin.log

[j4anvii]

please let me know if any other changes are needed @sanjay7178

[sanjay7178]

Please resolve the pending comments first , I would look this PR later , thanks .

[j4anvii]

done @sanjay7178

[sanjay7178]

/priority P1

[atharvamhaske]

hey @sanjay7178 is this PR open or should i review once and check if the issue is really solved or not?

[sanjay7178]

Contributor might not be active rn . you may raise PR , referring to these changes .

[sanjay7178]

closing as it is stale