Apache Guacamole errors after upgrading Pangolin from 1.13.1 to 1.14.1, Newt from 1.7.0 to 1.8.1 and Traefik from 3.6.5 to 3.6.6.

[DaveCorsello]

After upgrading Pangolin from 1.13.1 to 1.14.1, Newt from 1.7.0 to 1.8.1 and Traefik from 3.6.5 to 3.6.6., logging into Apache Guacamole, a public resource in Pangolin, became unreliable. After entering MFA in Guacamole, its UI showed the generic message:

“An error has occurred and this action cannot be completed…”

Key observations:

Traefik access logs showed POST /guacamole/api/tokens alternating between:

• 502 (upstream unreachable / bad gateway),

• 500 with no ServiceName (request handled before routing; suggests router/service missing temporarily / dynamic config instability),

• and occasionally 403 from the backend (Guacamole returned JSON 403).

Pangolin-generated Traefik config routed Guacamole to a dynamic overlay endpoint:

• 1-Guacamole-service -> http://100.89.128.16:<ephemeral_port>

I have searched for other recent discussions in this forum on the topic of api tokens and have found none.

Gerbil's WireGuard interface:

• wg0 on 100.89.128.1/24,

Newt 1.8.1 logs on the remote side repeatedly showed:

• Ping failed ... recurring every minute (this happens across multiple versions of Newt and doesn't seem to me to be related to the Guacamole problems)

• Connection to server lost ... reconnecting,

• then Tunnel connection established ... Started tcp proxy to <guacamole's private IP>:8080

This reconnection loop correlated with the 502/500 bursts and errors/inaccessible pages in the Guacamole UI.

Rollback and roll forward results:

Rolling back Pangolin to 1.13.1, Newt to 1.7.0 and Traefik to 3.6.5 restored Guacamole stability.
Rolling forward Pangolin to 1.14.1 and Newt to 1.8.1 while keeping Traefik fixed at 3.6.5 resulted in the Guacamole issues described above.
Rolling back Pangolin and Newt to version 1.13.1 and 1.7.0 respectively again restored Guacamole stability.

Conclusion:

It appears that the errors in Guacamole were caused by upgrading to Pangolin 1.14.1 and Newt 1.8.1.

Originally posted by @DaveCorsello in #2234