Grafana warning msg="Could not create user agent" error="invalid user agent format"

[bahree]

Is there an existing issue for this?

• I have searched the existing issues

What happened?

I just upgraded to the latest version (1.31.1) and not sure if this is related to that or if it has been for a while. Everything seems to be running OK but in docker I see a ton of these warnings.

grafana-1 | logger=plugin.context t=2024-11-03T15:17:05.894593108Z level=warn msg="Could not create user agent" error="invalid user agent format"

Expected Behavior

Given this is just a warning, I don't know if I should just ignore it or if there is a issue?

Steps To Reproduce

Goto any dashboard in Grafana

Relevant log output

grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.551745422Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.553000485Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.553021765Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.556027669Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.562634752Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:23.806344093Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=live t=2024-11-03T15:16:24.007699806Z level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/Y8upc6ZRk address=grafana/dashboard/uid/Y8upc6ZRk
grafana-1    | logger=plugin.context t=2024-11-03T15:16:24.183948854Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.400740522Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.400861311Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.405167549Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.405169016Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.453738319Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.587118493Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.595441147Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.602657338Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=live t=2024-11-03T15:16:29.800020584Z level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/zm7wN6Zgz address=grafana/dashboard/uid/zm7wN6Zgz
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.961103218Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:29.961105111Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:30.017793316Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:30.035193074Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:30.510691838Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:35.753099527Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:35.753195794Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:35.759887537Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:35.760594549Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:35.780539525Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:36.012338187Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:36.017633019Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:36.017913772Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:16:36.019312402Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=infra.usagestats t=2024-11-03T15:16:41.093220738Z level=info msg="Usage stats are ready to report"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:00.601414158Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:00.606765943Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:00.606865982Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:00.612947449Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:00.625463106Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=live t=2024-11-03T15:17:00.827218337Z level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/kOuP_Fggz address=grafana/dashboard/uid/kOuP_Fggz
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.00806124Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.033405329Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.039703493Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.055058555Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.140273217Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.245245022Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.245943806Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.249839969Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.266545043Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.33832449Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.452290235Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.464515532Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.489257306Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.49051619Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.547767296Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.656992947Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.68023559Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:01.698372648Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:03.350867118Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:03.357027509Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:03.357853671Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.657895616Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.658049472Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.658777809Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.664768062Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.701441782Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.859854816Z level=warn msg="Could not create user agent" error="invalid user agent format"
grafana-1    | logger=plugin.context t=2024-11-03T15:17:05.894593108Z level=warn msg="Could not create user agent" error="invalid user agent format"

Screenshots

No response

Additional data

No response

Type of installation

Docker

Version

v1.31.1

[JakobLichterfeld]

Thanks for reporting. Can you please share your docker-compose file (without your secrets)?

[bahree]

It really hasn't changed other than bumping up the postgres version. Shared below. I don't use MQTT, etc so have those removed.

services:
  teslamate:
    image: teslamate/teslamate:latest
    restart: always
    depends_on:
      - database
    environment:
      - ENCRYPTION_KEY=${API_TOKEN_ENCRYPTION_KEY}
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - DISABLE_MQTT=true
      - CHECK_ORIGIN=https://${FQDN_TM}
      - TZ=${TM_TZ}
    volumes:
      - ./import:/opt/app/import
    ports:
      - "4000:4000"

  database:
    image: postgres:17
    restart: always
    environment:
      - POSTGRES_USER=${TM_DB_USER}
      - POSTGRES_PASSWORD=${TM_DB_PASS}
      - POSTGRES_DB=${TM_DB_NAME}
    volumes:
      - teslamate-db:/var/lib/postgresql/data

  grafana:
    image: teslamate/grafana:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - GRAFANA_PASSWD=${GRAFANA_PW}
      - GF_SECURITY_ADMIN_USER=${GRAFANA_USER}
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PW}
      - GF_AUTH_ANONYMOUS_ENABLED=false
      - GF_SERVER_DOMAIN=${FQDN_TM}
      - GF_SERVER_ROOT_URL=https://${FQDN_TM}/grafana  # Correct root URL for Grafana
      - GF_SERVER_SERVE_FROM_SUB_PATH=true
      - DATABASE_SSL_MODE=disable
    volumes:
      - teslamate-grafana-data:/var/lib/grafana
    ports:
      - "4001:3000"  # Expose Grafana on port 4001, but Grafana listens internally on port 3000

volumes:
  teslamate-db:
  teslamate-grafana-data:

[JakobLichterfeld]

It really hasn't changed other than bumping up the postgres version.

And you followed the https://docs.teslamate.org/docs/maintenance/upgrading_postgres and not just bump to 17, right?

Even if it seems unrelated, for security reasons I suggest adding according to the docs:

cap_drop:
  - all

Regarding your issue: I can not reproduce it on my end. I suggest you do a full backup, delete your Grafana volume and spin it up again as I assume you edited something in the past.

[b89dan]

I get the same error:
logger=plugin.context t=2024-11-07T14:21:39.374989634Z level=warn msg="Could not create user agent" error="invalid user agent format"
Looking around i see a similar issue is solved in grafana 11.3.0: grafana/grafana#95006
Current Teslamate build uses grafana 11.2.3, so i guess we have to wait for the bump and see.

[JakobLichterfeld]

Looking around i see a similar issue is solved in grafana 11.3.0: grafana/grafana#95006
Current Teslamate build uses grafana 11.2.3, so i guess we have to wait for the bump and see.

Unfortunately Grafana 11.3 does still have the annoying fit to zoom not working bug, that's why we rolled back 3 months ago, see #4299 for Grafana 11.3.0 progress

@swiffer Can you reproduce the issue by OP?

[swiffer]

next version of teslamate upgrades grafana to 11.2.3 which should fix the issue.