No LORAWAN Frames: level=error msg="gateway: handle gateway stats error" error="update gateway state error: object does not exist"

[sergorl]

• The issue is present in the latest release.
• I have searched the issues of this repository and believe that this is not a duplicate.

What happened?

I use this gateway without any connected devices.
In application-server Status seen is a few seconds ago, Gateway activity is None, GPS coordinates is (0, 0), Live LORAWAN Frames is None.

What did you expect?

Right status in application-server: correct GPScoordinates and LORAWAN frames.

Steps to reproduce this issue

Steps:

1. Run docker-chirpstack by docker-compose up
2. Create gateway in application-server.

Could you share your log output?

time="2022-05-06T10:57:20.073539229Z" level=info msg="starting ChirpStack Network Server" band=EU868 docs="https://www.chirpstack.io/" net_id=000000 version=
time="2022-05-06T10:57:20.077905827Z" level=info msg="storage: setting up storage module"
time="2022-05-06T10:57:20.077929453Z" level=info msg="storage: setting up Redis client"
time="2022-05-06T10:57:20.07898924Z" level=info msg="storage: connecting to PostgreSQL"
time="2022-05-06T10:57:20.13645922Z" level=info msg="storage: applying PostgreSQL data migrations"
time="2022-05-06T10:57:20.162216879Z" level=info msg="gateway/mqtt: connecting to mqtt broker" server="tcp://mosquitto:1883"
time="2022-05-06T10:57:20.16459562Z" level=info msg="api: starting network-server api server" bind="0.0.0.0:8000" ca-cert= tls-cert= tls-key=
time="2022-05-06T10:57:20.164829214Z" level=info msg="starting downlink device-queue scheduler"
time="2022-05-06T10:57:20.164846376Z" level=info msg="starting multicast scheduler"
time="2022-05-06T10:57:20.16916317Z" level=info msg="backend/gateway: connected to mqtt server"
time="2022-05-06T10:57:20.171068114Z" level=info msg="gateway/mqtt: subscribing to gateway event topic" qos=0 topic=gateway/+/event/+
time="2022-05-06T10:57:46.910962741Z" level=info msg="finished unary call with code OK" ctx_id=a97c0c3c-eb8c-48e9-830f-189628d0072e grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:57:46Z" grpc.time_ms=5.032 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:57:55.634779455Z" level=info msg="finished unary call with code OK" ctx_id=c4392ad7-d18a-463b-b950-1b1f122b617e grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:57:55Z" grpc.time_ms=6.272 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:57:58.452940993Z" level=info msg="finished unary call with code OK" ctx_id=34e4bffb-1629-49cd-8b27-2a4780902134 grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:57:58Z" grpc.time_ms=1.383 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:57:58.472481087Z" level=info msg="finished unary call with code OK" ctx_id=8ab8846b-28c1-4e54-a0f9-4c6a0b6694fe grpc.code=OK grpc.method=GetServiceProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:57:58Z" grpc.time_ms=2.663 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:58:03.770056614Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=8a5d3a35-d7ce-4fd7-8daa-c3cdbdf8d527
time="2022-05-06T10:58:03.771701283Z" level=error msg="gateway: handle gateway stats error" ctx_id=8a5d3a35-d7ce-4fd7-8daa-c3cdbdf8d527 error="update gateway state error: object does not exist"
time="2022-05-06T10:58:05.177095977Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=StreamFrameLogsForGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:58:01Z" grpc.time_ms=3465.271 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:58:06.625644469Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=198dfc0c-5167-4d6f-bd6f-0367c4fa18d3
time="2022-05-06T10:58:06.629585586Z" level=info msg="storage: gateway state updated" ctx_id=198dfc0c-5167-4d6f-bd6f-0367c4fa18d3 gateway_id=e45f01fffe14ccfe
time="2022-05-06T10:58:06.632363666Z" level=warning msg="creating insecure application-server client" server="chirpstack-application-server:8001"
time="2022-05-06T10:58:06.642720914Z" level=info msg="finished client unary call" ctx_id=198dfc0c-5167-4d6f-bd6f-0367c4fa18d3 grpc.code=OK grpc.ctx_id=599ed36b-a238-4f68-8434-890ac75140c9 grpc.duration=8.401583ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T10:58:33.77209287Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=48cb2767-2b3c-4131-a32f-4fe47010a472
time="2022-05-06T10:58:33.773250731Z" level=error msg="gateway: handle gateway stats error" ctx_id=48cb2767-2b3c-4131-a32f-4fe47010a472 error="update gateway state error: object does not exist"
time="2022-05-06T10:58:36.632609925Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=f01a777e-75f7-4810-97bc-6fb7139a6d68
time="2022-05-06T10:58:36.634864186Z" level=info msg="storage: gateway state updated" ctx_id=f01a777e-75f7-4810-97bc-6fb7139a6d68 gateway_id=e45f01fffe14ccfe
time="2022-05-06T10:58:36.64197737Z" level=info msg="finished client unary call" ctx_id=f01a777e-75f7-4810-97bc-6fb7139a6d68 grpc.code=OK grpc.ctx_id=ad91f17d-4f80-43c6-9cf7-d806fcbed532 grpc.duration=5.627705ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T10:59:00.520923253Z" level=info msg="finished unary call with code OK" ctx_id=770ffec8-9e24-4846-abc9-dbafdbf997fb grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:00Z" grpc.time_ms=1.807 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:03.529176286Z" level=info msg="finished unary call with code OK" ctx_id=124a6084-ec42-43e2-ae33-37a7f084c801 grpc.code=OK grpc.method=GetServiceProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:03Z" grpc.time_ms=1.248 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:03.667266252Z" level=info msg="finished unary call with code OK" ctx_id=e93e1bd1-c34a-4e13-8944-de171d3b4935 grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:03Z" grpc.time_ms=1.426 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:03.773893891Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=bde69211-f028-4417-88c9-fdde210578fa
time="2022-05-06T10:59:03.775056162Z" level=error msg="gateway: handle gateway stats error" ctx_id=bde69211-f028-4417-88c9-fdde210578fa error="update gateway state error: object does not exist"
time="2022-05-06T10:59:06.631545456Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=9f2fc61b-5184-4957-b6e6-cf49bc9a74ea
time="2022-05-06T10:59:06.633671238Z" level=info msg="storage: gateway state updated" ctx_id=9f2fc61b-5184-4957-b6e6-cf49bc9a74ea gateway_id=e45f01fffe14ccfe
time="2022-05-06T10:59:06.640293579Z" level=info msg="finished client unary call" ctx_id=9f2fc61b-5184-4957-b6e6-cf49bc9a74ea grpc.code=OK grpc.ctx_id=cd86c7bc-7901-4e17-befa-490234b73699 grpc.duration=5.341726ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T10:59:08.853991855Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=StreamFrameLogsForGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:07Z" grpc.time_ms=1265.902 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:31.032261223Z" level=info msg="finished unary call with code OK" ctx_id=01c44d7c-9ca8-4808-b71e-6edf940299b7 grpc.code=OK grpc.method=GetVersion grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:31Z" grpc.time_ms=0.06 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:32.128675649Z" level=info msg="finished unary call with code OK" ctx_id=73e1bafc-a81b-4873-a1f5-09b927b708fc grpc.code=OK grpc.method=GetServiceProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:32Z" grpc.time_ms=0.967 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:33.305539574Z" level=info msg="finished unary call with code OK" ctx_id=1770baf9-f854-4df0-95ba-2f90fce6b2b3 grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:33Z" grpc.time_ms=1.4 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T10:59:33.773356113Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=ba9fbf02-dcf9-41b1-85b2-2333486319db
time="2022-05-06T10:59:33.775696995Z" level=error msg="gateway: handle gateway stats error" ctx_id=ba9fbf02-dcf9-41b1-85b2-2333486319db error="update gateway state error: object does not exist"
time="2022-05-06T10:59:36.635678994Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=1a2f3f14-2566-4174-8ef8-078984595330
time="2022-05-06T10:59:36.637772161Z" level=info msg="storage: gateway state updated" ctx_id=1a2f3f14-2566-4174-8ef8-078984595330 gateway_id=e45f01fffe14ccfe
time="2022-05-06T10:59:36.644217771Z" level=info msg="finished client unary call" ctx_id=1a2f3f14-2566-4174-8ef8-078984595330 grpc.code=OK grpc.ctx_id=eabee916-ed37-4600-ac4a-1e3fadff4ab3 grpc.duration=5.233474ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T10:59:57.872564563Z" level=info msg="finished unary call with code OK" ctx_id=b0227908-a48f-45b3-a2b4-057fb4c59edc grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T10:59:57Z" grpc.time_ms=1.32 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:01.918865984Z" level=info msg="storage: gateway deleted" ctx_id=d70b34ea-f763-4221-8fee-de3edc5bd720 gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:00:01.919580853Z" level=info msg="finished unary call with code OK" ctx_id=d70b34ea-f763-4221-8fee-de3edc5bd720 grpc.code=OK grpc.method=DeleteGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:01Z" grpc.time_ms=3.302 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:03.774083289Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=cfcd2736-da68-4310-a16e-98e69697a9db
time="2022-05-06T11:00:03.77476636Z" level=error msg="gateway: handle gateway stats error" ctx_id=cfcd2736-da68-4310-a16e-98e69697a9db error="update gateway state error: object does not exist"
time="2022-05-06T11:00:05.899627493Z" level=info msg="storage: gateway created" ctx_id=4203366c-8c20-4829-aff4-6f0a372e0afb gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:00:05.902265679Z" level=info msg="finished unary call with code OK" ctx_id=4203366c-8c20-4829-aff4-6f0a372e0afb grpc.code=OK grpc.method=CreateGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:05Z" grpc.time_ms=9.278 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:06.635982507Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=82522a21-c8d0-4ba2-a1d6-075e112e2991
time="2022-05-06T11:00:06.640079737Z" level=info msg="storage: gateway state updated" ctx_id=82522a21-c8d0-4ba2-a1d6-075e112e2991 gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:00:06.647874558Z" level=info msg="finished client unary call" ctx_id=82522a21-c8d0-4ba2-a1d6-075e112e2991 grpc.code=OK grpc.ctx_id=ffcc03a1-3a30-481d-bbfb-641c0ff7a74d grpc.duration=5.239293ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T11:00:11.717350292Z" level=info msg="finished unary call with code OK" ctx_id=a75cd3b4-244d-4dca-9795-13d091681cdf grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:11Z" grpc.time_ms=1.264 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:21.308988848Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=StreamFrameLogsForGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:17Z" grpc.time_ms=3783.655 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:23.94905556Z" level=info msg="finished unary call with code OK" ctx_id=72c181dd-f9f6-4b2e-9aee-dc9f5be08301 grpc.code=OK grpc.method=GetServiceProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:23Z" grpc.time_ms=0.828 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:24.08725999Z" level=info msg="finished unary call with code OK" ctx_id=32ab79b4-2eeb-44d8-a8b5-efd35fb7b248 grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:24Z" grpc.time_ms=1.66 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:27.74650359Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=StreamFrameLogsForGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:26Z" grpc.time_ms=1493.1 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:29.457035818Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=StreamFrameLogsForGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:00:28Z" grpc.time_ms=1128.697 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:00:33.779365311Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=d66b790b-6e98-44bd-9029-f463bbb077a6
time="2022-05-06T11:00:33.780493411Z" level=error msg="gateway: handle gateway stats error" ctx_id=d66b790b-6e98-44bd-9029-f463bbb077a6 error="update gateway state error: object does not exist"
time="2022-05-06T11:00:36.633147192Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=99441caf-72e5-4836-82ab-c577e7e40bce
time="2022-05-06T11:00:36.635000178Z" level=info msg="storage: gateway state updated" ctx_id=99441caf-72e5-4836-82ab-c577e7e40bce gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:00:36.6419344Z" level=info msg="finished client unary call" ctx_id=99441caf-72e5-4836-82ab-c577e7e40bce grpc.code=OK grpc.ctx_id=18af9b26-7b8b-43e0-ad61-918ea5664f00 grpc.duration=5.595077ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T11:01:03.780032275Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=5dc0dee0-ce64-45aa-982b-9ebd09212f8f
time="2022-05-06T11:01:03.7815038Z" level=error msg="gateway: handle gateway stats error" ctx_id=5dc0dee0-ce64-45aa-982b-9ebd09212f8f error="update gateway state error: object does not exist"
time="2022-05-06T11:01:06.635165969Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=705dc21a-58f7-4ccd-ac2a-7b4757cb398b
time="2022-05-06T11:01:06.637257711Z" level=info msg="storage: gateway state updated" ctx_id=705dc21a-58f7-4ccd-ac2a-7b4757cb398b gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:01:06.64437753Z" level=info msg="finished client unary call" ctx_id=705dc21a-58f7-4ccd-ac2a-7b4757cb398b grpc.code=OK grpc.ctx_id=016cc9df-73d2-4e83-9773-bf1f176458bf grpc.duration=6.138224ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T11:01:17.991957982Z" level=info msg="finished unary call with code OK" ctx_id=3b5004f0-8ea9-4a68-964c-76bfbf237ed5 grpc.code=OK grpc.method=GetGatewayProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:01:17Z" grpc.time_ms=1.699 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:01:17.995450716Z" level=info msg="finished unary call with code OK" ctx_id=28622d19-984e-481d-8213-03d8decf40fa grpc.code=OK grpc.method=GetServiceProfile grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:01:17Z" grpc.time_ms=0.773 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:01:26.440138254Z" level=info msg="finished unary call with code OK" ctx_id=0f34c16a-467c-4bd4-99a0-544796a6a704 grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:01:26Z" grpc.time_ms=1.394 peer.address="172.19.0.7:54974" span.kind=server system=grpc
time="2022-05-06T11:01:33.779532138Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccbf stats_id=1a221018-95b9-4af9-bec9-4cd83e64a8b5
time="2022-05-06T11:01:33.781604674Z" level=error msg="gateway: handle gateway stats error" ctx_id=1a221018-95b9-4af9-bec9-4cd83e64a8b5 error="update gateway state error: object does not exist"
time="2022-05-06T11:01:36.636941355Z" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=e45f01fffe14ccfe stats_id=99b209bd-2fc6-403a-82b2-41af83a6b718
time="2022-05-06T11:01:36.638613716Z" level=info msg="storage: gateway state updated" ctx_id=99b209bd-2fc6-403a-82b2-41af83a6b718 gateway_id=e45f01fffe14ccfe
time="2022-05-06T11:01:36.645072653Z" level=info msg="finished client unary call" ctx_id=99b209bd-2fc6-403a-82b2-41af83a6b718 grpc.code=OK grpc.ctx_id=e2395aa6-e5c8-48b8-9a8a-fe42714b9496 grpc.duration=5.149953ms grpc.method=HandleGatewayStats grpc.service=as.ApplicationServerService span.kind=client system=grpc
time="2022-05-06T11:01:39.879388696Z" level=warning msg="[core] grpc: addrConn.createTransport failed to connect to {chirpstack-application-server:8001 chirpstack-application-server:8001 <nil> 0 <nil>}. Err: connection error: desc = \"transport: Error while dialing dial tcp 172.19.0.7:8001: connect: connection refused\". Reconnecting..."
time="2022-05-06T11:01:40.232488006Z" level=info msg="signal received" signal=terminated
time="2022-05-06T11:01:40.232527609Z" level=warning msg="stopping chirpstack-network-server"
time="2022-05-06T11:01:40.23255544Z" level=info msg="gateway/mqtt: closing backend"
time="2022-05-06T11:01:40.232562924Z" level=info msg="gateway/mqtt: unsubscribing from event topic" topic=gateway/+/event/+
time="2022-05-06T11:01:40.232830349Z" level=info msg="backend/gateway: handling last messages"
time="2022-05-06T11:01:40.232845549Z" level=info msg="uplink: waiting for pending actions to complete"
time="2022-05-06T11:02:33.865924777Z" level=info msg="starting ChirpStack Network Server" band=EU868 docs="https://www.chirpstack.io/" net_id=000000 version=
time="2022-05-06T11:02:33.8664491Z" level=info msg="storage: setting up storage module"
time="2022-05-06T11:02:33.866465721Z" level=info msg="storage: setting up Redis client"
time="2022-05-06T11:02:33.866485507Z" level=info msg="storage: connecting to PostgreSQL"
time="2022-05-06T11:02:33.999585578Z" level=info msg="storage: applying PostgreSQL data migrations"
time="2022-05-06T11:02:34.028964467Z" level=info msg="gateway/mqtt: connecting to mqtt broker" server="tcp://mosquitto:1883"
time="2022-05-06T11:02:34.031231246Z" level=info msg="api: starting network-server api server" bind="0.0.0.0:8000" ca-cert= tls-cert= tls-key=
time="2022-05-06T11:02:34.032330587Z" level=debug msg="api/roaming: roaming is disabled"
time="2022-05-06T11:02:34.033482729Z" level=info msg="starting downlink device-queue scheduler"
time="2022-05-06T11:02:34.033503498Z" level=info msg="starting multicast scheduler"
time="2022-05-06T11:02:34.033670129Z" level=info msg="backend/gateway: connected to mqtt server"
time="2022-05-06T11:02:34.034164007Z" level=info msg="gateway/mqtt: subscribing to gateway event topic" qos=0 topic=gateway/+/event/+
time="2022-05-06T11:02:34.034503989Z" level=debug msg="running class-b / class-c scheduler batch" ctx_id=731be5cd-b667-42cb-a528-363daebf8539
time="2022-05-06T11:02:34.034693722Z" level=debug msg="running multicast scheduler batch" ctx_id=8500208a-4430-4f2b-9375-adc761e30c19
time="2022-05-06T11:02:34.072631872Z" level=debug msg="sql query executed" args="[100 2022-05-06 11:02:34.04512879 +0000 UTC m=+0.429555878]" duration=27.487702ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
time="2022-05-06T11:02:34.08460174Z" level=debug msg="sql query executed" args="[100 371075h2m54.045673786s 2022-05-06 11:02:34.045676374 +0000 UTC m=+0.430103396]" duration=38.915953ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
time="2022-05-06T11:02:35.073930643Z" level=debug msg="running multicast scheduler batch" ctx_id=b2129658-2824-44ae-96ef-29b6c6c18273
time="2022-05-06T11:02:35.075483986Z" level=debug msg="sql query executed" args="[100 2022-05-06 11:02:35.074850114 +0000 UTC m=+1.459277196]" duration="622.244µs" query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
time="2022-05-06T11:02:35.085525301Z" level=debug msg="running class-b / class-c scheduler batch" ctx_id=7d306c6d-a566-405a-ac2d-11fe686e59ba
time="2022-05-06T11:02:35.087709104Z" level=debug msg="sql query executed" args="[100 371075h2m55.086316323s 2022-05-06 11:02:35.086319231 +0000 UTC m=+1.470746251]" duration=1.379808ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
time="2022-05-06T11:02:36.076413836Z" level=debug msg="running multicast scheduler batch" ctx_id=96709718-8367-48d0-804e-e9dacb00dd0a
time="2022-05-06T11:02:36.077562777Z" level=debug msg="sql query executed" args="[100 2022-05-06 11:02:36.076682658 +0000 UTC m=+2.461109694]" duration="869.617µs" query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
time="2022-05-06T11:02:36.089261153Z" level=debug msg="running class-b / class-c scheduler batch" ctx_id=acd5ecc2-cbcd-473e-acb2-e91050717fa9
time="2022-05-06T11:02:36.09091492Z" level=debug msg="sql query executed" args="[100 371075h2m56.089597078s 2022-05-06 11:02:36.089600142 +0000 UTC m=+2.474027161]" duration=1.303374ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
time="2022-05-06T11:02:36.757449603Z" level=debug msg="sql query executed" args="[[228 95 1 255 254 20 204 254]]" duration=2.491186ms query="select * from gateway where gateway_id = $1"
time="2022-05-06T11:02:36.762787786Z" level=debug msg="sql query executed" args="[e45f01fffe14ccfe]" duration=1.033187ms query="\n\t\tselect\n\t\t\tfpga_id,\n\t\t\tfine_timestamp_key\n\t\tfrom\n\t\t\tgateway_board\n\t\twhere\n\t\t\tgateway_id = $1\n\t\torder by\n\t\t\tid\n\t\t"
time="2022-05-06T11:02:36.763327419Z" level=info msg="finished unary call with code OK" ctx_id=d8b2271e-2a75-430c-bd46-276ecc4e0089 grpc.code=OK grpc.method=GetGateway grpc.service=ns.NetworkServerService grpc.start_time="2022-05-06T11:02:36Z" grpc.time_ms=8.391 peer.address="172.19.0.7:52108" span.kind=server system=grpc
time="2022-05-06T11:02:37.078743649Z" level=debug msg="running multicast scheduler batch" ctx_id=af6e95cc-1305-46fb-b854-58d1cd14273a
time="2022-05-06T11:02:37.080212972Z" level=debug msg="sql query executed" args="[100 2022-05-06 11:02:37.079562016 +0000 UTC m=+3.463989121]" duration="640.139µs" query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
time="2022-05-06T11:02:37.093096242Z" level=debug msg="running class-b / class-c scheduler batch" ctx_id=cbc6ac22-2c7d-40a4-8a04-f96e37a77feb
time="2022-05-06T11:02:37.095078756Z" level=debug msg="sql query executed" args="[100 371075h2m57.093800663s 2022-05-06 11:02:37.093803517 +0000 UTC m=+3.478230532]" duration=1.264837ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
time="2022-05-06T11:02:38.081924985Z" level=debug msg="running multicast scheduler batch" ctx_id=0f0f746d-43c9-4d63-bfe5-f84232aa6782
time="2022-05-06T11:02:38.083032579Z" level=debug msg="sql query executed" args="[100 2022-05-06 11:02:38.082188915 +0000 UTC m=+4.466616009]" duration="832.966µs" query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
time="2022-05-06T11:02:38.096584439Z" level=debug msg="running class-b / class-c scheduler batch" ctx_id=bd2b3420-feb7-486b-a42f-e1059dae2f06
time="2022-05-06T11:02:38.098591235Z" level=debug msg="sql query executed" args="[100 371075h2m58.096787259s 2022-05-06 11:02:38.096790311 +0000 UTC m=+4.481217323]" duration=1.790273ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n   

Your Environment

Component	Version
Application Server	v?.?.?
Network Server
Gateway Bridge
Chirpstack API
Geolocation
Concentratord

[winnieXY]

I upgraded my gatway from debian buster to debian bullseye yesterday and after that the gateway/network server stopped working. So far I'm not able to find the issue - but i get a very similiar error message:

Jun 21 08:55:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:55:02.37173928+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffeaXXXX stats_id=95ba73ce-1cd6-499c-8163-c2f03bf0cc24

Jun 21 08:55:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:55:02.375361748+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=95ba73ce-1cd6-499c-8163-c2f03bf0cc24 error="update gateway state error: object does not exist"

Jun 21 08:55:32 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:55:32.364950213+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffeaXXXX stats_id=a848aa4a-d42f-40f9-bb08-5613e9d81118

Jun 21 08:55:32 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:55:32.369636422+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=a848aa4a-d42f-40f9-bb08-5613e9d81118 error="update gateway state error: object does not exist"

Jun 21 08:56:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:56:02.365769431+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffeaXXXX stats_id=9b9dd315-b759-45e1-9751-41c1d7ba20e4

Jun 21 08:56:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T08:56:02.370086998+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=9b9dd315-b759-45e1-9751-41c1d7ba20e4 error="update gateway state error: object does not exist"

My setup looks like this: raspberry pi with rasbian (now bullseye):

root@rak-gateway:/opt/ttn-gateway/packet_forwarder/lora_pkt_fwd# dpkg -l | grep chirpstack
ii chirpstack-application-server 3.17.6 armhf ChirpStack Application Server
ii chirpstack-gateway-bridge 3.14.0 armhf ChirpStack Gateway Bridge
ii chirpstack-network-server 3.16.2 armhf ChirpStack Network Server

I've already created new postgresql databases for chirpstack_as and chirpstack_ns (switched to chirpstack_as2 and chirpstack_ns2). I can reproduce this error even with this fresh setup.

Within the applicaton-server frontend the gatway is always listed as "Never" - so the communication is no longer working - although the gateway-bridge and the semtech_udp concentrator are up and running:

lora_pkt_fwd output:

##### 2022-06-21 08:01:27 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 1
# CRC_OK: 0.00%, CRC_FAIL: 100.00%, NO_CRC: 0.00%
# RF packets forwarded: 0 (0 bytes)
# PUSH_DATA datagrams sent: 0 (0 bytes)
# PUSH_DATA acknowledged: 0.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 3 (100.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
# BEACON queued: 0
# BEACON sent so far: 0
# BEACON rejected: 0
### [JIT] ###
# SX1301 time (PPS): 2665592
src/jitqueue.c:448:jit_print_queue(): INFO: [jit] queue is empty
### [GPS] ###
# Invalid time reference (age: 1655798487 sec)
# no valid GPS coordinates available yet
##### END #####

gateway-bridge output

Jun 21 09:01:02 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:01:02.37340725+01:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=5c182a54-6fef-431b-84ba-5653a89e70a5 topic=gateway/b827ebfffeaXXXX/event/stats
Jun 21 09:01:27 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:01:27.193790161+01:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=b5801d7f-dab0-42ef-9113-16774c7836c8 topic=gateway/b827ebfffeaXXXX/event/stats
Jun 21 09:01:32 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:01:32.370461048+01:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=7e53b92b-644e-4d64-98ca-b06dcc72d2cd topic=gateway/b827ebfffeaXXXX/event/stats
Jun 21 09:02:02 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:02:02.36933739+01:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=409220ae-5e5e-4910-a041-ce0de90f2add topic=gateway/b827ebfffeaXXXX/event/stats

Any help appreciated getting it back running.

[winnieXY]

FYI I'm going through this debug help now:
https://www.chirpstack.io/project/guides/connect-gateway/

INFO: [down] PULL_ACK received in 0 ms
##### 2022-06-21 08:07:35 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 0
# CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 0 (0 bytes)
# PUSH_DATA datagrams sent: 1 (113 bytes)
# PUSH_DATA acknowledged: 100.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 3 (100.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
# BEACON queued: 0
# BEACON sent so far: 0
# BEACON rejected: 0
### [JIT] ###
# SX1301 time (PPS): 89660477
src/jitqueue.c:448:jit_print_queue(): INFO: [jit] queue is empty
### [GPS] ###
# Invalid time reference (age: 1655798855 sec)
# no valid GPS coordinates available yet
##### END #####
JSON up: {"stat":{"time":"2022-06-21 08:07:35 GMT","rxnb":0,"rxok":0,"rxfw":0,"ackr":100.0,"dwnb":0,"txnb":0}}
INFO: [up] PUSH_ACK received in 1 ms

=> Package Forwarder is working

Subscribing to topic for gateway-bridge is working:

sudo journalctl -f -n 1000 -u chirpstack-gateway-bridge | grep subscribing
Jun 21 08:15:32 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T08:15:32.363117187+01:00" level=info msg="integration/mqtt: subscribing to topic" qos=0 topic="gateway/b827ebfffea341b8/command/#"

Receiving Data on gateway-bridge is working:

Jun 21 09:06:08 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:06:08.412908762+01:00" level=info msg="integration/mqtt: publishing event" event=up qos=0 topic=gateway/b827ebfffea341b8/event/up uplink_id=12b670b1-0957-46c6-9805-41911ba75073
Jun 21 09:06:32 rak-gateway chirpstack-gateway-bridge[6788]: time="2022-06-21T09:06:32.372681453+01:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=99476335-fd93-4425-b54e-9e74118a39eb topic=gateway/b827ebfffea341b8/event/stats

I get the stats also within mosquitto_sub:

root@rak-gateway:/opt/ttn-gateway/packet_forwarder/lora_pkt_fwd# mosquitto_sub -v -t "gateway/#"
gateway/b827ebfffef614ed/state/conn                                                                                                                                                                           ▒'▒▒▒▒▒
gateway/b827ebfffea341b8/state/conn                                                                                                                                                                           ▒'▒▒▒▒A▒
gateway/b827ebfffea341b8/event/stats                                                                                                                                                                          ▒'▒▒▒▒A▒▒ŕJ192.168.68.120Z▒▒φ▒L*▒▒▒fb▒▒
gateway/b827ebfffea341b8/event/stats                                                                                                                                                                          ▒'▒▒▒▒A▒▒ŕJ192.168.68.120Zdg▒▒V▒A▒▒▒6▒▒▒▒
gateway/b827ebfffea341b8/event/stats                                                                                                                                                                          ▒▒4▒▒▒A▒▒ŕJ192.168.68.120Z▒y▒▒▒▒Os▒▒
gateway/b827ebfffea341b8/event/stats                                                                                                                                                                          ▒'▒▒▒▒A▒▒ŕJ192.168.68.120Z▒*ņ▒EL▒▒蹺▒']

(Sorry - binary is not shown here somehow).

The network-server receives the gateway stats packet:

Jun 21 09:17:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T09:17:02.385186343+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffea341b8 stats_id=9c490976-4c61-4476-8581-1084077f7a34
Jun 21 09:17:02 rak-gateway chirpstack-network-server[6803]: time="2022-06-21T09:17:02.38887053+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=9c490976-4c61-4476-8581-1084077f7a34 error="update gateway state error: object does not exist"

As the below output with more debug seems to fail after a sql update statement I downgraded the postgresql database again to 11 (from 13) - however this didn't solve that problem.

Same output with debug level increased - last entry within the post - warning long snipplet!

Jun 21 09:20:02 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:02.383846941+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffea341b8 stats_id=710f20e1-1a20-419a-b816-254392708ab5
Jun 21 09:20:02 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:02.389909649+01:00" level=debug msg="sql query executed" args="[b827ebfffea341b8 2022-06-21 09:20:02.384181001 +0100 BST m=+85.815392995 2022-06-21 09:20:02.384181001 +0100 BST m=+85.815392995 <nil> <nil>]" duration=5.653701ms query="\n\t\tupdate gateway set\n\t\t\tfirst_seen_at = coalesce(first_seen_at, $2),\n\t\t\tlast_seen_at = $3,\n\t\t\tlocation = coalesce($4, location),\n\t\t\taltitude = coalesce($5, altitude)\n\t\twhere\n\t\t\tgateway_id = $1"
Jun 21 09:20:02 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:02.390547092+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=710f20e1-1a20-419a-b816-254392708ab5 error="update gateway state error: object does not exist"
Jun 21 09:20:02 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:02.720600134+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=3fab9bd5-bc1e-45a3-88df-d41e78b87174
Jun 21 09:20:02 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:02.728755845+01:00" level=debug msg="sql query executed" args="[100 372176h20m22.722070225s 2022-06-21 09:20:02.722095225 +0100 BST m=+86.153306906]" duration=6.569266ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:03 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:03.366547798+01:00" level=debug msg="running multicast scheduler batch" ctx_id=ffd8a793-831a-4894-aecd-40a41506dc14
Jun 21 09:20:03 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:03.370234224+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:03.367616174 +0100 BST m=+86.798828012]" duration=2.516385ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:03 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:03.731385748+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=f3ec9604-2cd5-4c4b-91f2-14cb17704b8f
Jun 21 09:20:03 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:03.739484741+01:00" level=debug msg="sql query executed" args="[100 372176h20m23.732497301s 2022-06-21 09:20:03.732520634 +0100 BST m=+87.163732420]" duration=6.874836ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:04 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:04.372028093+01:00" level=debug msg="running multicast scheduler batch" ctx_id=dfd76747-2d11-41eb-9d22-729d3a875e1a
Jun 21 09:20:04 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:04.375954934+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:04.373262926 +0100 BST m=+87.804474660]" duration=2.617009ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:04 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:04.742276049+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=4f7b52f6-b52f-4b59-985c-eb30c2816e82
Jun 21 09:20:04 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:04.749957598+01:00" level=debug msg="sql query executed" args="[100 372176h20m24.743338227s 2022-06-21 09:20:04.743371248 +0100 BST m=+88.174582930]" duration=6.509736ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:05 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:05.377941406+01:00" level=debug msg="running multicast scheduler batch" ctx_id=dcd402ab-0b7e-4104-a415-14a75951c286
Jun 21 09:20:05 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:05.381625645+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:05.379114104 +0100 BST m=+88.810325838]" duration=2.422792ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:05 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:05.753080413+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=45eb9114-daa6-4077-8656-bd2f19c9ade5
Jun 21 09:20:05 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:05.760824149+01:00" level=debug msg="sql query executed" args="[100 372176h20m25.754025926s 2022-06-21 09:20:05.754047384 +0100 BST m=+89.185259118]" duration=6.690099ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:06 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:06.384029561+01:00" level=debug msg="running multicast scheduler batch" ctx_id=7432ac9d-d4d5-4018-b52f-a2e76ba966df
Jun 21 09:20:06 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:06.387804476+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:06.385129968 +0100 BST m=+89.816341701]" duration=2.600134ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:06 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:06.764125661+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=fc5e7ae4-b78f-46fa-b9c5-3f188933793e
Jun 21 09:20:06 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:06.770790864+01:00" level=debug msg="sql query executed" args="[100 372176h20m26.765067996s 2022-06-21 09:20:06.765088569 +0100 BST m=+90.196300199]" duration=5.631357ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:07 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:07.389561626+01:00" level=debug msg="running multicast scheduler batch" ctx_id=52639fa3-12a4-41b1-96ce-2cdf2c5ea1a4
Jun 21 09:20:07 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:07.393192793+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:07.390562087 +0100 BST m=+90.821773768]" duration=2.542634ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:07 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:07.773667432+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=e94bef51-9a86-42a4-8212-1e5882f1f457
Jun 21 09:20:07 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:07.780951431+01:00" level=debug msg="sql query executed" args="[100 372176h20m27.774603361s 2022-06-21 09:20:07.774625861 +0100 BST m=+91.205837595]" duration=6.244009ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:08 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:08.395193014+01:00" level=debug msg="running multicast scheduler batch" ctx_id=9ae7f647-63fa-49df-81d9-c3bcb7398de0
Jun 21 09:20:08 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:08.398711265+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:08.396164516 +0100 BST m=+91.827376927]" duration=2.470499ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:08 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:08.783931436+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=6f59a039-c4f9-4987-b95f-6b7789ae78f1
Jun 21 09:20:08 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:08.791286789+01:00" level=debug msg="sql query executed" args="[100 372176h20m28.785015749s 2022-06-21 09:20:08.785039447 +0100 BST m=+92.216251129]" duration=6.170051ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:09 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:09.400718101+01:00" level=debug msg="running multicast scheduler batch" ctx_id=d5dc7af8-b9f2-470a-8aa1-e208db8a48e2
Jun 21 09:20:09 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:09.404614786+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:09.402042829 +0100 BST m=+92.833254615]" duration=2.486958ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:09 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:09.794864445+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=ce88dd85-f0d9-42a7-b1a0-6b6a6f0b660b
Jun 21 09:20:09 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:09.803589942+01:00" level=debug msg="sql query executed" args="[100 372176h20m29.79590204s 2022-06-21 09:20:09.795942769 +0100 BST m=+93.227154502]" duration=7.565768ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:10 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:10.40645329+01:00" level=debug msg="running multicast scheduler batch" ctx_id=85226c1d-417d-4b08-8a2f-dca847031a3b
Jun 21 09:20:10 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:10.409030247+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:10.407382865 +0100 BST m=+93.838594443]" duration=1.600142ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:10 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:10.805913338+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=1efb3edf-b6b3-429b-be7b-f3b05ed14d13
Jun 21 09:20:10 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:10.814930187+01:00" level=debug msg="sql query executed" args="[100 372176h20m30.807005255s 2022-06-21 09:20:10.807027599 +0100 BST m=+94.238239333]" duration=7.813995ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:11 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:11.410479535+01:00" level=debug msg="running multicast scheduler batch" ctx_id=27f687b5-fce4-49f2-9876-f92cdfb6d169
Jun 21 09:20:11 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:11.414761321+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:11.411836398 +0100 BST m=+94.843048132]" duration=2.838413ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:11 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:11.817105198+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=251a45a1-477f-44cc-8855-f2300a1ce42c
Jun 21 09:20:11 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:11.825258409+01:00" level=debug msg="sql query executed" args="[100 372176h20m31.818148679s 2022-06-21 09:20:11.818183314 +0100 BST m=+95.249395048]" duration=6.996294ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:12 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:12.417099716+01:00" level=debug msg="running multicast scheduler batch" ctx_id=c8f30ddf-7eec-44e9-9fff-43251b9b9d36
Jun 21 09:20:12 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:12.421076505+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:12.418391632 +0100 BST m=+95.849603314]" duration=2.606019ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:12 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:12.827693783+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=f7ce4932-7601-4fe2-8458-f04387966a0e
Jun 21 09:20:12 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:12.835016532+01:00" level=debug msg="sql query executed" args="[100 372176h20m32.828671222s 2022-06-21 09:20:12.828693774 +0100 BST m=+96.259905456]" duration=6.246196ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:13 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:13.423061987+01:00" level=debug msg="running multicast scheduler batch" ctx_id=64cebc5a-5d93-468a-92e1-4705321e2cc0
Jun 21 09:20:13 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:13.426727216+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:13.424250622 +0100 BST m=+96.855462356]" duration=2.404562ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:13 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:13.837845392+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=35aabe17-1977-49b6-9bec-e8dfb87b511b
Jun 21 09:20:13 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:13.845367359+01:00" level=debug msg="sql query executed" args="[100 372176h20m33.838967987s 2022-06-21 09:20:13.838991268 +0100 BST m=+97.270203053]" duration=6.300415ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:14 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:14.429007278+01:00" level=debug msg="running multicast scheduler batch" ctx_id=734a01e5-c377-446b-a453-9f024c06b49c
Jun 21 09:20:14 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:14.432478237+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:14.430032842 +0100 BST m=+97.861244523]" duration=2.375188ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:14 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:14.847997001+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=19efab8b-4b9f-4793-b951-115dffa50b7b
Jun 21 09:20:14 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:14.858310037+01:00" level=debug msg="sql query executed" args="[100 372176h20m34.851816395s 2022-06-21 09:20:14.851840145 +0100 BST m=+98.283051827]" duration=6.321924ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:15 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:15.435071839+01:00" level=debug msg="running multicast scheduler batch" ctx_id=71193098-067e-46dc-9986-b321d6b724f0
Jun 21 09:20:15 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:15.438708526+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:15.436087038 +0100 BST m=+98.867298772]" duration=2.546905ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:15 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:15.861144991+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=3481d1e9-917e-4fc9-841c-f62888e794fc
Jun 21 09:20:16 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:15.868967059+01:00" level=debug msg="sql query executed" args="[100 372176h20m35.862279356s 2022-06-21 09:20:15.862301283 +0100 BST m=+99.293512964]" duration=6.589891ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:16 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:16.440544842+01:00" level=debug msg="running multicast scheduler batch" ctx_id=10863835-ab9d-4cca-9570-2043dd5bad0c
Jun 21 09:20:16 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:16.444745014+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:16.442387847 +0100 BST m=+99.873599581]" duration=2.287845ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:16 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:16.87095379+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=524d8fee-3839-4f60-b9c8-f5f6529ca78b
Jun 21 09:20:16 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:16.879024971+01:00" level=debug msg="sql query executed" args="[100 372176h20m36.872790858s 2022-06-21 09:20:16.872813878 +0100 BST m=+100.304025560]" duration=6.134479ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:17 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:17.447962204+01:00" level=debug msg="running multicast scheduler batch" ctx_id=20812535-df54-459b-a490-762ce195452b
Jun 21 09:20:17 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:17.452117949+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:17.449435733 +0100 BST m=+100.880647466]" duration=2.606332ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:17 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:17.881599614+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=5d42f6cd-3067-4b93-bb21-bde429701471
Jun 21 09:20:17 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:17.889052622+01:00" level=debug msg="sql query executed" args="[100 372176h20m37.882711583s 2022-06-21 09:20:17.882745593 +0100 BST m=+101.313957327]" duration=6.227811ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:18 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:18.455350295+01:00" level=debug msg="running multicast scheduler batch" ctx_id=d7248786-66a2-4a1f-ab7a-0caa9f1fc0d0
Jun 21 09:20:18 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:18.459056044+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:18.456420025 +0100 BST m=+101.887631759]" duration=2.560916ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:18 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:18.892237208+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=dd2a9cc8-6d5f-460c-9064-1e63cb110696
Jun 21 09:20:18 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:18.899785059+01:00" level=debug msg="sql query executed" args="[100 372176h20m38.893337615s 2022-06-21 09:20:18.893370271 +0100 BST m=+102.324582005]" duration=6.335571ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:19 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:19.46068481+01:00" level=debug msg="running multicast scheduler batch" ctx_id=f97c0a1d-9ae7-4ab4-9f5c-5d300b241cb1
Jun 21 09:20:19 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:19.46461337+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:19.461734332 +0100 BST m=+102.892946066]" duration=2.803413ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:19 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:19.901921946+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=5ac70c4d-5d75-4336-9631-d3d007622fd3
Jun 21 09:20:19 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:19.909453235+01:00" level=debug msg="sql query executed" args="[100 372176h20m39.902921573s 2022-06-21 09:20:19.902944749 +0100 BST m=+103.334156431]" duration=6.339164ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:20 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:20.466349166+01:00" level=debug msg="running multicast scheduler batch" ctx_id=88579b78-f70f-444b-89b1-79502e4c9f21
Jun 21 09:20:20 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:20.470246372+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:20.467521864 +0100 BST m=+103.898733546]" duration=2.649248ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:20 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:20.913267294+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=b999c0c5-3578-4778-a061-2875428f3766
Jun 21 09:20:20 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:20.920571815+01:00" level=debug msg="sql query executed" args="[100 372176h20m40.9142539s 2022-06-21 09:20:20.914278171 +0100 BST m=+104.345489905]" duration=6.217551ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:21 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:21.472592215+01:00" level=debug msg="running multicast scheduler batch" ctx_id=c6358838-015b-4143-bfd0-18feb4b91eab
Jun 21 09:20:21 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:21.47665645+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:21.473794444 +0100 BST m=+104.905006229]" duration=2.788309ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:21 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:21.923143228+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=aff31306-0cc7-41a7-9a4e-361011ecabee
Jun 21 09:20:21 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:21.930603841+01:00" level=debug msg="sql query executed" args="[100 372176h20m41.924272542s 2022-06-21 09:20:21.924296343 +0100 BST m=+105.355507973]" duration=6.227759ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:22 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:22.478472454+01:00" level=debug msg="running multicast scheduler batch" ctx_id=a79f6ef2-022f-4773-95fd-fd0bdd819568
Jun 21 09:20:22 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:22.482072319+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:22.479574215 +0100 BST m=+105.910785949]" duration=2.418729ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:22 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:22.933254317+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=a9fd1bb0-b06f-4304-b350-01d07104841e
Jun 21 09:20:22 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:22.940648732+01:00" level=debug msg="sql query executed" args="[100 372176h20m42.934355818s 2022-06-21 09:20:22.934378994 +0100 BST m=+106.365590728]" duration=6.19677ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:23 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:23.48469519+01:00" level=debug msg="running multicast scheduler batch" ctx_id=c07d490d-75f8-479b-b170-1d854d209262
Jun 21 09:20:23 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:23.488228233+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:23.485700286 +0100 BST m=+106.916911967]" duration=2.442114ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:23 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:23.943316343+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=c949a3c1-9476-437c-8481-54fae20e67a7
Jun 21 09:20:23 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:23.950736435+01:00" level=debug msg="sql query executed" args="[100 372176h20m43.944392427s 2022-06-21 09:20:23.944416177 +0100 BST m=+107.375627911]" duration=6.238696ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:24 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:24.490713605+01:00" level=debug msg="running multicast scheduler batch" ctx_id=e265a261-9c03-4bca-a187-937e584223f5
Jun 21 09:20:24 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:24.494435136+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:24.491746201 +0100 BST m=+107.922957934]" duration=2.612529ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:24 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:24.953588628+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=bb7b90a3-f9f1-4e8f-a0a3-1baad57c14dd
Jun 21 09:20:24 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:24.961515174+01:00" level=debug msg="sql query executed" args="[100 372176h20m44.955225384s 2022-06-21 09:20:24.955249342 +0100 BST m=+108.386461128]" duration=6.187499ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:25 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:25.496105151+01:00" level=debug msg="running multicast scheduler batch" ctx_id=a5360b5c-ae2f-49b4-b882-5e270a6443e5
Jun 21 09:20:25 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:25.499522517+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:25.497013633 +0100 BST m=+108.928225366]" duration=2.431854ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:25 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:25.963886797+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=7ebfa771-0170-4cdd-a686-52b4f898aad5
Jun 21 09:20:25 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:25.971464284+01:00" level=debug msg="sql query executed" args="[100 372176h20m45.96488882s 2022-06-21 09:20:25.964911007 +0100 BST m=+109.396122741]" duration=6.47484ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:26 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:26.501322687+01:00" level=debug msg="running multicast scheduler batch" ctx_id=dbfffefd-54b8-459b-8221-d9e3bf1e100e
Jun 21 09:20:26 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:26.505042291+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:26.502546896 +0100 BST m=+109.933758629]" duration=2.414146ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:26 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:26.973591743+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=2258eb59-d283-4d45-85c5-741c3f10f22f
Jun 21 09:20:26 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:26.98139027+01:00" level=debug msg="sql query executed" args="[100 372176h20m46.974664703s 2022-06-21 09:20:26.974707202 +0100 BST m=+110.405918884]" duration=6.60911ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:27 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:27.507416623+01:00" level=debug msg="running multicast scheduler batch" ctx_id=1315f532-858b-4b57-b0c6-c5d1da871967
Jun 21 09:20:27 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:27.51101331+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:27.508600727 +0100 BST m=+110.939812409]" duration=2.337323ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:27 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:27.983616009+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=a11acd8c-0088-4809-a3a7-843aac87c89f
Jun 21 09:20:27 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:27.991199173+01:00" level=debug msg="sql query executed" args="[100 372176h20m47.984557876s 2022-06-21 09:20:27.98458074 +0100 BST m=+111.415792422]" duration=6.543642ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:28 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:28.513704202+01:00" level=debug msg="running multicast scheduler batch" ctx_id=87650441-65d2-4b6d-9490-ed215b71e7d2
Jun 21 09:20:28 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:28.517990103+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:28.515076065 +0100 BST m=+111.946287799]" duration=2.83659ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:28 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:28.993200591+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=666f752e-816b-4a8e-8654-8b5c1e0213ba
Jun 21 09:20:29 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:29.000519174+01:00" level=debug msg="sql query executed" args="[100 372176h20m48.994232093s 2022-06-21 09:20:28.994254801 +0100 BST m=+112.425466535]" duration=6.18552ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:29 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:29.520207561+01:00" level=debug msg="running multicast scheduler batch" ctx_id=d9da720e-152d-4e9d-9782-1dcd251c180a
Jun 21 09:20:29 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:29.524246172+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:29.521680829 +0100 BST m=+112.952892563]" duration=2.494354ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:30 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:30.003262982+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=a4b0a763-f641-467c-91bf-05d989f09c42
Jun 21 09:20:30 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:30.010530732+01:00" level=debug msg="sql query executed" args="[100 372176h20m50.004254119s 2022-06-21 09:20:30.004279692 +0100 BST m=+113.435491373]" duration=6.170468ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:30 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:30.526054936+01:00" level=debug msg="running multicast scheduler batch" ctx_id=ac70e4d2-bb61-4b3d-9871-acda249a0530
Jun 21 09:20:30 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:30.529547353+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:30.526998261 +0100 BST m=+113.958209942]" duration=2.463051ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:31 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:31.013324279+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=4074ddbe-5545-4cad-9d42-dbadfa8a8ba8
Jun 21 09:20:31 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:31.020803068+01:00" level=debug msg="sql query executed" args="[100 372176h20m51.014394738s 2022-06-21 09:20:31.014418905 +0100 BST m=+114.445630586]" duration=6.285154ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:31 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:31.53161325+01:00" level=debug msg="running multicast scheduler batch" ctx_id=67049c38-e545-4392-8aff-81a130e22121
Jun 21 09:20:31 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:31.53569134+01:00" level=debug msg="sql query executed" args="[100 2022-06-21 09:20:31.53284126 +0100 BST m=+114.964052994]" duration=2.771747ms query="\n\t\tselect\n\t\t\t*\n\t\tfrom\n\t\t\tmulticast_queue\n\t\twhere\n\t\t\tschedule_at <= $2\n\t\torder by\n\t\t\tid\n\t\tlimit $1\n\t\tfor update skip locked\n\t"
Jun 21 09:20:32 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:32.02325167+01:00" level=debug msg="running class-b / class-c scheduler batch" ctx_id=5a699f7b-4f34-47de-9308-046cb2d2dc2f
Jun 21 09:20:32 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:32.030887021+01:00" level=debug msg="sql query executed" args="[100 372176h20m52.024184162s 2022-06-21 09:20:32.024206349 +0100 BST m=+115.455418031]" duration=6.603224ms query="\n        select\n            d.*\n        from\n            device d\n        where\n\t\t\td.mode in ('B', 'C')\n            -- we want devices with queue items\n            and exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n                    and (\n\t\t\t\t\t\td.mode = 'C'\n                    \tor (\n\t\t\t\t\t\t\td.mode = 'B'\n                    \t\tand dq.emit_at_time_since_gps_epoch <= $2\n                    \t)\n                    )\n            )\n\t\t\t-- exclude device which have one of the following below\n            and not exists (\n                select\n                    1\n                from\n                    device_queue dq\n                where\n                    dq.dev_eui = d.dev_eui\n\t\t\t\t\tand (\n\t\t\t\t\t\t-- pending queue-item with timeout_after in the future\n\t\t\t\t\t\t(dq.is_pending = true and dq.timeout_after > $3)\n\n\t\t\t\t\t\t-- or retry_after set to a timestamp in the future\n\t\t\t\t\t\tor (dq.retry_after is not null and dq.retry_after > $3)\n\t\t\t\t\t)\n            )\n        order by\n            d.dev_eui\n        limit $1\n        for update of d skip locked"
Jun 21 09:20:32 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:32.38028974+01:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=b827ebfffea341b8 stats_id=d6c17ffd-bbb0-4b59-861b-05e7663af57b
Jun 21 09:20:32 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:32.383915333+01:00" level=debug msg="sql query executed" args="[b827ebfffea341b8 2022-06-21 09:20:32.380700101 +0100 BST m=+115.811911834 2022-06-21 09:20:32.380700101 +0100 BST m=+115.811911834 <nil> <nil>]" duration=3.147109ms query="\n\t\tupdate gateway set\n\t\t\tfirst_seen_at = coalesce(first_seen_at, $2),\n\t\t\tlast_seen_at = $3,\n\t\t\tlocation = coalesce($4, location),\n\t\t\taltitude = coalesce($5, altitude)\n\t\twhere\n\t\t\tgateway_id = $1"
Jun 21 09:20:32 rak-gateway chirpstack-network-server[8052]: time="2022-06-21T09:20:32.384567776+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=d6c17ffd-bbb0-4b59-861b-05e7663af57b error="update gateway state error: object does not exist"

[sagar-patel-sls]

Hi @winnieXY

Jun 21 09:20:32 rak-gateway chirpstack-network-server [8052]: time="2022-06-21T09:20:32.384567776+01:00" level=error msg="gateway: handle gateway stats error" ctx_id=d6c17ffd-bbb0-4b59-861b-05e7663af57b error="update gateway state error: object does not exist"

object does not exist error shows your gateway entity is not available in the database. I think you need to delete the gateway and add it to again in the Application Server.

[winnieXY]

I've added logging to the postgresql database to see the actual update statement. In fact this statement failed/updated no entry as with the update the device ID of the gateway had changed a little bit. I didn't checked that.

After deleting the old gateway and adding the new one I get at least the stats and the gateway is online according to the application-server.

However I still have no packages from my devices - no clue why yet. Maybe I've to force a join again/restart them.