history_node table keeps growing

[xtyinbiao]

temporal version: 2.22.3
db: mysql

set Retention Period 3 days:

I see workflows total count is :

and before 3 days just 7274， I think retention works.

but , history_node table keep growing, now has 19402165 count

[yycptt]

Hello @xtyinbiao, can you check if worker.historyScannerEnabled flag is enabled on the cluster? It should be enabled by defualt.

If it's enabled, can you use temporal -n temporal-system workflow list -q 'WorkflowType = "temporal-sys-history-scanner-workflow"' to see those internal scanner workflows are running properly?

In some failure cases, the first event batch of a workflow will be left in DB until the history scanner cleans them up. You can also try reducing worker.historyScannerDataMinAge (default is 60days) and let history scanner kick in sooner.

It looks like you are also using a quite old temporal version 1.22.4. This latest is 1.29.

[xtyinbiao]

Hello @xtyinbiao, can you check if worker.historyScannerEnabled flag is enabled on the cluster? It should be enabled by defualt.

If it's enabled, can you use temporal -n temporal-system workflow list -q 'WorkflowType = "temporal-sys-history-scanner-workflow"' to see those internal scanner workflows are running properly?

In some failure cases, the first event batch of a workflow will be left in DB until the history scanner cleans them up. You can also try reducing worker.historyScannerDataMinAge (default is 60days) and let history scanner kick in sooner.

It looks like you are also using a quite old temporal version 1.22.4. This latest is 1.29.

hi, thank you very mush for your reply, i run the command, result is: