Query log filtering very slow on underpowered devices such as routers

[timkgh]

Prerequisites

• I have checked the Wiki and Discussions and found no answer

• I have searched other issues and found no duplicates

• I want to report a bug and not ask a question

Operating system type

Linux, Other (please mention the version in the description)

CPU architecture

64-bit MIPS

Installation

GitHub releases or script from README

Setup

On a router, DHCP is handled by the router

AdGuard Home version

v0.107.21

Description

Filtering the query log on an EdgeRouter ER-12 is very slow and causes very high CPU use. This means I can't really keep more than 7 days of logs and have a usable UI.

[ghost]

Hi there, are you still experiencing this?

[timkgh]

Hi there, are you still experiencing this?

Yes, nothing changed, still slow. Can you please reopen?

[overwatch3560]

What do you mean with very slow?

[timkgh]

What do you mean with very slow?

It takes minutes to get any results when I click on the Dashboard > Blocked malware/phishing
E.g. this URL: https://192.168.1.1/#logs?response_status=blocked_safebrowsing

[ainar-g]

This is likely just an inevitable consequence of having to filter through a lot of data fetched from the disk. See #2290.

You can enable verbose logging to see the progress of the fetching and filtering. It is likely that it's just inherently slow on that hardware, and in that case the recommendation would be to limit the size of the querylog or, if RAM allows, keep it in memory.

[Cebeerre]

What do you mean with very slow?

It takes minutes to get any results when I click on the Dashboard > Blocked malware/phishing E.g. this URL: https://192.168.1.1/#logs?response_status=blocked_safebrowsing

Assuming you're using OpenWRT on this Uniquiti router, are you using the squash image so all the "/data" AGH directory is sitting on the ramdisk ?

[timkgh]

I'm using EdgeOS, not OpenWRT, but all the AGH logs are in /tmp (tmpfs) mainly to no wear the flash. Keeping only 14 days of logs (7 days rotation), ~125MB and it's still very slow.