Skip to content

feat: filter benign operational logs from Squid access.log#432

Merged
Mossaka merged 3 commits intomainfrom
copilot/filter-benign-logs-squid
Feb 4, 2026
Merged

feat: filter benign operational logs from Squid access.log#432
Mossaka merged 3 commits intomainfrom
copilot/filter-benign-logs-squid

Conversation

Copy link
Contributor

Copilot AI commented Jan 27, 2026

Squid's access.log contained operational noise from Docker healthcheck probes (::1/127.0.0.1) and shutdown-time keep-alive closures. Both appeared as error:transaction-end-before-headers with NONE_NONE:HIER_NONE status.

Changes

Squid config (src/squid-config.ts): Filter localhost healthchecks at log-write time (~65% of benign entries)

acl healthcheck_localhost src 127.0.0.1 ::1
log_access deny healthcheck_localhost

Log aggregator (src/logs/log-aggregator.ts): Skip error:transaction-end-before-headers entries during statistics computation

for (const entry of entries) {
  // Skip benign operational entries (connection closures without HTTP headers)
  if (entry.url === 'error:transaction-end-before-headers') {
    continue;
  }
  // ... process real requests
}

Raw logs retain all entries for debugging. Statistics reflect only actual HTTP/HTTPS traffic.

Original prompt

This section details on the original issue you should resolve

<issue_title>Filter benign operational logs from Squid access.log</issue_title>
<issue_description>## Problem

Squid's access.log contains benign operational entries that clutter the logs:

  1. Docker healthcheck probes (from ::1/127.0.0.1) - TCP connections that close immediately without sending HTTP headers
  2. Shutdown-time connection closures (from agent IP) - Keep-alive connections terminated when containers stop

These appear as error:transaction-end-before-headers with NONE_NONE:HIER_NONE status:

1769030982.251 ::1:33230 - -:- 0.0 - 0 NONE_NONE:HIER_NONE error:transaction-end-before-headers "-"
1769031025.056 172.30.0.20:49698 - -:- 0.0 - 0 NONE_NONE:HIER_NONE error:transaction-end-before-headers "-"

Proposed Solution

1. Filter localhost healthcheck probes at log time

Add to generated squid.conf in src/squid-config.ts:

# Don't log healthcheck probes from localhost
acl healthcheck_localhost src 127.0.0.1 ::1
log_access deny healthcheck_localhost

This filters ~65% of benign entries (healthcheck probes).

2. Filter remaining entries during log analysis

Update src/logs/log-aggregator.ts to ignore transaction-end-before-headers entries when aggregating statistics. This handles the remaining shutdown-time entries without losing them for debugging purposes.</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Add ACL and log_access directive to filter localhost healthcheck probes
- Update log aggregator to skip transaction-end-before-headers entries
- Add comprehensive tests for both changes

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Copilot AI changed the title [WIP] Filter benign operational logs from Squid access.log feat: filter benign operational logs from Squid access.log Jan 27, 2026
Copilot AI requested a review from Mossaka January 27, 2026 02:48
@Mossaka
Copy link
Collaborator

Mossaka commented Feb 3, 2026

@copilot please merge origin/main

@Mossaka Mossaka marked this pull request as ready for review February 4, 2026 17:06
Copilot AI review requested due to automatic review settings February 4, 2026 17:06
@Mossaka Mossaka merged commit 769a6f5 into main Feb 4, 2026
@Mossaka Mossaka deleted the copilot/filter-benign-logs-squid branch February 4, 2026 17:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR filters benign operational noise from Squid's access.log to improve log clarity and statistical accuracy. The implementation uses a two-pronged approach: filtering localhost healthcheck probes at log-write time via Squid configuration, and skipping connection closure errors during log analysis.

Changes:

  • Added Squid ACL configuration to prevent logging healthcheck probes from localhost (127.0.0.1 and ::1)
  • Modified log aggregator to skip error:transaction-end-before-headers entries when computing statistics
  • Added comprehensive test coverage for the filtering behavior

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/squid-config.ts Adds ACL and log_access directives to filter localhost healthcheck probes before writing to access.log
src/squid-config.test.ts Adds tests verifying the healthcheck filter configuration is present and correctly ordered
src/logs/log-aggregator.ts Updates aggregation logic to skip benign operational entries while preserving time range tracking
src/logs/log-aggregator.test.ts Adds comprehensive tests for filtering behavior including edge cases and time range handling

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI added a commit that referenced this pull request Feb 4, 2026
* Initial plan

* feat: filter benign operational logs from Squid access.log

- Add ACL and log_access directive to filter localhost healthcheck probes
- Update log aggregator to skip transaction-end-before-headers entries
- Add comprehensive tests for both changes

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Mossaka added a commit that referenced this pull request Feb 4, 2026
The `log_access` directive was removed in Squid 5.0. The ubuntu/squid:latest
Docker image uses Squid 5+ which does not recognize this directive, causing
Squid to fail to parse the config and crash with exit code 1.

This replaces the deprecated syntax:
```
acl healthcheck_localhost src 127.0.0.1 ::1
log_access deny healthcheck_localhost
access_log /var/log/squid/access.log firewall_detailed
```

With the modern Squid 5+ syntax that uses ACL filters on access_log:
```
acl healthcheck_localhost src 127.0.0.1 ::1
access_log /var/log/squid/access.log firewall_detailed !healthcheck_localhost
```

The `!` negates the ACL, meaning "log everything EXCEPT healthcheck_localhost".

Fixes CI breakage after PR #432.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Mossaka added a commit that referenced this pull request Feb 4, 2026
The `log_access` directive was removed in Squid 5.0. The ubuntu/squid:latest
Docker image uses Squid 5+ which does not recognize this directive, causing
Squid to fail to parse the config and crash with exit code 1.

This replaces the deprecated syntax:
```
acl healthcheck_localhost src 127.0.0.1 ::1
log_access deny healthcheck_localhost
access_log /var/log/squid/access.log firewall_detailed
```

With the modern Squid 5+ syntax that uses ACL filters on access_log:
```
acl healthcheck_localhost src 127.0.0.1 ::1
access_log /var/log/squid/access.log firewall_detailed !healthcheck_localhost
```

The `!` negates the ACL, meaning "log everything EXCEPT healthcheck_localhost".

Fixes CI breakage after PR #432.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Filter benign operational logs from Squid access.log

2 participants