The definitive analysis for your proxy

⭐ Give me one star — it will motivate me to keep improving it!!

Table of contents

1. About the Project
   • Goals of this project
2. Getting Started
   • Prerequisites
   • Installation Script
   • Installation Manual
   • Testing Information
   • Uninstall SquidStats
   • Reset forgotten admin password
3. To do
4. Contributing
5. License
6. Contact
7. Technologies Used
8. Special thanks

About the Project

A modern tool for parsing and analyzing Squid logs, providing a sleek and user-friendly dashboard to visualize real-time connection data. This project helps network administrators monitor and manage Squid proxy connections effectively.

You can view the example here.

Goals of this project :

• Real-time Log Parsing: Parses active Squid connections and displays detailed information.
• User Monitoring: Identifies connections by username, URI, and log type.
• Metrics Overview:
  • Total read and written data for each connection.
  • Number of requests per connection.
  • Delay pool usage.
• Interactive Dashboard: Clean interface for easy data interpretation.
• Squid Cache Statistics:
  • Stored entries.
  • Used and free capacity
  • Maximum and current cache size
  • Disk space and inode usage
  • Age of cached objects
• Logs Users:
  • User activity monitoring 👥
  • Beautiful visualizations 📊
  • Advanced filtering & search 🔍
  • Paginated results 📄
• Top Graphs:
  • Top 20 Users Activity
  • Top 20 Users Data Usage
  • Total Users
  • Total Transmitted Data
  • Total Request
  • And More...
• And More

Getting Started

The entire installation process in more detail can be found here.

Prerequisites

• Python 3.10+
• Squid proxy server

apt install git python3 python3-pip python3-venv libmariadb-dev curl

Note: If your Squid proxy does not use user authentication (i.e., you do not use login or password for clients), you can keep the default log format that comes with Squid. The detailed format below is only required for full compatibility with user-based reports.

• ⚠️ !!Important (Only work before version 7.1) ⚠️ For compatibility with user logs, use this format in /etc/squid/squid.conf:

  logformat detailed \
  "%ts.%03tu %>a %ui %un [%tl] \"%rm %ru HTTP/%rv\" %>Hs %<st %rm %ru %>a %mt %<a %<rm %Ss/%Sh %<st

  access_log /var/log/squid/access.log detailed

🔧 Cache Manager Configuration (Critical for SquidStats)

SquidStats requires proper Cache Manager configuration to function correctly. Please follow the official Squid Cache Manager documentation for complete setup.

Essential Configuration Steps:

1. Configure Cache Manager Access Controls in /etc/squid/squid.conf:

   # Allow localhost access to cache manager
   acl manager proto cache_object
   acl localhost src 127.0.0.1/32 ::1
   
   # Basic cache manager access
   http_access allow localhost manager
   http_access deny manager

2. Configure Remote Access (if SquidStats runs on different server):

   # Replace 192.168.1.100 with your SquidStats server IP
   acl managerAdmin src 192.168.1.100
   http_access allow managerAdmin manager

3. Verify Cache Manager is Working:

   # Test cache manager access
   curl http://127.0.0.1:3128/squid-internal-mgr/menu
   
   # Or using squidclient
   squidclient -h 127.0.0.1 -p 3128 mgr:info

Why This Configuration is Critical:

• Real-time Statistics: SquidStats uses Cache Manager to retrieve live proxy statistics, active connections, and performance metrics
• Cache Information: Provides data about cached objects, memory usage, disk utilization, and hit ratios
• Connection Monitoring: Enables monitoring of active client connections and bandwidth usage
• Administrative Functions: Allows SquidStats to perform cache management operations like cache refresh and statistics retrieval

⚠️ Security Note: The Cache Manager provides sensitive information about your proxy server. Always restrict access to trusted IP addresses and consider using authentication for production environments.

Common Issues Without Proper Configuration:

• Empty or missing statistics in SquidStats dashboard
• "Connection refused" errors when accessing cache data
• Missing real-time connection information
• Incomplete bandwidth and user activity reports

Installation Script

1. Get Script With curl o wget:

 wget https://github.com/kaelthasmanu/SquidStats/releases/download/1.0/install.sh

2. Add permission execution:

 sudo chmod +x install.sh

3. Execute the script:

 sudo ./install.sh

Installation Options

The installer now supports multiple modes:

# Full installation (interactive)
sudo ./install.sh

# Update existing installation
sudo ./install.sh --update

# Configure blacklist only
sudo ./install.sh --configure-blacklist

# Show help
./install.sh --help

🛡️ Blacklist Configuration

During installation, you can configure which domains notify like block:

• Option 1: Use default blacklist (facebook.com, twitter.com, instagram.com, etc.)
• Option 2: Configure custom domains
• Option 3: Skip blacklist configuration

Example custom configuration:

# When prompted, enter domains separated by commas:
facebook.com,twitter.com,youtube.com,netflix.com,tiktok.com

Installation Manual

1. Clone the repository:

   mkdir -p /opt/squidstats
   git clone https://github.com/kaelthasmanu/SquidStats.git /opt/squidstats

2. Creating a virtual environment (venv):

 cd /opt/squidstats/
 python3 -m venv "/opt/squidstats/venv"
 source venv/bin/activate
 pip install --upgrade pip

3. Install requeriments python with pip or pip3:

  pip install -r /opt/squidstats/requirements.txt

4. Create a .env file in the project root and add the following content:
   Note: for use MARIADB need your own database running

   VERSION=2
   SECRET_KEY="your-secret-key-here"  # Generate with: python3 -c 'import secrets; print(secrets.token_hex(32))'
   SQUID_HOST="127.0.0.1"
   SQUID_PORT=3128
   LOG_FORMAT=DETAILED
   FLASK_DEBUG=True
   DATABASE_TYPE="SQLITE"
   SQUID_LOG="/var/log/squid/access.log"
   SQUID_CACHE_LOG="/var/log/squid/cache.log"
   DATABASE_STRING_CONNECTION="/opt/SquidStats/"
   REFRESH_INTERVAL=60
   BLACKLIST_DOMAINS="facebook.com,twitter.com,instagram.com,tiktok.com,youtube.com,netflix.com"
   HTTP_PROXY=""
   SQUID_CONFIG_PATH=/home/manuel/Desktop/config/squid.conf
   ACL_FILES_DIR=/home/manuel/Desktop/config/acls
   LISTEN_HOST=127.0.0.1
   LISTEN_PORT=8080
   FIRST_PASSWORD="mipassword"

Forwarding Squid logs from a remote Squid host (important)

If you install SquidStats on a different machine than the Squid proxy (i.e. Squid is not local on the SquidStats server), you must ensure that the Squid access log (/var/log/squid/access.log) is available on the SquidStats server. The recommended and reliable way to do this is to forward Squid's log entries via syslog (rsyslog or syslog-ng) from the proxy host to the SquidStats host.

Below are example configurations for both sending (on the Squid proxy host) and receiving (on the SquidStats server). Adjust IP addresses, ports and file paths to your environment.

NOTE: forwarding logs avoids copying files and ensures live updates for the dashboard.

• Using rsyslog (proxy = 192.168.1.10, squidstats = 192.168.1.20)

  On the Squid proxy (send): edit /etc/rsyslog.d/30-squid.conf and add:

  # rsyslog - forward squid access log lines to remote host
  # create a local file watch (imfile) to read Squid access.log
  module(load="imfile" PollingInterval="10")
  input(type="imfile"
        File="/var/log/squid/access.log"
        Tag="squid_access:"
        Severity="info"
        Facility="local0")
  
  # forward to remote syslog collector
  *.* @@192.168.1.20:514
  

  On the SquidStats server (receive): enable UDP/TCP syslog reception and store to file. Example /etc/rsyslog.d/10-remote.conf:

  # rsyslog - accept remote logs
  module(load="imudp")
  input(type="imudp" port="514")
  
  # store squid forwarded logs
  if $programname == 'squid_access' or $syslogtag contains 'squid_access' then {
      /var/log/remote/squid/access.log
      stop
  }
  

  Create the directory and set permissions:

  sudo mkdir -p /var/log/remote/squid
  sudo chown syslog:adm /var/log/remote/squid
  sudo systemctl restart rsyslog

• Using syslog-ng (proxy send):

  Edit /etc/syslog-ng/conf.d/squid-forward.conf on the proxy host:

  source s_squid { file("/var/log/squid/access.log" follow-freq(1)); };
  destination d_remote { tcp("192.168.1.20" port(514)); };
  log { source(s_squid); destination(d_remote); };
  

  On the SquidStats server (receive) with syslog-ng, accept and write to disk:

  source s_network { tcp(port(514)); }; 
  destination d_squid { file("/var/log/remote/squid/access.log"); }; 
  filter f_squid { program("squid_access") or match("^\d+\.\d+\.\d+\.\d+ .*squid" type("regexp")); };
  log { source(s_network); filter(f_squid); destination(d_squid); }; 
  

• Firewall and permissions

  • Open port 514 (or the port you choose) on the SquidStats server and allow the proxy host to connect.
  • Use TCP if you care about delivery guarantees; UDP is faster but lossy.
  • Ensure the syslog daemon user can write to the destination directory (/var/log/remote/squid).

• Update SQUID_HOST in .env

  • If you forward logs, set SQUID_HOST in the SquidStats .env to the IP of the original Squid host or keep it as 127.0.0.1 if SquidStats will process them as local files (i.e., they are written under /var/log/remote/squid/access.log locally).

Examples & notes

• This approach is preferred over mounting remote filesystems or using scp/rsync cron jobs because it provides near real-time updates and is resilient to log rotation when configured correctly.

5. Run App with python or python3 🚀:

  python3 app.py

6. With your preferred browser, visit the installation URL:

  http://ip/hostname:5000

Accessing the Admin Panel

To access the admin panel, you need to configure the initial password and restart the service.

1. Edit the .env file using your preferred editor (vi, vim, or nano). Here's an example using nano:

   nano .env

   Add or modify the following line in the .env file:

   FIRST_PASSWORD="mipassword"

   Save the file and exit the editor (in nano, press Ctrl+X, then Y, then Enter).

2. Restart the SquidStats service:

   sudo systemctl restart squidstats.service

3. Access the admin panel by visiting http://your-ip:your-port/admin in your browser.

4. Log in with the username admin and the password you set in FIRST_PASSWORD (in this example, "mipassword").

Resetting forgotten password (localhost only)

If you lose the admin password, you can reset it from the server where SquidStats runs (localhost only):

1. Make sure the app is running (default listens on 127.0.0.1:5000, or the LISTEN_PORT you set in .env).
2. Run this from the SquidStats host:

curl -X POST http://127.0.0.1:5000/auth/reset-password \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "new_password": "NewSecurePass123"}'

Notes:

• Must be executed on localhost; remote calls are rejected.
• new_password must be at least 8 characters.
• You can also send form data if preferred:

  curl -X POST http://127.0.0.1:5000/auth/reset-password \
    -d "username=admin" \
    -d "new_password=NewSecurePass123"

⚠️ First Execution Alert ⚠️

Warning: 🚨 The first execution may cause high CPU usage.

Update project(web) with Script

1. Get Script With curl o wget:

 wget https://github.com/kaelthasmanu/SquidStats/releases/download/1.0/install.sh

2. Add permission execution:

 sudo chmod +x install.sh

3. Execute the script with parameters update:

 sudo ./install.sh --update

🕒 Run on System Startup To ensure the application starts automatically when the system boot :

1. Copy file service:

  cp ./utils/squidstats.service /etc/systemd/system/squidstats.service

2. Restart daemons:

  systemctl daemon-reload

3. Enable service:

  systemctl enable squidstats.service

3. Start service:

  systemctl start squidstats.service

Testing Information

This software has been thoroughly tested and is compatible with Squid version 6.12 in Ubuntu 24.04 and Debian12. Please ensure your Squid installation matches this version or newer for optimal performance.

Uninstall SquidStats

The installation script now includes a complete uninstall option that cleans all system components:

Using the Uninstall Script

1. Get the installation script (if you don't have it):

wget https://github.com/kaelthasmanu/SquidStats/releases/download/1.0/install.sh
chmod +x install.sh

2. Run the uninstallation:

sudo ./install.sh --uninstall

What Does the Uninstall Do?

The COMPLETE uninstallation performs the following actions:

• 🛑 Stops and disables the systemd service squidstats.service
• 🗑️ Removes files from the /opt/squidstats directory
• ⚠️ Requires confirmation from user before proceeding

Manual Uninstallation

If you prefer to uninstall manually:

# Stop and disable service
sudo systemctl stop squidstats.service
sudo systemctl disable squidstats.service
sudo rm -f /etc/systemd/system/squidstats.service
sudo systemctl daemon-reload

# Remove project files
sudo rm -rf /opt/squidstats

⚠️ Important Note about Uninstallation

• Uninstallation will permanently delete all SquidStats data and configurations
• It's recommended to backup any custom configurations before uninstalling

Upcoming Features

• Interface Improvements 🎨

  • Report export (PDF, Excel)

• Security Features 🔒

  • User authentication
  • System audit logs
  • Automated alerts

• Optimization and Performance ⚡

  • Enhanced data caching
  • Historical log compression
  • Multi-proxy support

Contributing

1. Fork the repository:

   git checkout -b feature-name

   2.Create a new branch for your feature or fix:

git checkout -b feature-name

3.Commit your changes and push the branch:

git push origin feature-name

4.Open a pull request.

License

The SquidStats project is released under the MIT license.

Contact

Need help? Feel free to contact me 📨 manuelalberto.gorrin@gmail.com

Related project: (CuCuota)

Technologies Used

Backend: Python, Flask
Frontend: HTML, CSS, JavaScript

Special thanks

• To the Cuban Sysadmins community.
• To everyone who gave me his help when I had doubts.