Apache2 Prometheus Exporter

A lightweight, bash-based Prometheus exporter for Apache HTTP server statistics via server-status. This exporter uses only bash, curl, and socat to provide comprehensive Apache metrics for monitoring with Prometheus and Grafana.

Features

• Pure Bash Implementation: No external dependencies except socat and curl
• Comprehensive Metrics: Exports all available Apache server-status statistics including:
  • Request rates, throughput, and total accesses
  • Worker status (busy, idle) and detailed worker states (11 different states)
  • Connection statistics (total, async writing, keep-alive, closing)
  • CPU metrics (load, user/system time, children processes)
  • System load averages (1m, 5m, 15m)
  • TLS session cache statistics (usage, operations, hits/misses)
  • Request duration and performance metrics
  • Server version and uptime information
  • Per-instance monitoring with labels
  • Calculated metrics like load percentage
• HTTP Server: Built-in HTTP server using socat for serving metrics
• Systemd Integration: Ready-to-use systemd service file
• Grafana Dashboard: Pre-built comprehensive dashboard
• Multi-Instance Support: Monitor multiple Apache servers from single exporter

Quick Start

Prerequisites

• Apache HTTP server with mod_status enabled
• socat package installed
• curl command available
• Prometheus server for scraping metrics

Basic Installation

1. Clone the repository:

git clone https://github.com/itefixnet/prometheus-apache2-exporter.git
cd prometheus-apache2-exporter

2. Test the exporter:

./apache2-exporter.sh test

3. Start the HTTP server:

./http-server.sh start

4. Access metrics at http://localhost:9117/metrics

System Installation

For production deployment, install as a system service:

# Create user and directories
sudo useradd -r -s /bin/false apache2-exporter
sudo mkdir -p /opt/apache2-exporter

# Copy files
sudo cp *.sh /opt/apache2-exporter/
sudo cp config.sh /opt/apache2-exporter/
sudo cp apache2-exporter.conf /opt/apache2-exporter/
sudo cp apache2-exporter.service /etc/systemd/system/

# Set permissions
sudo chown -R apache2-exporter:apache2-exporter /opt/apache2-exporter
sudo chmod +x /opt/apache2-exporter/*.sh

# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable apache2-exporter
sudo systemctl start apache2-exporter

Configuration

Environment Variables

The exporter can be configured using environment variables or configuration files:

Variable	Default	Description
APACHE_STATUS_URL	http://localhost/server-status?auto	Apache server-status URL
APACHE_TIMEOUT	10	Request timeout in seconds
LISTEN_PORT	9117	HTTP server port
LISTEN_ADDRESS	0.0.0.0	HTTP server bind address
METRICS_PREFIX	apache2	Prometheus metrics prefix
MAX_CONNECTIONS	10	Maximum concurrent HTTP connections
TIMEOUT	30	Request timeout in seconds
CURL_OPTS	-s --max-time	Additional curl options

Configuration Files

1. config.sh: Shell configuration file (sourced by scripts)
2. apache2-exporter.conf: Systemd environment file

Apache Configuration

Ensure Apache has mod_status enabled and server-status configured:

1. Enable mod_status:

sudo a2enmod status

2. Add to your Apache configuration (/etc/apache2/sites-available/000-default.conf or similar):

<Location "/server-status">
    SetHandler server-status
    Require local
    # For remote monitoring, add:
    # Require ip 192.168.1.0/24
</Location>

# Extended status provides additional metrics
ExtendedStatus On

3. Restart Apache:

sudo systemctl restart apache2

Metrics

The exporter provides comprehensive Apache HTTP server metrics:

Basic Statistics

• apache2_accesses_total - Total number of requests served (counter)
• apache2_sent_bytes_total - Total bytes sent (counter)
• apache2_uptime_seconds_total - Server uptime in seconds (counter)
• apache2_uptime_formatted{formatted="..."} - Human-readable server uptime with label
• apache2_requests_per_second - Current requests per second
• apache2_bytes_per_second - Current bytes per second
• apache2_bytes_per_request - Average bytes per request
• apache2_duration_per_request_microseconds - Average request duration in microseconds
• apache2_total_duration_microseconds - Cumulative duration of all requests in microseconds (counter)
• apache2_version_info{version="..."} - Apache server version information with label

Worker Statistics

• apache2_workers_busy - Number of busy workers
• apache2_workers_idle - Number of idle workers
• apache2_workers_total - Total number of workers
• apache2_load_percentage - Server load as percentage of busy workers

Connection Statistics

• apache2_connections_total - Total connections
• apache2_connections_async_writing - Connections in async writing state
• apache2_connections_async_keep_alive - Connections in async keep-alive state
• apache2_connections_async_closing - Connections in async closing state

Worker State Distribution

• apache2_worker_state_count{state="waiting_for_connection"} - Workers waiting for connection
• apache2_worker_state_count{state="starting_up"} - Workers starting up
• apache2_worker_state_count{state="reading_request"} - Workers reading requests
• apache2_worker_state_count{state="sending_reply"} - Workers sending replies
• apache2_worker_state_count{state="keepalive"} - Workers in keep-alive state
• apache2_worker_state_count{state="dns_lookup"} - Workers performing DNS lookups
• apache2_worker_state_count{state="closing_connection"} - Workers closing connections
• apache2_worker_state_count{state="logging"} - Workers logging requests
• apache2_worker_state_count{state="gracefully_finishing"} - Workers gracefully finishing
• apache2_worker_state_count{state="idle_cleanup"} - Workers in idle cleanup
• apache2_worker_state_count{state="open_slot"} - Open worker slots

CPU & Performance Metrics

• apache2_cpu_load - CPU load percentage
• apache2_cpu_user_seconds_total - Parent server CPU user time in seconds (counter)
• apache2_cpu_system_seconds_total - Parent server CPU system time in seconds (counter)
• apache2_cpu_children_user_seconds_total - Child server CPU user time in seconds (counter)
• apache2_cpu_children_system_seconds_total - Child server CPU system time in seconds (counter)

System Load Averages

• apache2_load_1 - System load average over 1 minute
• apache2_load_5 - System load average over 5 minutes
• apache2_load_15 - System load average over 15 minutes

TLS Session Cache Metrics

• apache2_tls_session_cache_info{type="..."} - TLS session cache configuration information
• apache2_tls_session_cache_shared_memory_bytes - TLS session cache shared memory size in bytes
• apache2_tls_session_cache_current_entries - TLS session cache current entries
• apache2_tls_session_cache_subcaches - TLS session cache subcaches
• apache2_tls_session_cache_indexes_per_subcache - TLS session cache indexes per subcache
• apache2_tls_session_cache_usage_percent - TLS session cache usage percentage
• apache2_tls_session_cache_index_usage_percent - TLS session cache index usage percentage
• apache2_tls_session_cache_time_left_oldest_avg_seconds - Average seconds before oldest TLS session expires
• apache2_tls_session_cache_time_left_oldest_min_seconds - Minimum seconds before oldest TLS session expires
• apache2_tls_session_cache_time_left_oldest_max_seconds - Maximum seconds before oldest TLS session expires
• apache2_tls_session_cache_store_total - Total TLS sessions stored (counter)
• apache2_tls_session_cache_replace_total - Total TLS sessions replaced (counter)
• apache2_tls_session_cache_expire_total - Total TLS sessions expired (counter)
• apache2_tls_session_cache_discard_total - Total TLS sessions discarded (counter)
• apache2_tls_session_cache_retrieve_hit_total - Total TLS session cache hits (counter)
• apache2_tls_session_cache_retrieve_miss_total - Total TLS session cache misses (counter)
• apache2_tls_session_cache_remove_hit_total - Total TLS session cache remove hits (counter)
• apache2_tls_session_cache_remove_miss_total - Total TLS session cache remove misses (counter)

Additional Metrics

• apache2_info{version="..."} - Apache server information with version label

Usage Examples

Manual Testing

# Test connection to Apache
./apache2-exporter.sh test

# Collect metrics once
./apache2-exporter.sh collect

# Start HTTP server manually
./http-server.sh start

# Test HTTP endpoints
curl http://localhost:9117/metrics
curl http://localhost:9117/health
curl http://localhost:9117/

Prometheus Configuration

Add jobs to your prometheus.yml for single or multiple Apache instances:

scrape_configs:
  # Single instance
  - job_name: 'apache2-exporter'
    static_configs:
      - targets: ['localhost:9117']
    scrape_interval: 30s
    metrics_path: /metrics
    
  # Multiple instances with labels
  - job_name: 'apache2-servers'
    static_configs:
      - targets: ['web1.example.com:9117', 'web2.example.com:9117']
        labels:
          environment: 'production'
          datacenter: 'dc1'
      - targets: ['staging.example.com:9117']
        labels:
          environment: 'staging' 
          datacenter: 'dc2'
    scrape_interval: 30s
    metrics_path: /metrics

Grafana Dashboard

Import the provided grafana-dashboard.json file into your Grafana instance:

1. Go to Dashboards → Import
2. Upload grafana-dashboard.json or copy/paste the JSON content
3. Configure Data Source: Select your Prometheus datasource from the dropdown
4. Click "Import"

Troubleshooting Dashboard Import:

• If you get "data source was not found" error, ensure your Prometheus datasource is properly configured in Grafana
• Make sure your Prometheus is scraping the Apache2 exporter endpoints
• Verify metrics are available by checking: http://your-grafana/explore → Select Prometheus → Query apache2_uptime_seconds_total

Dashboard Features: The comprehensive Grafana dashboard includes:

• Row 1 - Key Performance Indicators:

  • Request Rate (requests/minute) with 5-minute rate calculation
  • Server Load % (busy workers percentage)

• Row 2 - Throughput & Connections:

  • Throughput (Bytes/sec) with binary bytes unit
  • Connection Status (total, async writing, keep-alive, closing)

• Totals Row (collapsible):

  • Server Uptime by Instance with color-coded thresholds
  • Total Requests by Instance
  • Total Bytes Sent by Instance

• CPU Metrics Row (collapsible):

  • CPU Load % over time
  • CPU Usage Breakdown (user, system, children user/system)

• Worker Metrics Row (collapsible):

  • Worker Status (busy vs idle workers)
  • Worker States Distribution (stacked area chart)

• System Load Averages Row (collapsible):

  • Load 1 minute average
  • Load 5 minute average
  • Load 15 minute average

• TLS Session Row (collapsible):

  • TLS Session Cache Utilization (usage and index usage percentages)
  • TLS Session Cache Operations (store, retrieve hits/misses, replace, expire, discard, remove hits/misses)

All panels support:

• Multi-instance filtering with template variables
• Auto-refresh (30 second default)
• Time range selection
• Server name labels for easy identification
• Collapsed rows for better organization and performance

Multi-Instance Support: The dashboard includes template variables for monitoring multiple Apache instances:

• Instance: Filter by specific instance (e.g., web1.example.com:9117, localhost:9117)
• Job: Filter by Prometheus job name

To monitor multiple instances, configure your prometheus.yml:

scrape_configs:
  - job_name: 'apache2-web-servers'
    static_configs:
      - targets: 
        - 'web1.example.com:9117'
        - 'web2.example.com:9117'
        - 'web3.example.com:9117'

The dashboard supports:

• Multi-Instance Monitoring: Automatically discovers all Apache2 exporters
• Flexible Filtering: Filter by instance (host:port) and job name
• Multi-Select Variables: Monitor multiple instances simultaneously
• Instance Labeling: All metrics show which instance they come from

Dashboard features:

• Request rate monitoring per instance
• Worker status trends with instance breakdown
• Load percentage by server
• Connection state distribution
• CPU usage by Apache instance
• Throughput analysis per server
• Support for any host:port combination

Troubleshooting

Common Issues

1. Permission Denied:

   • Ensure scripts are executable: chmod +x *.sh
   • Check apache2-exporter user permissions

2. Cannot Connect to Apache:

   • Verify Apache is running: systemctl status apache2
   • Check mod_status is enabled: apache2ctl -M | grep status
   • Test server-status manually: curl http://localhost/server-status?auto
   • Verify server-status location configuration

3. Port Already in Use:

   • Change LISTEN_PORT in configuration
   • Check for other services: netstat -tlnp | grep 9117

4. Missing Dependencies:

   # Install socat (Ubuntu/Debian)
   sudo apt-get install socat curl
   
   # Install socat (CentOS/RHEL)
   sudo yum install socat curl

5. Invalid server-status Response:

   • Ensure ExtendedStatus On is set in Apache configuration
   • Check that ?auto parameter is included in status URL
   • Verify Apache access permissions for server-status

Logging

• Service logs: journalctl -u apache2-exporter -f
• Manual logs: Scripts output to stderr

Performance Tuning

For high-traffic web servers:

• Increase MAX_CONNECTIONS
• Adjust TIMEOUT value
• Monitor system resources
• Consider running multiple exporter instances
• Tune Apache server-status access frequency

Authentication

If your server-status requires authentication, configure in config.sh:

export APACHE_USER="your-username"
export APACHE_PASSWORD="your-password"
export APACHE_AUTH_TYPE="basic"  # or "digest"

Development

Testing

# Run basic tests
./apache2-exporter.sh test
./http-server.sh test

# Test with different configurations
APACHE_STATUS_URL=http://example.com/server-status?auto ./apache2-exporter.sh test

Contributing

1. Fork the repository
2. Create a feature branch
3. Test thoroughly
4. Submit a pull request

License

This project is licensed under the BSD 2-Clause License - see the LICENSE file for details.

Support

• GitHub Issues: https://github.com/itefixnet/prometheus-apache2-exporter/issues
• Documentation: This README and inline script comments

Alternatives

This exporter focuses on simplicity and minimal dependencies. For more advanced features, consider:

• apache_exporter (Go-based)
• Custom telegraf configurations
• Native Prometheus monitoring solutions