The Enhanced Web Security Scanner is a comprehensive, automated tool designed to assess web application security through in-depth crawling, endpoint discovery, and vulnerability detection. It combines several powerful security tools into a unified, easy-to-use bash script that systematically identifies potential security risks in web applications.
- Comprehensive Crawling: Leverages Katana for efficient web content discovery
- JavaScript Analysis: Extracts and analyzes JavaScript files for security vulnerabilities
- API Endpoint Detection: Identifies potential API endpoints and tests them for vulnerabilities
- Parameter Testing: Optional fuzzing of URL parameters to detect injection flaws
- Multiple Discovery Methods: Combined approach using direct crawling, historical data (Wayback), and other sources
- Vulnerability Classification: Automatic categorization of findings by severity (Critical, High, Medium, Low)
- Interactive HTML Reports: Generates comprehensive, easy-to-navigate HTML reports
- Parallel Processing: Optimized performance with configurable concurrent operations
- Robust Error Handling: Graceful recovery from failures with detailed logging
- Flexible Configuration: Customizable settings via configuration file or command-line parameters
- Bash (version 4.0+)
- curl
- grep
- awk
- sed
- jq
- md5sum
- openssl
- Nuclei - Vulnerability scanner
- Katana - Web crawling framework
- waybackurls - Fetch URLs from the Wayback Machine
- gau - Get All URLs
- httpx - HTTP toolkit
- ffuf - Fast web fuzzer
- Download the script:
curl -O https://raw.githubusercontent.com/yourusername/security-scanner/main/webscan.sh- Make it executable:
chmod +x webscan.sh- Run the script:
./webscan.shThe script will check for required dependencies and guide you through installing any missing components.
For Debian/Ubuntu:
apt update
apt install -y curl grep gawk sed jq opensslFor CentOS/RHEL:
yum install -y curl grep gawk sed jq opensslFor macOS (using Homebrew):
brew install curl grep gawk gnu-sed jq opensslIf you have Go installed:
# Install Nuclei
go install github.com/projectdiscovery/nuclei/v2/cmd/nuclei@latest
# Install Katana
go install github.com/projectdiscovery/katana/cmd/katana@latest
# Install waybackurls
go install github.com/tomnomnom/waybackurls@latest
# Install gau
go install github.com/lc/gau/v2/cmd/gau@latest
# Install httpx
go install github.com/projectdiscovery/httpx/cmd/httpx@latest
# Install ffuf
go install github.com/ffuf/ffuf@latestAdd Go bin directory to your PATH:
echo 'export PATH=$PATH:$(go env GOPATH)/bin' >> ~/.bashrc
source ~/.bashrcScan a website:
./webscan.sh https://example.comThe script will prompt for input if no URL is provided:
./webscan.sh
# It will then prompt: Enter target URL:The script accepts various command-line options:
./webscan.sh [options] [URL]
Options:
-h, --help Show this help message
-v, --verbose Increase verbosity (can be used multiple times)
-t, --threads NUMBER Set maximum threads (default: 8)
-d, --depth NUMBER Set crawl depth (default: 3)
-o, --output DIRECTORY Set output directory
-c, --config FILE Use custom config file
--timeout SECONDS Set scan timeout (default: 600)
--download-timeout SECONDS Set download timeout (default: 30)
--enable-wayback Enable Wayback Machine URL discovery
--enable-gau Enable GAU URL discovery
--custom-templates DIR Use custom Nuclei templatesThe scanner creates a structured output directory containing:
[domain]_[timestamp]/
├── js_files/ # JavaScript and JSON files
│ ├── js_urls.txt # List of discovered JS files
│ ├── json_urls.txt # List of discovered JSON files
│ └── downloaded/ # Downloaded files for analysis
├── api_endpoints/ # API-related discoveries
│ ├── api_urls.txt # Potential API endpoints
│ ├── param_urls.txt # URLs with parameters
│ ├── active_endpoints.txt # Verified active endpoints
│ └── ...
├── reports/ # Analysis reports
│ ├── critical_vulns.txt # Critical vulnerabilities
│ ├── high_vulns.txt # High-severity vulnerabilities
│ ├── medium_vulns.txt # Medium-severity vulnerabilities
│ ├── low_vulns.txt # Low-severity vulnerabilities
│ ├── report.html # Interactive HTML report
│ ├── scan_summary.json # JSON summary of findings
│ └── ...
├── crawl_results.txt # All discovered URLs
├── scan_[timestamp].log # Detailed log file
└── scan_state.json # Scan state information
The script uses a configuration file located at ~/.webscan_config. This file is automatically created on first run with default values:
# Web Security Scanner Configuration
MAX_THREADS=8
DOWNLOAD_TIMEOUT=30
SCAN_TIMEOUT=600
CRAWL_DEPTH=3
OUTPUT_DIR="./scan_results"
USER_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
VERBOSE=1
ENABLE_WAYBACK=false
ENABLE_GAU=false
CUSTOM_NUCLEI_TEMPLATES=""You can edit this file to customize default behavior.
The HTML report (reports/report.html) provides a comprehensive overview of scan results:
- Executive Summary: Key statistics and vulnerability counts
- Vulnerability Details: Collapsible sections for each severity level
- Scan Details: Configuration and scan parameters
- Interactive Elements: Expandable sections for detailed information
Findings are classified into four severity levels:
- Critical: Severe vulnerabilities requiring immediate attention (RCE, SQLi, etc.)
- High: Significant security issues with high potential impact
- Medium: Moderate security issues that should be addressed
- Low: Minor security concerns with limited impact
The reports/scan_summary.json file contains structured data about the scan:
{
"scan_id": "abcd1234",
"target": "https://example.com",
"timestamp": "2023-05-01T12:34:56Z",
"duration": "00:15:30",
"stats": {
"total_urls": 250,
"js_files": 45,
"json_files": 12,
"api_endpoints": 18,
"downloaded_files": 57,
"analyzed_files": 57,
"vulnerabilities": {
"total": 8,
"critical": 1,
"high": 2,
"medium": 3,
"low": 2
}
}
}You can use custom Nuclei templates for specialized scanning:
./webscan.sh --custom-templates /path/to/templates https://example.comOr set in config:
echo 'CUSTOM_NUCLEI_TEMPLATES="/path/to/templates"' >> ~/.webscan_configEnable Wayback Machine and/or GAU for historical URL discovery:
./webscan.sh --enable-wayback --enable-gau https://example.comThe script can automatically test URL parameters for common vulnerabilities if ffuf is installed:
# This functionality is automatically used when ffuf is available
# No special flags neededIf the script reports missing tools, install them as described in the Installation section.
chmod +x webscan.sh- Ensure you have active internet access
- Check if the target website is accessible
- Try increasing timeouts in the configuration file
- Reduce MAX_THREADS in the configuration file
- Decrease CRAWL_DEPTH for faster scanning
- Use a more focused target URL with a specific path
Detailed logs are stored in the scan output directory:
cat [domain]_[timestamp]/scan_[timestamp].logIncrease verbosity for more detailed logs:
./webscan.sh --verbose --verbose https://example.com- Always ensure you have proper authorization before scanning any website
- This tool is designed for security professionals and system administrators
- Unauthorized security scanning may be illegal in many jurisdictions
- Use responsibly and ethically
Contributions are welcome! To contribute:
- Fork the repository
- Create a feature branch:
git checkout -b new-feature - Commit your changes:
git commit -am 'Add new feature' - Push to the branch:
git push origin new-feature - Submit a pull request
This project is licensed under the MIT License - see the LICENSE file for details.
- The ProjectDiscovery team for Nuclei and Katana
- Tom Hudson for waybackurls
- Corben Leo for gau
- The ffuf team for the fast web fuzzer
- All open-source security tools that made this project possible
For questions, feedback, or issues, please open an issue on the GitHub repository.
Disclaimer: This tool is provided for educational and legitimate security testing purposes only. The authors are not responsible for any misuse or damage caused by this program. Always ensure you have proper authorization before conducting security testing.