EndlessFractal/hosts

Retrieves a list of domain names from various sources and filters them to create a consolidated list of unique domains. The filtered list is then saved to a file called hosts.txt.

Description

The script performs the following steps:

1. Reads from a list of URLs from which to retrieve domain lists.
2. Sends HTTP requests to each URL and retrieves the response text.
3. Parses the response text, filters out empty lines and comments, and extracts the domain names.
4. Stores the unique domain names in a set data structure.
5. Sorts the domain names in alphabetical order.
6. Retrieves the current date and time as the last update timestamp.
7. Counts the number of unique domains.
8. Constructs a header section with the last update timestamp and the number of unique domains.
9. Writes the header and the filtered domain names to the hosts.txt file.

Sources

The script retrieves domain lists from the following sources:

• Ad Wars Hosts
• AdguardDNS
• Admiral
• Anudeep's Blacklist
• BarbBlock Hosts File
• Easylist
• Easyprivacy
• First-Party Trackers Hosts
• Mandiant APT1 Report Appendix D
• NoTrack Malware Blocklist
• NoTrack Tracker Blocklist
• Referrer Spam Blacklist
• Spam404 Main Blacklist
• Stalkerware Indicators
• Threat Intel Latest Domains
• VeleSila YHosts
• W3KBL
• 💊 Dandelion Sprout's Anti-Malware List

File Output

The filtered domain names are saved to the hosts.txt file, which is formatted as follows:

# Title: EndlessFractal/hosts 

# Date of Last Update: [Last Update Timestamp]
# Number of Unique Domains: [Number of Unique Domains]

# Fetch the latest version of this file: [Raw File URL]
# Project home page: [Project Repository URL]

# ===============================================================

[Filtered Domain 1]
[Filtered Domain 2]
[Filtered Domain 3]
...

The file includes a header section with the last update timestamp and the number of unique domains. It also provides links to fetch the latest version of the file and the project home page.