PageGen is a multithreaded web crawler, which will start from a given keyword, and will then grab related pages starting from Google, up to a crawl dept specified by the user. After fetching the pages, it will re-arrange its contents, for example by applying Markov chains. This software can be used for generating search engine content.
Entry point of this program is here