Labyrinth

Have you ever crawled a site? You probably have.

Now try to crawl this site. It should take some time :)

Installing

• Use rvm to use Ruby 2.1.0
• Run gem install bundler
• Run bundler Possibly from the rvm install dir
• Run rake:setup
• Run rake:migrate
• Run rackup

TODO

• Add logging for requested urls
• Use something not SQLite for logging This is real!
• Cache the wordlist to give the disc some peace