Writing WhatWeb Plugins

Note to New Contributors

Please read these guidelines before writing a plugin. All contributions are welcomed.

1. Check for your target application in the ./plugins/ directory.
2. Check out Writing WhatWeb Plugins for some plugin writing tips.
3. Test your plugin(s) well against a wide range of targets. Google and ShodanHQ are useful resources to find live targets for testing.
4. Submit your new plugins via email or forking the project or via the Issues page.

Getting Started

To get started modify plugin-template.rb.txt in the ./my-plugins/ folder. Rename it to the name of the application you will be fingerprinting. WhatWeb automatically loads plugins in the ./my-plugins/ directory.

Check out the How to develop WhatWeb plugins tutorial for an in depth guide to plugin development.

Typical Plugin (Example)

A typical plugin looks like this:

	Plugin.define "Plone" do
	author "Andrew Horton"
	version "0.2"
	description "An open-source content management system (CMS) written in Python."
	website "http://plone.org/"

	dorks [
	'"powered by plone"'
	]

	matches [
	{:name=>"meta generator tag",
	:regexp=>/<meta name="generator" content="[^>]*http:\/\/plone.org" \/>/},

	{:name=>"plone css",
	:regexp=>/(@import url|text\/css)[^>]*portal_css\/.*plone.*css(\)|")/}, #"

	{:name=>"plone javascript",
	:regexp=>/src="[^"]*ploneScripts[0-9]+.js"/}, #"

	{:text=>'<div class="visualIcon contenttype-plone-site">'},

	{:name=>"div tag, visual-portal-wrapper",
	:certainty=>75,
	:text=>'<div id="visual-portal-wrapper">'},
	]

	def passive
		m=[]
		#X-Caching-Rule-Id: plone-content-types
		#X-Cache-Rule: plone-content-types
		m << {:name=>"X-Caching-Rule-Id: plone-content-types" } if @meta["x-caching-rule-id"] =~ /plone-content-types/i
		m << {:name=>"X-Cache-Rule: plone-content-types" } if @meta["x-cache-rule"] =~ /plone-content-types/i
		m
	end


	end

Matches

There are 3 levels to a plugin. Simple matches, passive and aggressive tests. You don’t need to know ruby to write plugins with simple matches. Passive and aggressive tests are written in ruby.

The matches [] array contains a set of ways to match a website to a system.

The methods are:

* :text			Matches text within the webpage ( case sensitive )
* :regexp		A regular expression. Append `/i` for case insensitive matches
* :ghdb			Like a google query ( supports `intitle`, `inurl`, and `filetype` )
* :md5			MD5 hash of the HTML page
* :tagpattern		Pattern of HTML tags
* :url			The URL has to match this. Used for passive and aggressive testing
* :status		Matches HTTP status code. 
* :name			This is the name of the match, and is optional.
* :certainty		Optional, defaults to 100. Values are `maybe (25)`, `probably (75)` and `certain (100)`.

* :version		As a string or number this is a version returned when other methods match
* :version		As a regular expression, this extracts the version information from the HTML. Also requires :regexp_offset=>0
* :model		As a string or number this is a device model returned when other methods match
* :model		As a regular expression, this extracts the device model information from the HTML. Also requires :regexp_offset=>0
* :module		As a string or number this is a web application module returned when other methods match
* :module		As a regular expression, this extracts the web application module information from the HTML. Also requires :regexp_offset=>0
* :firmware		As a string or number this is a device firmware version returned when other methods match
* :firmware		As a regular expression, this extracts the device firmware version information from the HTML. Also requires :regexp_offset=>0
* :filepath		As a string or number this is a file path returned when other methods match
* :filepath		As a regular expression, this extracts the file path information from the HTML. Also requires :regexp_offset=>0
* :account		As a string or number this is a user credential returned when other methods match
* :account		As a regular expression, this extracts the user credential information from the HTML. Also requires :regexp_offset=>0

Each plugin can access @body, @meta, @status, @base_uri, @md5, @ip and @tagpattern variables.

Passive tests add matches to the m array. Each match is a hash containing the name of the match, probability and more. The entire hash is returned with Full output. Brief output returns just the match and any associated information, such as :version, :string, :modules, :accounts, etc

To discover the regular expressions to match against, wget about 20-30 examples into the ./tests/ folder. Be aware that some software can have dramatic variations between versions.

Generating a signature with ease

Cookies

If the software uses unique cookies (such as Set-Cookie: PastVisitor=1; expires=Wed, 02-Jun-2010 04:05:29 GMT; path=/) then you can search for more instances of the software on ShodanHQ.

# A typical match would look like:
m << { :name=>"PastVisitor Cookie" } if @meta["set-cookie"] =~ /PastVisitor=[0-9]+.*/

Note the value in @meta is always lowercase.

HTTP Headers

If the software uses unique HTTP headers (such as Server: BarracudaHTTP or X-Caching-Rule-Id: plone-content-types) then you can search for more instances of the software on ShodanHQ.

# A typical match would look like:
m << {:name=>"X-Caching-Rule-Id: plone-content-types" } if @meta["x-caching-rule-id"] =~ /plone-content-types/i

Note the value in @meta is always lowercase.

URL Pattern

If the software uses a unique URL pattern (such as inurl:web.config filetype:config "ConnectionString") then you can search for more instances of the software on Google using inurl:

# A typical match would look like:
{ :ghdb=>'inurl:web.config filetype:config "ConnectionString"' },

Note the GHDB queries are case insensitive, as a Google query is.

Unique Phrases

If the software uses a unique phrase (such as Powered by ABC Software) then you can search for more instances of the software on Google using intext: or intitle:

# A typical meta generator match looks like :
{ :regexp=>/<meta name="generator" content="[^>]*http:\/\/plone.org" \/>/ }

# A typical HTML div id match with a certainty of 75 looks like :
{ :certainty=>75, :text=>'<div id="visual-portal-wrapper">' }

Note the text queries are case sensitive.

HTML Tag Pattern

You can generate a page URL pattern easily by using ./plugin-development/get-pattern script:

 $ [[./plugin-development/get-pattern|http://github.com/urbanadventurer/WhatWeb/blob/master/plugin-development/get-pattern]] http://www.google.com/

Although it's not foolproof, we can use this approach as a final pattern. This is often useful for redirect, intro or frameset pages when there's little else to indentify the application.

GHDB

If you port a GHDB match, use :ghdb. GHDB matches support the intitle, inurl, and filetype Google codes. For example:

# A typical GHDB match looks like :

# http://johnny.ihackstuff.com/ghdb?function=detail&id=1840
{ :ghdb=>'"Powered by Vsns Lemon" intitle:"Vsns Lemon"' }

It's often worth rewriting GHDB matches with regular expressions to avoid false positives, especially if they require inurl:

Note the GHDB queries are case insensitive, as a Google query is.

Resources

• Search ShodanHQ for example software with customer HTTP headers (server, set-cookie, x-powered-by, etc).
• Search Google for unique phrases, titles or URL structures.
• Check for demonstration or showcase websites hosted by the vendor.

Tips

More than one copy of WhatWeb

If you've installed WhatWeb and you're using a second copy of WhatWeb to develop or edit plugins then you'll need to change the name of the plugin in Plugin.define or else the plugin output will be overwritten by the output from the installed plugin.

Regex matches

When you're matching HTML with regex make sure you match the entire HTML element. For example:

For the href parameter:

/<a[\s]+href=['"]?http:\/\/this\/is\/a\/unique\/path\/to\/fingerprint\/['"]?[^>]*>/i

For the title parameter:

/<a[\s]+title=['"]?this is a unique title to fingerprint['"]?[^>]*>/i

Passive Matches

Passive matches in the def passive section require the same regex in both the @body regular expression and the @body.scan regular expression. For example:

	# Passive #
	def passive
		m=[]

		# Version Detection # Powered by text
		if @body =~ /powered by ABC software version ([\d\.]+)/i
			m << { :version=>@body.scan(/powered by ABC software version ([\d\.]+)/i) }
		end

		# Return passive matches
		m
	end

Note that if the two regular expressions differ then the plugin will fail.

Regular Expressions

lrn2regexp - Learn it, Live it, Love it http://www.rubular.com/