Discovery Plugins Tutorial

Why Discovery Plugins?

Collecting data requires crawling lots of websites. No matter how much you automate you still have to validate. Discovery Plugins provide a consistent data schema that provide enough flexibility to make them easy to create, validate, and distribute. The source code is open source so you can inspect the source code to see how it works. Discovery Plugins are a domain specific template.

Discovery Plugins can perform the following actions:

1. Open a tab to a specific website pre-populating any required or optional parameter values, cookie values, form values, or headers, and authentication tokens.
2. Request entity extraction or record enrichment using internal or external web services

Anatomy of a Discovery Plugin:

1. Basic -> The minimum set of values that need to be setup for a Discovery Plugin to function
2. Content Types -> Sets the HTTP Header Content-Type along with the HTTP Method to use.
3. Authentication -> Lets you easily populate common HTTP headers that are used for authenticating against web services
4. Field Mappings -> Map data from Your Rapport into a format that the Discovery Plugin service can understand
5. Header Mappings -> Set advanced HTTP Header values
6. Support -> Use this section to define support resources (e.g., URL or contact) and select the country in which this Discovery Plugin should be used.
7. Description -> A description of what the discovery plugin does, helpful for providing some context around what the Discovery Plugin does.