You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+7-5Lines changed: 7 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,8 @@ If you would like to test (this is optional and requires the installation with t
14
14
grunt test
15
15
```
16
16
17
+
*To use some features you’ll need to install [phantomjs](http://phantomjs.org/download.html), if you haven’t already*
18
+
17
19
# Getting started
18
20
19
21
Scraperjs exposes two different scrapers,
@@ -106,15 +108,14 @@ var scraperPromise = scraperjs.StaticScraper.create() // or DynamicScraper
106
108
The following promises can be made over it, they all return a scraper promise,
107
109
+```onStatusCode(code:number, callback:function(utils))```, executes the callback when the status code is equal to the code,
108
110
+```onStatusCode(callback:function(code:number, utils))```, executes the callback when receives the status code. The callback receives the current status code,
109
-
+```delay(time:number, callback:function(utils))```, delays the execution of the chain by time milliseconds,
110
-
+```timeout(time:number, callback:function(utils))```, executes the callback function after time milliseconds,
111
+
+```delay(time:number, callback:function(utils))```, delays the execution of the chain by time (in milliseconds),
112
+
+```timeout(time:number, callback:function(utils))```, executes the callback function after time (in milliseconds),
111
113
+```then(callback:function(utils))```, executes the callback after the last promise,
112
-
+```async(callback:function(done, utils))```, executes the callback, stopping the promise chain, resuming it when the ```done``` function is called,
113
-
+```onError(callback:function(utils))```, executes the callback when there was an error, errors block the execution of the chain even if the promise was not defined,
114
+
+```async(callback:function(done:function(result:?, err:?), utils))```, executes the callback, stopping the promise chain, resuming it when the ```done``` function is called. You can provide a result to be passed down the promise chain, or an error to trigger the onError,
115
+
+```onError(callback:function(error, utils))```, executes the callback when there was an error, errors block the execution of the chain even if the promise was not defined,
114
116
+```done(callback:function(utils))```, executes the callback at the end of the promise chain, this is always executed, even if there was an error,
115
117
+```get(url:string)```, makes a simple HTTP GET request to the url. This promise should be used only once per scraper.
116
118
+```request(options:Object)```, makes a (possibly) more complex HTTP request, scraperjs uses the [request](https://github.com/mikeal/request) module, and this method is a simple wrapper of ```request.request()```. This promise should be used only once per scraper.
117
-
+```use(ScraperPromise)```, uses a ScraperPromise already instantiated.
118
119
+```scrape(scrapeFn:function(...?), callback:function(result:?, utils)=, ...?)```, scrapes the page. It executes the scrapeFn and passes it's result to the callback. When using the StaticScraper, the scrapeFn receives a jQuery function that is used to scrape the page. When using the DynamicScraper, the scrapeFn doesn't receive nothing and can only return a [JSON-serializable](https://github.com/sgentle/phantomjs-node/wiki#evaluating-pages) type. Optionally an arbitrary number of arguments can be passed to the scraping function. A callback may not be provided, if so, the result of the scraping may be accessed with ``` utils.lastReturn ``` in the next promise.
119
120
120
121
All callback functions receive as their last parameter a utils object, with it the parameters of an url from a router can be accessed. Also the chain can be stopped.
@@ -172,6 +173,7 @@ The following promises can be made over it,
172
173
+```createStatic()```, associates a static scraper to use to scrape the matched page, this returns ScraperPromise, so any promise made from now on will be made over a ScraperPromise of a StaticScraper. Also the ```done``` promise of the scraper will not be available.
173
174
+```createDynamic()```, associates a dynamic scraper to use to scrape the matched page, this returns ScraperPromise, so any promise made from now on will be made over a ScraperPromise of a DynamicScraper. Also the ```done``` promise of the scraper will not be available.
174
175
+```route(url:string, callback:function(boolean))```, routes an url through all matched paths, calls the callback when it's executed, true is passed if the route was successful, false otherwise.
176
+
+```use(ScraperPromise)```, uses a ScraperPromise already instantiated.
175
177
+```otherwise(callback:function(url:string))```, executes the callback function if the routing url didn't match any path.
176
178
+```onError(callback:function(url:string, error:Error))```, executes the callback when an error occurred on the routing scope, not on any scraper, for that situations you should use the ```onError``` promise of the scraper.
0 commit comments