First version -beta

.dockerignore

@@ -0,0 +1,6 @@
+node_modules
+npm-debug.log
+Dockerfile
+.dockerignore
+.git
+.gitignore

.gitignore

@@ -0,0 +1,116 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+lerna-debug.log*
+
+# Diagnostic reports (https://nodejs.org/api/report.html)
+report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
+
+# Runtime data
+pids
+*.pid
+*.seed
+*.pid.lock
+
+# Directory for instrumented libs generated by jscoverage/JSCover
+lib-cov
+
+# Coverage directory used by tools like istanbul
+coverage
+*.lcov
+
+# nyc test coverage
+.nyc_output
+
+# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# Bower dependency directory (https://bower.io/)
+bower_components
+
+# node-waf configuration
+.lock-wscript
+
+# Compiled binary addons (https://nodejs.org/api/addons.html)
+build/Release
+
+# Dependency directories
+node_modules/
+jspm_packages/
+
+# Snowpack dependency directory (https://snowpack.dev/)
+web_modules/
+
+# TypeScript cache
+*.tsbuildinfo
+
+# Optional npm cache directory
+.npm
+
+# Optional eslint cache
+.eslintcache
+
+# Microbundle cache
+.rpt2_cache/
+.rts2_cache_cjs/
+.rts2_cache_es/
+.rts2_cache_umd/
+
+# Optional REPL history
+.node_repl_history
+
+# Output of 'npm pack'
+*.tgz
+
+# Yarn Integrity file
+.yarn-integrity
+
+# dotenv environment variables file
+.env
+.env.test
+
+# parcel-bundler cache (https://parceljs.org/)
+.cache
+.parcel-cache
+
+# Next.js build output
+.next
+out
+
+# Nuxt.js build / generate output
+.nuxt
+dist
+
+# Gatsby files
+.cache/
+# Comment in the public line in if your project uses Gatsby and not Next.js
+# https://nextjs.org/blog/next-9-1#public-directory-support
+# public
+
+# vuepress build output
+.vuepress/dist
+
+# Serverless directories
+.serverless/
+
+# FuseBox cache
+.fusebox/
+
+# DynamoDB Local files
+.dynamodb/
+
+# TernJS port file
+.tern-port
+
+# Stores VSCode versions used for testing VSCode extensions
+.vscode-test
+
+# yarn v2
+.yarn/cache
+.yarn/unplugged
+.yarn/build-state.yml
+.yarn/install-state.gz
+.pnp.*

Dockerfile

@@ -0,0 +1,20 @@
+FROM node:10-jessie
+
+RUN apt-get update -y && \
+    apt-get install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget libgbm1 && \
+    apt-get clean && rm -rf /tmp/* /var/lib/apt/lists/* /var/tmp/*
+RUN mkdir -p /home/node/flaresolverr && chown -R node:node /home/node/flaresolverr
+WORKDIR /home/node/flaresolverr
+
+COPY package*.json ./
+USER node
+RUN npm install
+COPY --chown=node:node . .
+
+ENV LOG_LEVEL=info
+ENV LOG_HTML=false
+ENV PORT=8191
+ENV HOST=0.0.0.0
+
+EXPOSE 8191
+CMD [ "node", "index.js" ]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Diego Heras (ngosang)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,2 +1,113 @@
-# FlareSolverr
-Proxy server to bypass Cloudflare protection
+## FlareSolverr
+
+Proxy server to bypass Cloudflare protection
+
+:warning: This project is in beta state. Some things may not work and the API can change at any time.
+See the known issues section.
+
+### How it works
+
+FlareSolverr starts a proxy server and it waits for user requests in idle state using few resources.
+When some request arrives, it uses [puppeteer](https://github.com/puppeteer/puppeteer) with the
+[stealth plugin](https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth)
+to create an headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare
+challenge is solved (or timeout). The HTML code and the cookies are sent back to the user and those cookies can
+be used to bypass Cloudflare using other HTTP clients.
+
+### Installation
+
+It requires NodeJS.
+
+Run `npm install` to install FlareSolverr dependencies.
+
+### Usage
+
+Run `node index.js` to start FlareSolverr.
+
+Example request:
+```bash
+curl -L -X POST 'http://localhost:8191/v1' \
+-H 'Content-Type: application/json' \
+--data-raw '{
+	"url":"http://www.google.com/",
+	"userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0"
+}'
+```
+Parameter | Notes
+|--|--|
+url | Mandatory
+userAgent | Optional. Will be used by the headless browser
+
+Example response:
+```json
+{
+  "status": "ok",
+  "message": "",
+  "startTimestamp": 1591679463498,
+  "endTimestamp": 1591679472781,
+  "version": "1.0.0",
+  "solution": {
+    "url": "https://www.google.com/?gws_rd=ssl",
+    "response": "<!DOCTYPE html><html ...",
+    "cookies": [
+      {
+        "name": "ANID",
+        "value": "AHWqTUnRRMcmD0SxIOLAhv88SiY555FZpb4jeYCaSNZPHtYyBuY85AmaZEqLFTHe",
+        "domain": ".google.com",
+        "path": "/",
+        "expires": 1625375465.915947,
+        "size": 68,
+        "httpOnly": true,
+        "secure": true,
+        "session": false,
+        "sameSite": "None"
+      },
+      {
+        "name": "1P_JAR",
+        "value": "2020-6-9-5",
+        "domain": ".google.com",
+        "path": "/",
+        "expires": 1594271465,
+        "size": 16,
+        "httpOnly": false,
+        "secure": true,
+        "session": false
+      }
+    ],
+    "userAgent": " Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0"
+  }
+}
+```
+
+#### Environment variables
+
+To set the environment vars in Linux run `export LOG_LEVEL=debug` and then start FlareSolverr in the same shell.
+
+Name | Default value
+|--|--|
+LOG_LEVEL | info
+LOG_HTML | false
+PORT | 8191
+HOST | 0.0.0.0
+
+### Docker
+
+You can edit environment variables in `./Dockerfile` and build your own image.
+
+```bash
+docker build -t flaresolverr:latest .
+docker run --restart=always --name flaresolverr -p 8191:8191 -d flaresolverr:latest
+```
+
+### Known issues / Roadmap
+
+The current implementation is not able to bypass Cloudflare because they are detecting the headless browser.
+I hope this will be fixed soon in the [puppeteer stealth plugin](https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth)
+
+TODO:
+* Fix remaining issues in the code (see TODOs)
+* Make the maxTimeout configurable by the user
+* Add support for more HTTP methods (POST, PUT, DELETE ...)
+* Add support for user HTTP headers
+* Hide sensitive information in logs 
+* Reduce Docker image size