Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Captcha Support #104

Merged
merged 22 commits into from
Nov 12, 2018
Merged

Add Captcha Support #104

merged 22 commits into from
Nov 12, 2018

Conversation

pr1sm
Copy link
Collaborator

@pr1sm pr1sm commented Nov 10, 2018

Changes

  • Add Captcha storage and management to the TaskManager
  • Add New Events for loading/unloading captchas
    • Captcha windows that are open are automatically allocated to solve captchas when they are needed by tasks
  • Connect to Captcha Storage in TaskManager from frontend
  • Open Captcha Window automatically if captcha is needed and no window is already open

Checks

  • Unit tests pass (CI)
  • coverage is stable (delta < 2%)
  • Tasks automatically trigger a captcha when needed
    • Run a task, or use the debug command when in dev env
      • Open 1 or more captcha windows
      • From the main windows' devtool console, run: window.Bridge.sendDebugCmd('testStartHarvest') (or load/run a task manually)
      • All open windows should load a captcha to harvest
      • Verify tokens are harvested (check electron console output) or view harvested tokens (window.Bridge.sendDebugCmd('viewHarvestedFrontendTokens'))
    • When no captcha windows are open, run a task or use the debug command when in dev env
      • Repeat the same steps as before, but don't open windows in the beginning
  • Multi-Task Support
    • When running multiple tasks, the captcha tokens should get passed to all tasks evenly

fixes #65

@pr1sm pr1sm added type:enhancement New feature or request area:frontend Related to Nebula's Frontend Electron app area:task-runner Related to Nebula's Task Runner package labels Nov 10, 2018
@pr1sm pr1sm requested a review from walmat November 11, 2018 00:40
@pr1sm
Copy link
Collaborator Author

pr1sm commented Nov 11, 2018

@walmat I think this PR is complete feature wise -- ready for your review. If you could test out the checks in the OP and let me know if there are any errors, that would be great too!

@pr1sm pr1sm changed the title [WIP] Add Captcha Support Add Captcha Support Nov 11, 2018
Copy link
Owner

@walmat walmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. Just wow. Excellent work man! Just some small touch ups. Most of them were your own TODO tags, so if you're planning on doing them after this PR, just ignore those!

Super good work as usual 👍

frontend/public/js/captcha.js Outdated Show resolved Hide resolved
frontend/public/js/captcha.js Outdated Show resolved Hide resolved
frontend/public/js/captcha.js Outdated Show resolved Hide resolved
frontend/lib/_electron/taskManagerWrapper.js Show resolved Hide resolved
frontend/lib/_electron/taskManagerWrapper.js Show resolved Hide resolved
frontend/lib/_electron/windowManager.js Outdated Show resolved Hide resolved
frontend/lib/_electron/windowManager.js Outdated Show resolved Hide resolved
frontend/lib/_electron/windowManager.js Outdated Show resolved Hide resolved
frontend/lib/_electron/windowManager.js Show resolved Hide resolved
frontend/public/js/captcha.js Outdated Show resolved Hide resolved
walmat and others added 18 commits November 12, 2018 03:04
This commit adds a handler that gets called when task captcha events are emitted from the TaskManager. These events are related to starting/stopping harvesting tokens for a specific task. These events will call a method on the window manager to handle showing/spawning captcha windows.

The specific event handler body/parameters are subject to change as the task manager development is still in progress.
This commit adds stubs to the window manager for handling the starting/stopping of captcha harvesting. These methods will be implemented next

Windows might need to get allocated for a specific task -- Stubs are added for that case. If this isn't needed, they can be removed.
This commit adds more progress to connect the start/stop harvest events to the captcha window. The harvest events flow from the window manager to the captcha window managers to the captcha windows themselves. Handlers were added to listen for the correct events and properly load the captcha when it is necessary.
This commit adds a new Captcha Server Manager that starts when the captcha windows are opened. This is necessary for the captcha to render properly.
This commit updates the captcha server manager to dynamically choose a root dir for what to server based on the environment. This prevents the need to rebuild the frontend every time a change is made to captcha html/js
This commit adjusts the captcha html to add the js into a separate file. This simplifies the html file, and isolates all js to a single place.
This commit changes the source of the sitekey, so it can be added dynamically if needed. A default sitekey is used if no key is given.
This commit adds a hook for harvesting captcha tokens from the captcha windows for the task manager wrapper. This will allow a direct handler to link the captcha token back to the task manager.
This commit adds a new queue data structure that allows synchronous inserts, but asynchronous returns. This will be used as the structure for the captcha queues in TaskManager.
This commit updates the task manager captcha handling to use AsyncQueues for handling the register/deregister captcha functions. This allows the register call to act like a promise even though updates are triggered through event handlers.

The constants map was updated to include events for the TaskManager. This is used to standardize event channels surrounding captcha events.
This commit adjusts the implementation of the TaskRunner and Checkout to connect captcha related code to the new TaskManager captcha methods.
This commit updates the frontend's TaskManagerWrapper to correctly bind/call the right events to link harvesting captchas.
This commit fixes the flow for checkout connections now that the captcha request has been implemented.
This commit updates the WindowManager to automatically open a new captcha window if it receives a harvest request and no captcha windows are already open. This means that even if the user forgets to open the window, they will still be able to complete the captcha request.
This commit updates the task manager wrapper to handle a new debug command to view all the harvested tokens. The preload now listens to and prints out responses of debug events.
This commit transitions the task manager to keep a queue of reserved spaces for captcha tokens. This is under the assumption that currently all captchas use the same sitekey. Instead of allocating windows to specific runners. The tokens come in to the task manager and are assigned on a first come, first serve basis to running queues. This allows the tokens to be spread evenly among running tasks.
This commit fixes a failing unit tests due to adding blends as a supported site.
@pr1sm
Copy link
Collaborator Author

pr1sm commented Nov 12, 2018

Rebased onto master

@pr1sm pr1sm requested a review from walmat November 12, 2018 09:09
Copy link
Owner

@walmat walmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Merge 👍

@pr1sm
Copy link
Collaborator Author

pr1sm commented Nov 12, 2018

merging...

@pr1sm pr1sm merged commit 78cf1b4 into master Nov 12, 2018
@pr1sm pr1sm deleted the issue_65 branch November 12, 2018 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:frontend Related to Nebula's Frontend Electron app area:task-runner Related to Nebula's Task Runner package type:enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ACTION] – Harvest Google RECAPTCHA
2 participants