Introduce SeleniumBrowser #1733
Conversation
Tests for the new Selenium WebDriver addition
Inclusions of `SeleniumBrowserWrapper`, `SeleniumBrowser`, and several required helper functions that are part of the upcoming `ContentCollector` class and the `WebCollectionAgent`.
Provides an optional drop-in replacement for `SimpleTextBrowser` with `SeleniumBrowserWrapper` for use-cases including pages that depend on JavaScript and others that prevent calls from `requests`. Nearly all compatibility is held through with the exception of page numbering.
The ContentAgent class is a custom Autogen agent that can be used to collect and store online content from different web pages. It extends the ConversableAgent class and provides additional functionality for managing a list of additional links, storing collected content in local directories, and customizing request headers.
ContentAgent uses deque to manage a list of additional links for further exploration, with a maximum depth limit set by max_depth parameter. The collected content is stored in the specified storage path (storage_path) using local directories.
ContentAgent can be customized with request_kwargs and llm_config parameters during instantiation. The default User-Agent header is used for requests, but it can be overridden by providing a new dictionary of headers under request_kwargs.
Very minor updates prior to submitting a PR
small fix in the `fix_missing_protocol` function
Small addition to maintain a dictionary of processed html content, referenced by the source URL (Uniform Resource Locator)
We cover a small sample of websites, asserting expectations against a number of measurements performed on the collected content. The assertions include, but are not limited to: - the expected variables contain values - the presence of the expected output files - that the expected output files are not empty Further improvements can include: - evaluation against all choices of WebDriver to confirm functionality - evaluation against a larger sample of websites -
It's noted that `_set_page_content`, `_split_pages`, and `viewport` are likely not yet compatible but seemingly not necessary at this time for the selenium browser wrapper class.
Small updates on imports that have been recently refactored to other locations. Specifically: ``` from ..agent import Agent from .. import ConversableAgent, AssistantAgent, UserProxyAgent, GroupChatManager, GroupChat from ...oai.client import OpenAIWrapper ```
A small change to declaring `self.browser_kwargs` prior to initializing the parent class (ConversableAgent). This is done to avoid triggering an unexpected argument error for `browser_kwargs`.
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1733 +/- ##
==========================================
+ Coverage 37.05% 41.64% +4.59%
==========================================
Files 62 63 +1
Lines 6499 7096 +597
Branches 1438 1675 +237
==========================================
+ Hits 2408 2955 +547
- Misses 3898 3905 +7
- Partials 193 236 +43
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
fixing the following pre-commit errors: autogen/agentchat/contrib/content_agent.py:21:1: E402 Module level import not at top of file autogen/agentchat/contrib/content_agent.py:34:1: E402 Module level import not at top of file autogen/agentchat/contrib/content_agent.py:65:33: F811 Redefinition of unused `deque` from line 6 autogen/agentchat/contrib/content_agent.py:374:26: F811 Redefinition of unused `filename` from line 7
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
signalprime
had a problem deploying
to
openai1
GitHub Actions
Failure
|
Made some last updates based on the generous feedback from @skzhang1. The only two other things I can think to include would be an update to the docs for the graphical WebSurferAgent and additional tests to @sonichi thank you for running the OAI tests. I noticed |
signalprime
changed the title
There was a problem hiding this comment.
I'd like to get @afourney 's opinion on the structure of the changes, e.g., whether it's ok to directly modify the current web surfer agent or should a separate agent be created. And whether a separate optional dependency is desired. My other comments may not apply if a structural change is needed.
Hi @afourney, I saw in an issue that you were thinking to pass a browser object to the agent. Would you prefer that to the method I have configured? I could expose an option to provide a browser object while maintaining the current behavior where the browser type is specified in the config to keep things simple for the avg end-user. Your thoughts? |
signalprime
changed the title
signalprime
changed the title
change _set_page_content to set_page_content
change _set_page_content to set_page_content
Removing the exception messages related to Selenium
Minor fix to permit testing
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 10404662 | Triggered | Generic CLI Secret | 841ed31 | .github/workflows/dotnet-release.yml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
|
@signalprime i would like you to take a look at my PR maybe we can adopt one design as I’m working on a similar part. |
|
This PR is against AutoGen 0.2. AutoGen 0.2 has been moved to the 0.2 branch. Please rebase your PR on the 0.2 branch or update it to work with the new AutoGen 0.4 that is now in main. |
rysweet
added
the
awaiting-op-response
|
@signalprime unfortunately after rebase there are still some open conflicts. If you are still interested in bringing this one forward please see if you can get those resolved and green the latest CI. |
|
closing as stale, please reopen if you would like to update |
Key Contributions:
SeleniumBrowser Function:
Adds a headless, fully-functional desktop web driver to enable dynamic interactions with web pages, crucial for accessing content reliant on JavaScript or other client-side scripts.
SeleniumBrowserWrapper Class:
Offers a seamless alternative to the
SimpleTextBrowser, enhancing agent capabilities in interacting with web pages that require advanced browsing functionalities.WebSurferAgent Enhancements:
This update enables the ability to select between
SimpleTextBrowserorSeleniumBrowserWrapperbased on the providedbrowser_configand opens the door to vision-based function calling and interactions, significantly broadening the scope of tasks and potential use cases for Autogen agents in web interaction scenarios going forward.Unit Testing:
Extends unit tests to cover the enhanced
WebSurferAgent, ensuring both functionality and reliability in the agents' operations.Notebooks:
Added
agentchat_surfer_edge.ipynbto demonstrate cross compatibility and new graphical functionalityBoth notebooks specify
gpt-3.5-turboand rely on round-trip generations without follow-ons.Benefits
Types of changes
Checks
WebArchiverAgenthave passed.WebSurferAgenthave passed.