community[minor]: add proxy support to RecursiveUrlLoader

[ccq1]

Description
This PR introduces the proxies parameter to the RecursiveUrlLoader class, allowing the user to specify proxy servers for requests. This update enables crawling through proxy servers, providing enhanced flexibility for network configurations.
The key changes include:
1.Added an optional proxies parameter to the constructor (init).
2.Updated the documentation to explain the proxies parameter usage with an example.
3.Modified the _get_child_links_recursive method to pass the proxies parameter to the requests.get function.

Sample Usage

from bs4 import BeautifulSoup as Soup
from langchain_community.document_loaders.recursive_url_loader import RecursiveUrlLoader

proxies = {
    "http": "http://localhost:1080",
    "https": "http://localhost:1080",
}
url = "https://python.langchain.com/docs/concepts/#langchain-expression-language-lcel"
loader = RecursiveUrlLoader(
    url=url, max_depth=1, extractor=lambda x: Soup(x, "html.parser").text,proxies=proxies
)
docs = loader.load()

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 16, 2024 3:06am

[eyurtsev]

Looks good, any chance you'd be willing to add a sentence to the security note so that folks know they can specify proxies?

[ccq1]

hi @eyurtsev , I just fixed a lint check error in my commit and added a note in the security document. Please check this commit.

[eyurtsev]

@ccq1 you can auto format like this:

cd langchain/libs/community
make format

you'll need to set up the environment with ruff (poetry install in that directory)

[ccq1]

@ccq1 you can auto format like this:

cd langchain/libs/community
make format

you'll need to set up the environment with ruff (poetry install in that directory)

@eyurtsev thanks, I autoformatted this commit using ruff. Please check. (I have the experience for the next PR.)