Feature: Extract valid URLs

[reneleonhardt]

URL validation is already amazing, thank you!
Would it be possible to search and return multiple valid URLs in an input string, so the caller doesn't need to know where a valid url started and ended?

input_string = """
First: https://io.io (valid)
Second: http://tooshort.c (invalid)
Third and fourth: https://python.org and https://pypi.org (valid)
"""  # Extractable: ['https://io.io', 'https://python.org', 'https://pypi.org']

[yozachar]

Oh my! I thought I replied. I'm so sorry.

It is certainly possible, but out of scope for this package.

A simple method would be to iterate through the string looking for http or https followed by a set of characters until a whitespace. Then pass that to the validators library.

[reneleonhardt]

No stress, everyone has a life 😅

Okay, here is my workaround in case one of your users needs to extract valid links out of a text block separated by whitespace:

links = set(filter(validators.url, re.compile(r'(?:^|\W)(https?://\S+)').findall(input_string)))

My first contribution for a useful validators.get_urls() function would be:

def get_urls(links: Iterable[str] | str, collection: Type[list | set] = list):
    """
    >>> validators_get_urls("https://b.io https://a.io https://b.io")
    ['https://b.io', 'https://a.io', 'https://b.io']
    >>> validators_get_urls("https://b.io https://a.io https://b.io", set) == {'https://a.io', 'https://b.io'}
    True
    >>> sorted(validators_get_urls("https://b.io https://a.io https://b.io", set))
    ['https://a.io', 'https://b.io']
    """
    if isinstance(links, str):
        return collection(filter(validators.url, re.compile(r"(?:^|\W)(https?://\S+)").findall(links)))
    return collection(filter(validators.url, links))

Then I would refactor this by adding and basing it on a generator version, so dicts can be collected for example.

[yozachar]

This is darn good.