feat(transformer): support to expand lists from a remote url

[gfpcom]

for now, we only have one base64 transformer. And a new transformer list should be added to support to download source list from a remote url.

for example
https://github.com/snakem982/proxypool/blob/main/source/proxies.txt,list,

[devacius]

Hey @gfpcom Would love to contribute here can you help me with dev setup of the project couldn't find the steps in readme . Thanks

[gfpcom]

contribute

@devacius Thank you so much for your interest in contributing to our project!
please take a moment to review https://github.com/gfpcom/free-proxy-list/blob/main/CONTRIBUTING.md

[devacius]

Hey @gfpcom Thanks for the welcome. I have done the local setup and have read instructions about contribution . Can you help with the example url in the issue. Its not redirecting me to a list page . I am on the understanding that the transformer would take a remote url and then return the list of decoded strings back . Feel free to correct me .
So, It would be great if I can get the example url and a list of strings required for decoding it.

[gfpcom]

Hey @gfpcom Thanks for the welcome. I have done the local setup and have read instructions about contribution . Can you help with the example url in the issue. Its not redirecting me to a list page . I am on the understanding that the transformer would take a remote url and then return the list of decoded strings back . Feel free to correct me . So, It would be great if I can get the example url and a list of strings required for decoding it.

It is an example url (https://github.com/snakem982/proxypool/blob/main/source/proxies.txt).

There is a list of url instead of proxy list there. but our code is working on proxy list , not a proxy source list. that is what this issue is raised for.

http://141.147.161.50:12580/clash/proxies
http://155.248.172.106:12580/clash/proxies
http://66.42.50.118:12580/clash/proxies
https://clashe.eu.org/clash/proxies
https://proxy.crazygeeky.com/clash/proxies
https://proxypool.link/clash/proxies
https://proxypool1999.banyunxiaoxi.icu/clash/proxies

and check all other examples in ./sources.

[gfpcom]

get more source files from https://github.com/snakem982/proxypool/tree/main/source

[gfpcom]

@devacius #23 is a similar issue.

[devacius]

Hey @gfpcom is that the same issue and are we expecting same response /result .

[gfpcom]

@devacius maybe pipe methods should work better.
for #16 : https://github.com/snakem982/proxypool/blob/main/source/proxies.txt, read_lines | download | parse_clash, txt
for #23 : https://github.com/crossxx-labs/free-proxy, grep "https://github.com/crossxx-labs/free-proxy/*" | download | parse_clash, txt

how it works

• downloader will download content from 1st parameter, and send content to transformer (2nd parameter)
• read_lines | download | parse_clash: read lines from the content, download and decode them as clash yaml format, and merge all proxies into txt format
• grep "https://github.com/crossxx-labs/free-proxy/*" | download | parse_clash, txt: extract urls from the content, download and decode them as clash yaml format, and merge all proxies into txt format

[devacius]

Got it . Thanks that clear a lot of doubts .

[gfpcom]

@devacius please feel free to let me know if you have any question.