New DNS API for WEDOS provider

[mxtuma]

Hello
please merge new DNS API for Czech DNS provider WEDOS. I had tested it, seems everything works flawlessly. I do the spellcheck as suggested.

I'm going to edit WIKI page about DNS APIs.

Thanks for your project, really helps me.
Michal

[mxtuma]

Wow, let's celebrate, my code passed all the checks! After all.

[mxtuma]

OK, now I had saved the changes to Wiki page and prepared issue for bug reporting. Hopefully now it is acceptable for merge.

[Neilpang]

can you please update to latest dev branch, and check the CI for the workflow DNS?
https://github.com/acmesh-official/acme.sh/wiki/DNS-API-Test

[mxtuma]

Hi Neil,
I'm little bit in trouble with passing DNS tests/CI Workflow. Used API of WEDOS provider is secured by IP adresses from which you can manage their DNS entries. Do you know if it is possible to somehow setup single fixed IP adress for test? Then I would be able to add it to my wedos account and rerun the tests.

Otherwise, there is no option for me to pass the DNS tests.
Thanks for helping,
Michal

[Neilpang]

https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#ip-addresses-of-github-hosted-runners

Microsoft updates the Azure IP address ranges weekly in a JSON file that you can download from the Azure IP Ranges and Service Tags - Public Cloud website. You can use this range of IP addresses if you require an allow-list to prevent unauthorized access to your internal resources.

[Neilpang]

please add the ip range to your account first.

And delete the ip range after the test passes.

[mxtuma]

I see your point, but that is impossible. I had checked the IP adress ranges and that is so many adresses (about 12 hundreds of IP ranges) that is imposible to use WEDOS simple management page to allow so many of them. I ask WEDOS directly if they can help me somehow in order to pass the test.

Thanks for the documentation.

[mxtuma]

Thanks for the revisions you made. I will manage your requirements into the code at home when I finish my work.

[s482dcaw]

This needs more testing. Here is some feedbadk

1. wedos requires the hour to be in Prague timezone.
   The line hour=$(date +%H) needs to be replaced by hour=$(TZ=Europe/Prague date +%H)
2. the greps need to be fixed, or replaced with _egrep_o
3. the DNS entry is added to the first domain returned by wedos, instead of the requested domain in the command line. Ideally instead of dns-domains-list and trying to find out the domain, one should issue a domain-info or a dns-rows-list and check if the domains returns anything
4. for some reason, the line added is _acme-challeng instead of _acme-challenge

[jkalousek]

I would just like to bump this a little as it seems that it is really close to being implemented. @mxtuma did you get response from Wedos about helping you with passing the test?

[mxtuma]

Yes, I get stuck with implementation. I asked WEDOS to help, someone respond to me but after I put him particular actions to be performed, he gets silent. Not sure what to do. Also my account to wedos was cancelled because our company change the DNS provider, so I'm not able to perform tests anymore. Maybe I can cancel this pull request, and after I the WEDOS person will help me, I can finish with implementation. What do you think?

[jkalousek]

Yes, I get stuck with implementation. I asked WEDOS to help, someone respond to me but after I put him particular actions to be performed, he gets silent. Not sure what to do. Also my account to wedos was cancelled because our company change the DNS provider, so I'm not able to perform tests anymore. Maybe I can cancel this pull request, and after I the WEDOS person will help me, I can finish with implementation. What do you think?

I do have avalible wedos acount wich I could provide api key for testing (if wedos is not able to provide their api key for testing and want to use real acount) or as alternative I could implement changes in your code and try to talk to Wedos but as you alredy started and actually written the code I think it would be much faster if you could finish it.

[mxtuma]

Yes, I get stuck with implementation. I asked WEDOS to help, someone respond to me but after I put him particular actions to be performed, he gets silent. Not sure what to do. Also my account to wedos was cancelled because our company change the DNS provider, so I'm not able to perform tests anymore. Maybe I can cancel this pull request, and after I the WEDOS person will help me, I can finish with implementation. What do you think?

I do have avalible wedos acount wich I could provide api key for testing (if wedos is not able to provide their api key for testing and want to use real acount) or as alternative I could implement changes in your code and try to talk to Wedos but as you alredy started and actually written the code I think it would be much faster if you could finish it.

Sounds great. OK please review WIKI page, to see how to allow WAPI. Then please provide me username and WAPI pass. Then you will have to setup IPv4 adn v6 adresses of my server, from which I will perform testing. Then I finish implementation and prepare for the final testing at github cloud servers. What do you think about that?

[jkalousek]

Yes, I get stuck with implementation. I asked WEDOS to help, someone respond to me but after I put him particular actions to be performed, he gets silent. Not sure what to do. Also my account to wedos was cancelled because our company change the DNS provider, so I'm not able to perform tests anymore. Maybe I can cancel this pull request, and after I the WEDOS person will help me, I can finish with implementation. What do you think?

I do have avalible wedos acount wich I could provide api key for testing (if wedos is not able to provide their api key for testing and want to use real acount) or as alternative I could implement changes in your code and try to talk to Wedos but as you alredy started and actually written the code I think it would be much faster if you could finish it.

Sounds great. OK please review WIKI page, to see how to allow WAPI. Then please provide me username and WAPI pass. Then you will have to setup IPv4 adn v6 adresses of my server, from which I will perform testing. Then I finish implementation and prepare for the final testing at github cloud servers. What do you think about that?

Yep, no problem I alredy have WAPI active. Please send me email to REDACTED with IP address that you want me to allow and I will send you back credentials for WAPI.

[mxtuma]

This needs more testing. Here is some feedbadk

1. wedos requires the hour to be in Prague timezone.
   The line hour=$(date +%H) needs to be replaced by hour=$(TZ=Europe/Prague date +%H)
2. the greps need to be fixed, or replaced with _egrep_o
3. the DNS entry is added to the first domain returned by wedos, instead of the requested domain in the command line. Ideally instead of dns-domains-list and trying to find out the domain, one should issue a domain-info or a dns-rows-list and check if the domains returns anything
4. for some reason, the line added is _acme-challeng instead of _acme-challenge

Hello, thank you for your feedback. Currently it seems I get un-stuck with access to other WEDOS domain and it seems I will finish the implementation this/next week. For sure I will implement your recommendations but one of them is not clear to me. Please can you tell me, how did you meant it. Its regarding 3rd point.

I do understand that it is better to query domain, but the issue might be when you need to validate any subdomain. Did you mean that it would be better to start parsing input $fulldomain by dots from the right and always tries to verify with domain-info request if it is a primary domain?

For example $fulldomain=_acme_challenge.sub.domain.from.cz, first you would try from.cz as prime domain and if found as primary quit with $_domain and $_sub_domain set? If not try domain.from.cz ...

This solution seems to me much easier to implement as extracting from XML listings is a pain, even more if WEDOS change formatting of output XML (new lines). Maybe what you have written is right as I performed testing only with account with single domain. I will focus on that.

Last point, can you somehow help me with pushing on WEDOS to allow this DNS API pass github tests from AZURE cloud mentioned in this thread? Thanks indeed,
Michal

[jkalousek]

@mxtuma I supose that we could always use Self-hosted runners to have control over source IP but it seems like that is little bit overkill for such a simple task...

[s482dcaw]

@mxtuma I tried and it's working with minor changes.

1. What it says, it is a minor change
2. I replaced the greps and sha1sum and it works as expected
3. the problem is not with the dns-domains-list, but with how you parse it. In the end, it's the first domain that always matches (and nr 4 is caused by 3).

After all these changes (1 and 2), I only needed to do that to make it work:

Old:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | grep -o -E "( ).( )primary<
/type>( )active" | grep -o -E ".*"); do
_debug "Active and primary XML DOMAIN found: ${xml_domain}"
end_of_name=$((${#xml_domain} - 7))
xml_domain_name=$(echo "${xml_domain}" | cut -c 7-${end_of_name})

New:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | _egrep_o ".</data>"); do
xml_domain_name=$(echo $xml_domain | sed -n 's/.(.)</name>./\1/p');

The difference is in parsing the response; I tested and it understands (and modifies) the right domain / hostname and lesson learnt, patch doesn't post on github

[mxtuma]

@mxtuma I tried and it's working with minor changes.

1. What it says, it is a minor change
2. I replaced the greps and sha1sum and it works as expected
3. the problem is not with the dns-domains-list, but with how you parse it. In the end, it's the first domain that always matches (and nr 4 is caused by 3).

After all these changes (1 and 2), I only needed to do that to make it work:

Old:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | grep -o -E "( ).( )_primary< /type>( )_active" | grep -o -E ".*"); do
_debug "Active and primary XML DOMAIN found: ${xml_domain}"
end_of_name=$((${#xml_domain} - 7))
xml_domain_name=$(echo "${xml_domain}" | cut -c 7-${end_of_name})

New:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | egrep_o "."); do xml_domain_name=$(echo $xml_domain | sed -n 's/.(.)._/\1/p');

The difference is in parsing the response; I tested and it understands (and modifies) the right domain / hostname and lesson learnt, patch doesn't post on github

/.

@mxtuma I tried and it's working with minor changes.

1. What it says, it is a minor change
2. I replaced the greps and sha1sum and it works as expected
3. the problem is not with the dns-domains-list, but with how you parse it. In the end, it's the first domain that always matches (and nr 4 is caused by 3).

After all these changes (1 and 2), I only needed to do that to make it work:

Old:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | grep -o -E "( ).( )_primary< /type>( )_active" | grep -o -E ".*"); do
_debug "Active and primary XML DOMAIN found: ${xml_domain}"
end_of_name=$((${#xml_domain} - 7))
xml_domain_name=$(echo "${xml_domain}" | cut -c 7-${end_of_name})

New:
for xml_domain in $(echo "${response}" | tr -d '\012\015' | egrep_o "."); do xml_domain_name=$(echo $xml_domain | sed -n 's/.(.)._/\1/p');

The difference is in parsing the response; I tested and it understands (and modifies) the right domain / hostname and lesson learnt, patch doesn't post on github

Thank you for the codes, but I already have changed minor changes (using _digest(), _math(), _contains()) to my code locally without any issue. But the parsing of the XML lists (from domain list and domain rows list) are not correctly working in certain circumstances. I'm trying to rewrite them using tr and sed only as Neilpang suggest that. I think it should be working today. Also Jakub Kalousek granted me access to WEDOS account so I will be able to test these changes soon.

I have tried what you code but it seems non complete.

[mxtuma]

@Neilpang please can you help me with passing DNS/Docker action in a local repo? I can't understand why it is failing but I assume the virtual machine init is broken as the cloud image of ubuntu has been changed to version 20. I reset all the secures as in WIKI pages, also we pass IP whitelisting filter at provider. But this is really strange for me. Thanks a lot for pointing me to a solution. Michal

[mxtuma]

@Neilpang please can you help me with passing DNS/Docker action in a local repo? I can't understand why it is failing but I assume the virtual machine init is broken as the cloud image of ubuntu has been changed to version 20. I reset all the secures as in WIKI pages, also we pass IP whitelisting filter at provider. But this is really strange for me. Thanks a lot for pointing me to a solution. Michal

@Neilpang Please ignore this request. I didn't read the log carefully, sorry for disturbing. But if you could review the code in order to pass your check, the code is ready.

[Neilpang]

It seems that your DNS check is not passing yet?
https://github.com/mxtuma/acme.sh/actions/runs/374287059

[mxtuma]

It seems that your DNS check is not passing yet?
https://github.com/mxtuma/acme.sh/actions/runs/374287059

You are right, Mac OS is getting me in troubles. But hope it is only a newline replacement what make my DNS unsusable. Hard to test, but I pass somehow shortly.

[mxtuma]

OK guys, now hopefully all the test should pass. I found the issue with using sed. Previous versions use GNU extension + as a regex repeater (at least one or more match) and also on Mac OS X can't replace to \n character in substitution command.

The code was replaced with * as a repeater and duplicate what should match without repeater - result is the same, at least one match or more.
The \n replacement were substituted by replacing to \t and then translating \t to \n by tr. That should work on all platforms.

Now all the test started in my repository clone, hopefully the last commit for being able to be merged to master branch.
Cheers

[mxtuma]

OK, I was too quick with the summary. I'm not sure but PebbleScript test failed and I think it is not related to the dns API implementation. I'm not sure how to prevent it from failing. Please can someone help? Thanks indeed, I'm starting to be mad.

[Neilpang]

rebase to the latest code first.

[Neilpang]

You can get a solaris shell from here to debug your script:
https://github.com/vmactions/shell-solaris

[mxtuma]

You can get a solaris shell from here to debug your script:
https://github.com/vmactions/shell-solaris

Thanks, I'll do the testing with solaris. I remember solaris from the university and it crosses my life again. Neil, all the tests are real pain. You are really good tester. :)
With each commit I come closer and closer.

[Neilpang]

You can get a solaris shell from here to debug your script:
https://github.com/vmactions/shell-solaris

Thanks, I'll do the testing with solaris. I remember solaris from the university and it crosses my life again. Neil, all the tests are real pain. You are really good tester. :)
With each commit I come closer and closer.

Appreciate your patience and courage too.
The more pain we(the developer) get, the happier the user will be.
I believe you are really close.

Thanks.

[jkalousek]

Just a little poke @Neilpang. Any update? (Just note that old domain is no longer availible. In case you will be doing more testing let me know, I will tell you which one to use).

[mxtuma]

Just a little poke @Neilpang. Any update? (Just note that old domain is no longer availible. In case you will be doing more testing let me know, I will tell you which one to use).

Hi Jakub,
I'm sorry I did not manage that. I really appreciate your help but I lost couple of days with implementation and because of not being able to perform Solaris testing I quit that as is. I have tried today to contact few friends of mine if they can help and try to finish it. But because I have a family with two kids it is hard for me to manage some free time. Even in these stupid pandemic days as we all are, when I have to teach my kids instead of teachers. Uff.
Thank you for your help.
Best regards,
Michal