feat(deploy_ali_cdn): support Alibaba Cloud CDN deployment #5205

PMExtra · 2024-07-11T10:29:55Z

API Documents

zh-cn

en

PMExtra · 2024-07-11T10:32:08Z

deploy/ali_cdn.sh

+  Ali_Key="${Ali_Key:-$(_readaccountconf_mutable Ali_Key)}"
+  Ali_Secret="${Ali_Secret:-$(_readaccountconf_mutable Ali_Secret)}"
+  if [ -z "$Ali_Key" ] || [ -z "$Ali_Secret" ]; then
+    Ali_Key=""
+    Ali_Secret=""
+    _err "You don't specify aliyun api key and secret yet."
+    return 1
+  fi
+
+  #save the api key and secret to the account conf file.
+  _saveaccountconf_mutable Ali_Key "$Ali_Key"
+  _saveaccountconf_mutable Ali_Secret "$Ali_Secret"


Copy from

acme.sh/dnsapi/dns_ali.sh

Lines 13 to 24 in 377a37e

Ali_Key="${Ali_Key:-$(_readaccountconf_mutable Ali_Key)}"

Ali_Secret="${Ali_Secret:-$(_readaccountconf_mutable Ali_Secret)}"

if [ -z "$Ali_Key" ] || [ -z "$Ali_Secret" ]; then

Ali_Key=""

Ali_Secret=""

_err "You don't specify aliyun api key and secret yet."

return 1

fi

#save the api key and secret to the account conf file.

_saveaccountconf_mutable Ali_Key "$Ali_Key"

_saveaccountconf_mutable Ali_Secret "$Ali_Secret"

PMExtra · 2024-07-11T10:43:07Z

deploy/ali_cdn.sh

+# stdin stdout
+_url_encode_upper() {
+  encoded=$(_url_encode)
+
+  for match in $(echo "$encoded" | _egrep_o '%..' | sort -u); do
+    upper=$(echo "$match" | _upper_case)
+    encoded=$(echo "$encoded" | sed "s/$match/$upper/g")
+  done
+
+  echo "$encoded"
+}


Alibaba Cloud API signature needs parameters encoded in upper-case.

_url_encode function from acme.sh can only encode with lower-case.

We have an ali_urlencode function from dns_ali.sh but it does not support multi-line strings.

Maybe we can add an argument to _url_encode function to make it support upper-case? (it only needs a very simple change in the *) case).

acme.sh/acme.sh

Lines 884 to 887 in 0d8a314

#other hex

*)

printf '%%%s' "$_hex_code"

;;

Can we change it to:

*) case "$1" in '[uU]*') # for upper-case printf '%%%s' "$_hex_code" | _upper_case ;; *) # default to lower-case printf '%%%s' "$_hex_code" ;; esac ;;

PMExtra · 2024-07-11T10:43:53Z

deploy/ali_cdn.sh

+_ali_urlencode() {
+  _str="$1"
+  _str_len=${#_str}
+  _u_i=1
+  while [ "$_u_i" -le "$_str_len" ]; do
+    _str_c="$(printf "%s" "$_str" | cut -c "$_u_i")"
+    case $_str_c in [a-zA-Z0-9.~_-])
+      printf "%s" "$_str_c"
+      ;;
+    *)
+      printf "%%%02X" "'$_str_c"
+      ;;
+    esac
+    _u_i="$(_math "$_u_i" + 1)"
+  done
+}
+
+_ali_nonce() {
+  #_head_n 1 </dev/urandom | _digest "sha256" hex | cut -c 1-31
+  #Not so good...
+  date +"%s%N" | sed 's/%N//g'
+}
+
+_timestamp() {
+  date -u +"%Y-%m-%dT%H%%3A%M%%3A%SZ"
+}


Copy from https://github.com/acmesh-official/acme.sh/blob/3.0.7/dnsapi/dns_ali.sh

PMExtra · 2024-07-11T10:51:37Z

deploy/ali_cdn.sh

+# act ign mtd
+_ali_rest() {
+  act="$1"
+  ign="$2"
+  mtd="$3"
+
+  signature=$(printf "%s" "$mtd&%2F&$(_ali_urlencode "$query")" | _hmac "sha1" "$(printf "%s" "$Ali_Secret&" | _hex_dump | tr -d " ")" | _base64)
+  signature=$(_ali_urlencode "$signature")
+  url="$Ali_API?$query&Signature=$signature"
+
+  if [ "$mtd" = "GET" ]; then
+    response="$(_get "$url")"
+  else
+    # post payload is not supported yet because of signature
+    response="$(_post "" "$url")"
+  fi
+
+  _ret="$?"
+  _debug2 response "$response"
+  if [ "$_ret" != "0" ]; then
+    _err "Error <$act>"
+    return 1
+  fi
+
+  if [ -z "$ign" ]; then
+    message="$(echo "$response" | _egrep_o "\"Message\":\"[^\"]*\"" | cut -d : -f 2 | tr -d \")"
+    if [ "$message" ]; then
+      _err "$message"
+      return 1
+    fi
+  fi
+}


Reference https://github.com/acmesh-official/acme.sh/blob/3.0.7/dnsapi/dns_ali.sh

However, the original method was hard coded with the GET method.

So, I made a new argument to specify the method.

Notice that Alibaba Cloud API supports passing the parameters by either query or body. (https://help.aliyun.com/zh/sdk/product-overview/rpc-mechanism#section-9x3-wo3-8l9)

But we must sign with all the parameters, in alphabetical order, regardless of where they come from.

So, I didn't support the post-payload yet.

PMExtra · 2024-07-17T09:40:48Z

Solve #1461

iflamed · 2024-07-17T15:48:10Z

Solve #1461

How to use this deploy script?

PMExtra · 2024-07-18T06:45:12Z

@iflamed

# This deployment required following variables
# export Ali_Key="ALIACCESSKEY"
# export Ali_Secret="ALISECRETKEY"
# export DEPLOY_ALI_CDN_DOMAIN="cdn.example.com"
# If you have more than one domain, just
# export DEPLOY_ALI_CDN_DOMAIN="cdn1.example.com cdn2.example.com"
#
# The credentials are shared with all domains, also shared with dns_ali api

For example:

export Ali_Key=xxxx
export Ali_Secret=xxxx
export DEPLOY_ALI_CDN_DOMAIN=cdn.example.com # default to the certificate domain if not specified
acme.sh --deploy --deploy-hook ali_cdn -d example.com

or using docker:

docker exec \
  -e Ali_Key=xxxx \
  -e Ali_Secret=xxxx \
  -e DEPLOY_ALI_CDN_DOMAIN=cdn.example.com \
  acme.sh --deploy --deploy-hook ali_cdn -d example.com

PMExtra · 2024-07-18T07:34:56Z

@iflamed wiki: https://github.com/acmesh-official/acme.sh/wiki/deployhooks#33-deploy-your-certificate-to-alibaba-cloud-cdn-aliyun

iflamed · 2024-07-18T15:08:41Z

@iflamed wiki: https://github.com/acmesh-official/acme.sh/wiki/deployhooks#33-deploy-your-certificate-to-alibaba-cloud-cdn-aliyun

Hope this PR merged ASAP.

jtwangfos · 2024-07-26T03:44:02Z

Great job man, thanks a lot. <3

ShirasawaSama · 2024-08-02T06:50:29Z

@PMExtra Can DCDN be supported? Thanks.

Great job.

是否可以支持DCDN？

PMExtra · 2024-08-02T07:43:46Z

@ShirasawaSama 目前没有支持，但只要把这一行代码的 Cdn 改成 Dcdn 应该就行了，你暂且可以复制一份改一下先用着，回头我考虑下加个设置来控制。

acme.sh/deploy/ali_cdn.sh

Line 145 in 945b7de

query=$query'&Action=SetCdnDomainSSLCertificate'

PMExtra · 2024-08-02T07:45:57Z

@ShirasawaSama 哦，还漏了API版本，一共改两处，一个是 Cdn 改 Dcdn，另一个是 2018-05-10 改 2018-01-15

acme.sh/deploy/ali_cdn.sh

Lines 142 to 157 in 945b7de

    
           _set_cdn_domain_ssl_certificate_query() { 
        
             query='' 
        
             query=$query'AccessKeyId='$Ali_Key 
        
             query=$query'&Action=SetCdnDomainSSLCertificate' 
        
             query=$query'&CertType=upload' 
        
             query=$query'&DomainName='$1 
        
             query=$query'&Format=json' 
        
             query=$query'&SSLPri='$3 
        
             query=$query'&SSLProtocol=on' 
        
             query=$query'&SSLPub='$2 
        
             query=$query'&SignatureMethod=HMAC-SHA1' 
        
             query=$query"&SignatureNonce=$(_ali_nonce)" 
        
             query=$query'&SignatureVersion=1.0' 
        
             query=$query'&Timestamp='$(_timestamp) 
        
             query=$query'&Version=2018-05-10' 
        
           }

anjia0532 · 2024-09-12T06:51:50Z

@PMExtra 可以试试这个接口 https://api.aliyun.com/document/cas/2020-04-07/UploadUserCertificate ，阿里云主要的云产品基本都支持使用证书中心管理证书，cdn/dcdn/oss/slb/alb/clb ,相比较各个产品各自管各自的证书，不如用证书中心统一管理

PMExtra · 2024-09-12T06:53:51Z

@anjia0532 我认为这样是增加了复杂性，各个产品自己的接口还是需要调用，只是在调用接口的时候，传递一个证书payload，还是传递一个cas id的区别而已。既然都用acme接管证书管理了，就没必要再去用cas。

Neilpang · 2024-09-12T07:13:06Z

deploy/ali_cdn.sh

+
+# stdin stdout
+_url_encode_upper() {
+  encoded=$(_url_encode)


能不能把 encoded 直接全部 _upper_case 一下 ?

不可以，原始输入是 base64 ，要区分大小写的。
阿里云要求是特殊字符转义的 %XX 必须是大写的 HEX ，否则签名验证通不过。
但不能把 base64 给全转成大写。

其实就是为了处理 +, /, = 和 \n 字符对应的转义，其余字符必须保持原样。

anjia0532 · 2024-09-12T08:57:06Z

@anjia0532 我认为这样是增加了复杂性，各个产品自己的接口还是需要调用，只是在调用接口的时候，传递一个证书payload，还是传递一个cas id的区别而已。既然都用acme接管证书管理了，就没必要再去用cas。

确实cas增加了复杂性，但是好处是 1. 可以付费使用部署功能（支持部署到腾讯云，aws，华为云，以及阿里云自身云产品），2. 操作主动吊销泄露的证书更方便。3. 方便删除过期证书。 4. 支持从cas下载证书（具体产品里的证书不支持下载）

当然分开写确实更小而美一些。

PMExtra · 2024-09-12T09:09:02Z

既然你都手动操作部署了，那你不如手动上传证书。acme为的是无人值守，这些手动部署的需求我认为不在考虑范围内；
acme上传到cas的证书应该是不能通过cas来吊销的，如果需要吊销仍然是必须通过acme操作；
不部署到cas，就不需要删除过期证书了；
从cas下载证书这也是一个人工操作，再次强调使用acme为的是无人值守，既然你都手动操作了，不如手动上传。

yikuo123 · 2024-09-18T03:52:30Z

@iflamed

# If you have more than one domain, just
# export DEPLOY_ALI_CDN_DOMAIN="cdn1.example.com cdn2.example.com"
#
# The credentials are shared with all domains, also shared with dns_ali api

Dose it support muti-domains? I found it is said that:

The accelerated domain name for which you want to configure the SSL certificate. The type of request supported by the domain name must be HTTPS. You can specify only one domain name in each request.

https://www.alibabacloud.com/help/en/cdn/developer-reference/api-cdn-2018-05-10-setcdndomainsslcertificate#:~:text=You%20can%20specify%20only%20one%20domain%20name%20in%20each%20request.

And, there is an API that supports muti-domains:

https://www.alibabacloud.com/help/en/cdn/developer-reference/api-cdn-2018-05-10-batchsetcdndomainservercertificate

iflamed · 2024-09-18T03:57:34Z

@yikuo123

@iflamed
# If you have more than one domain, just
# export DEPLOY_ALI_CDN_DOMAIN="cdn1.example.com cdn2.example.com"
#
# The credentials are shared with all domains, also shared with dns_ali api
Dose it support muti-domains? I found it is said that:

The accelerated domain name for which you want to configure the SSL certificate. The type of request supported by the domain name must be HTTPS. You can specify only one domain name in each request.

https://www.alibabacloud.com/help/en/cdn/developer-reference/api-cdn-2018-05-10-setcdndomainsslcertificate#:~:text=You%20can%20specify%20only%20one%20domain%20name%20in%20each%20request.

And, there is an API that supports muti-domains:

https://www.alibabacloud.com/help/en/cdn/developer-reference/api-cdn-2018-05-10-batchsetcdndomainservercertificate

I haven't tested it yet.

PMExtra · 2024-09-18T07:02:01Z

@yikuo123 @iflamed

It supports multiple domains by a loop. It will call the SetCdnDomainSSLCertificate API for each domain.

acme.sh/deploy/ali_cdn.sh

Line 57 in 945b7de

for domain in $DEPLOY_ALI_CDN_DOMAIN; do

ShirasawaSama · 2024-09-18T07:38:30Z

@PMExtra 现在如果需要使用dcdn的话还是需要手动复制一份脚本吗？是否考虑过添加环境变量的形式区分是否为dcdn或者是正常cdn？又或者说再加一个ali_dcdn.sh脚本？

Now if I need to use dcdn, do I still need to copy the script manually? Have you considered adding an environment variable to determine if it's a dcdn deployment or a normal cdn? Or maybe adding another ali_dcdn.sh script?

PMExtra · 2024-09-18T07:57:54Z

@ShirasawaSama

考虑到可能有人同时需要 CDN 和 DCDN ，所以直接通过环境变量进行区分的方式可能不太合适。

而直接复制一份 ali_dcdn.sh，又会导致代码和 ali_cdn.sh 有90%以上的重复率。

我个人希望抽出一个 libs/ali_api.sh 这样的文件，供 deploy/ali_cdn.sh, deploy/ali_dcdn.sh 复用。甚至 dnsapi/dns_ali.sh 也可以重构一下，复用这个 lib 。

除了阿里，包括 awscn / 腾讯云 / 华为云等等，也都有不止一个服务有证书需求，也都有各自的dnsapi，而其中有大量可以复用的函数。

请问 @Neilpang 对此有什么想法？

Neilpang · 2024-09-18T08:02:34Z

@PMExtra 想法不错. 如果只是有些方法可以直接复用. 可以考虑直接把对应的文件 source 进来. 可以使用 _findHook() 方法来查找对应的文件. 然后, 自己 source 进来就好了.

PMExtra · 2024-09-18T08:15:17Z

@Neilpang

问题好像和 _findHook 没有关系？

对于特定的一个 deployhook 或者 dnsapi ，它需要哪个服务商的API这是确定的，可以直接用硬编码的相对路径引入。

例如 deploy/ali_cdn.sh 我希望开头有一行 source ../libs/ali_api.sh 。

这样以后加入 deploy/ali_dcdn.sh 和 deploy/ali_slb.sh 等等其它服务的时候，也都是同样的方式，引入阿里的公共函数，减少重复代码。

我现在是需要您评估一下，在acme的顶层目录结构新增一个 libs 目录，里面存放各个云服务厂商的公共函数，例如 libs/ali_api.sh, libs/tencent_api.sh, libs/huawei_api.sh 这样的结构是否能接受。

Neilpang · 2024-09-18T08:27:46Z

@PMExtra

不要直接写死相对路径. 用_findHook 来查找, 它会返回给你全路径.

加个libs变动有点大, 内部倒是问题不大, 但是很多第三方环境都集成了acme.sh, 他们可能已经假定了顶层的目录样子. 单加一个目录他们都需要更新安装脚本. 工作量有点大. 如果不更新, 以前工作的hook 可能就不工作了. 比如, 你把很多函数都抽走了.

而且, 所谓的需要公用的云服务商, 两只手都能数完. 对比起来感觉不划算.

咱们最重要的目标之一就是要尽最大努力保持兼容性, 除非外部因素变动. 不能让客户之前能工作的场景升级后爆了. 因为acme的价值和目标就是无人值守. 没有人愿意每天盯着.

简单你就source吧. 你说呢.

PMExtra · 2024-09-18T08:38:11Z

@Neilpang

没太理解，你的意思是有人会通过第三方脚本来安装和更新 acme.sh 吗？

对于使用常规手段进行安装和更新的用户(acme.sh --install 和 acme.sh --upgrade)，或者直接 git clone 的用户，是不会受影响的。

对于不更新脚本的用户，我们的新feature也是不会对它有影响的。

唯一可能受影响的，就是使用非官方建议的方式更新脚本，只拉取特定文件的用户。

那么这种情况下，无论目录结构怎么样，只要新增了文件，它都有可能缺失。我认为这种超出官方支持范围，「自作聪明」做精简的行为，就理应承担相应的风险。

或者退一步，我们不对现有的 dnsapi/dns_ali.sh 做重构。只对刚刚新增的 deploy/ali_dns.sh 和将来新增的功能引入 libs/ali_api.sh，这样是否可以接受呢？

Neilpang · 2024-09-18T08:46:25Z

@PMExtra

终极用户当然大多数都是用脚本安装的.

但是第三方环境, 例如宝塔之类的, 他们是写死路径的, 可能还要对路径设置相关权限. 他们不会用安装脚本.

既然要抽函数避免copy, 那就只保留一份. 保留已有的文件不动 dnsapi/dns_ali.sh. 其他的文件需要引用的, 就source 它.

让新增文件对旧文件保持严格单向依赖. 这样会减少很多麻烦.

ShirasawaSama · 2024-09-18T08:51:40Z

@PMExtra 除了dcdn是否还存在其他需要配置证书的场景？如oss、云建站、云函数等？

PMExtra · 2024-09-18T08:52:21Z

@PMExtra 除了dcdn是否还存在其他需要配置证书的场景？如oss、云建站、云函数等？

是有这些场景的，我在上面已经描述过了。

ShirasawaSama · 2024-09-18T08:55:34Z

@PMExtra 除了dcdn是否还存在其他需要配置证书的场景？如oss、云建站、云函数等？

是有这些场景的，我在上面已经描述过了。

我个人觉得是否提供一个 DEPLOY_ALI_SERVICES=cdn,dcdn,oss 是否能避免出现一大堆 ali_cdn.sh, ali_dcdn.sh, ali_oss.sh 文件的出现？

PMExtra · 2024-09-18T08:56:50Z

我个人觉得是否提供一个 DEPLOY_ALI_SERVICES=cdn,dcdn,oss 是否能避免出现一大堆 ali_cdn.sh, ali_dcdn.sh, ali_oss.sh 文件的出现？

@ShirasawaSama 你每个服务下的域名是不一样的，而且个别服务可能需要额外的参数，这对于只使用其中一个或少数服务的人来说，会增加配置的难度。

Neilpang · 2024-09-18T08:57:57Z

我个人觉得是否提供一个 DEPLOY_ALI_SERVICES=cdn,dcdn,oss

这取决于这几个文件的相似度. 如果仅仅是几个调用参数不一样, 那没问题.

如果几个服务差异比较大, 会导致 all-in-one 脚本太长, 过于复杂而不利于维护. 其中一个出了bug 没人愿意来修, 看一眼都头大.

Neilpang · 2024-09-18T09:00:55Z

DEPLOY_ALI_SERVICES=cdn,dcdn,oss

另外, 对于一个证书需要部署到多个服务的场景, 可以使用多个 --deploy-hook aaa --deploy-hook bbb --deploy-hook ccc 来解决.

PMExtra added 2 commits July 11, 2024 18:29

feat(deploy_ali_cdn): support Alibaba Cloud CDN deployment

3c35ead

feat(deploy_ali_cdn): improve upper-case

945b7de

PMExtra commented Jul 11, 2024

View reviewed changes

PMExtra mentioned this pull request Jul 17, 2024

Add a deploy hook for AliCDN #1466

Closed

Neilpang reviewed Sep 12, 2024

View reviewed changes

Neilpang merged commit e646138 into acmesh-official:dev Sep 13, 2024
2 checks passed

PMExtra deleted the feature/ali_cdn branch September 13, 2024 12:00

PMExtra mentioned this pull request Sep 19, 2024

refactor: Alibaba Cloud API #5294

Merged

	Ali_Key="${Ali_Key:-$(_readaccountconf_mutable Ali_Key)}"
	Ali_Secret="${Ali_Secret:-$(_readaccountconf_mutable Ali_Secret)}"
	if [ -z "$Ali_Key" ] \|\| [ -z "$Ali_Secret" ]; then
	Ali_Key=""
	Ali_Secret=""
	_err "You don't specify aliyun api key and secret yet."
	return 1
	fi

	#save the api key and secret to the account conf file.
	_saveaccountconf_mutable Ali_Key "$Ali_Key"
	_saveaccountconf_mutable Ali_Secret "$Ali_Secret"

feat(deploy_ali_cdn): support Alibaba Cloud CDN deployment #5205

feat(deploy_ali_cdn): support Alibaba Cloud CDN deployment #5205

Conversation

PMExtra commented Jul 11, 2024 • edited Loading

API Documents

zh-cn

en

PMExtra Jul 11, 2024

Choose a reason for hiding this comment

PMExtra Jul 11, 2024

Choose a reason for hiding this comment

PMExtra Jul 11, 2024 • edited Loading

Choose a reason for hiding this comment

PMExtra Jul 11, 2024

Choose a reason for hiding this comment

PMExtra Jul 11, 2024

Choose a reason for hiding this comment

PMExtra commented Jul 17, 2024

iflamed commented Jul 17, 2024

PMExtra commented Jul 18, 2024 • edited Loading

PMExtra commented Jul 18, 2024

iflamed commented Jul 18, 2024

jtwangfos commented Jul 26, 2024 • edited Loading

ShirasawaSama commented Aug 2, 2024 • edited Loading

PMExtra commented Aug 2, 2024

PMExtra commented Aug 2, 2024

anjia0532 commented Sep 12, 2024 • edited Loading

PMExtra commented Sep 12, 2024

Neilpang Sep 12, 2024

Choose a reason for hiding this comment

PMExtra Sep 12, 2024

Choose a reason for hiding this comment

anjia0532 commented Sep 12, 2024

PMExtra commented Sep 12, 2024

yikuo123 commented Sep 18, 2024 • edited Loading

iflamed commented Sep 18, 2024 • edited Loading

PMExtra commented Sep 18, 2024 • edited Loading

ShirasawaSama commented Sep 18, 2024

PMExtra commented Sep 18, 2024 • edited Loading

Neilpang commented Sep 18, 2024

PMExtra commented Sep 18, 2024

Neilpang commented Sep 18, 2024

PMExtra commented Sep 18, 2024

Neilpang commented Sep 18, 2024

ShirasawaSama commented Sep 18, 2024 • edited Loading

PMExtra commented Sep 18, 2024

ShirasawaSama commented Sep 18, 2024

PMExtra commented Sep 18, 2024

Neilpang commented Sep 18, 2024 • edited Loading

Neilpang commented Sep 18, 2024

PMExtra commented Jul 11, 2024 •

edited

Loading

PMExtra Jul 11, 2024 •

edited

Loading

PMExtra commented Jul 18, 2024 •

edited

Loading

jtwangfos commented Jul 26, 2024 •

edited

Loading

ShirasawaSama commented Aug 2, 2024 •

edited

Loading

anjia0532 commented Sep 12, 2024 •

edited

Loading

yikuo123 commented Sep 18, 2024 •

edited

Loading

iflamed commented Sep 18, 2024 •

edited

Loading

PMExtra commented Sep 18, 2024 •

edited

Loading

PMExtra commented Sep 18, 2024 •

edited

Loading

ShirasawaSama commented Sep 18, 2024 •

edited

Loading

Neilpang commented Sep 18, 2024 •

edited

Loading