truenas_ws deploy hook fails on TrueNAS Scale 25.04

[kapsh]

New truenas_wsdeploy hook in recent acme.sh v3.1.1 fails to find Truenas edition in api response for method system.info.

Excerpt from running acme.sh --deploy-hook truenas_ws on TrueNAS Scale 25.04:

Checking TrueNAS health...
_ws_call arg1='system.ready'
_ws_call arg2                                   
_ws_call arg3
_ws_response='True'                             
OK                                              
17:SAVED_DEPLOY_TRUENAS_APIKEY='xxx'
TrueNAS health: OK                              
Gather system info...
_ws_call arg1='system.info'                  
_ws_call arg2                                
_ws_call arg3                                
_ws_response='{"version": "25.04.0", ...
TrueNAS system: 25.04.0
TrueNAS version: 25.04.0                        
Cannot gather TrueNAS system. Nor CORE oder SCALE detected.
Error deploying for domain: xxx
Error encountered while deploying.  

In previous versions system.info call would output value "truenas-scale-x.y" but this was changed to short version. Full version string doesn't include "scale" either (quote below) and product type was renamed to COMMUNITY_EDITION:

$ midclt call system.info | jq .version
"25.04.0"
$ midclt call system.version      
TrueNAS-25.04.0
$ midclt call  system.product_type 
COMMUNITY_EDITION

[github-actions]

Please upgrade to the latest code and try again first. Maybe it's already fixed. acme.sh --upgrade If it's still not working, please provide the log with --debug 2, otherwise, nobody can help you.

[JkktBkkt]

This is a valid issue, I've experienced the same problem.

[JkktBkkt]

This might help, but in light of #6343 I'm unsure whether this is really valuable as-is as the user would still need a work-around

diff --git a/deploy/truenas_ws.sh b/deploy/truenas_ws.sh
index 940cde2..7973426 100644
--- a/deploy/truenas_ws.sh
+++ b/deploy/truenas_ws.sh
 # TrueNAS deploy script for SCALE/CORE using websocket
 # It is recommend to use a wildcard certificate
@@ -181,6 +181,13 @@ truenas_ws_deploy() {
   _ws_response=$(_ws_call "system.info")
   _truenas_system=$(printf "%s" "$_ws_response" | jq -r '."version"' | cut -d '-' -f 2 | tr '[:lower:]' '[:upper:]')
   _truenas_version=$(printf "%s" "$_ws_response" | jq -r '."version"' | cut -d '-' -f 3)
+  _truenas_unified_version=$(printf "%s" "$_truenas_system"| cut -d '.' -f 1)
+  _info "$_truenas_unified_version"
+  if [ "$_truenas_unified_version" == "25" ]; then
+    _truenas_system="SCALE";
+  elif [ "$_truenas_unified_version" == "13" ];
+    then _truenas_system="CORE";
+  fi
   _info "TrueNAS system: $_truenas_system"
   _info "TrueNAS version: $_truenas_version"
   if [ "$_truenas_system" != "SCALE" ] && [ "$_truenas_system" != "CORE" ]; then

[Horlogrium]

I think this issue can be closed as it as been fixed in the last version of the deploy hook.

########## System info

_info "Gather system info..."
_ws_response=$(_ws_call "system.info")
_truenas_version=$(printf "%s" "$_ws_response" | jq -r '."version"')
_info "TrueNAS version: $_truenas_version"

[kapsh]

Yeah this specific error was fixed, there is no check for literal "scale" or "core" as of a1f4f9d. Hook still doesn't work anyway for a number of another reasons, in order of encountered and worked around: recent changes in 25.10 require tls for websocket connections (export DEPLOY_TRUENAS_PROTOCOL=wss can fix it); error "ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'localhost'. (_ssl.c:1006)" is thrown by midclt when DEPLOY_TRUENAS_HOSTNAME set to "localhost" (setting it to target domain also can fix it, though I cannot imagine how this will work for first deploy with self-signed preinstalled cert, midclt has no cli switch to ignore tls checks); in the end it could not upload certificate saying:

Current WebUI certificate ID: 8                                                                                                                                                                                                                                                              
Current WebUI certificate name: Letsencrypt_2025-09-01_091243                                                                                                                                                                                                                                
Upload new certificate...                                                                                                                                                                                                                                                                    
New WebUI certificate name: acme_20251101_202724                                                                                                                                                                                                                                             
_certname='acme_20251101_202724'                                                                                                                                                                                                                                                             
Traceback (most recent call last):                                                                                                                                                                                                                                                                                          
  File "<stdin>", line 5, in <module>                                                                                                                                                                                                                                                                                       
  File "/usr/lib/python3/dist-packages/truenas_api_client/__init__.py", line 117, in __exit__                                                                                                                                                                                                                               
    return self.__client.__exit__(exc_type, exc_val, exc_tb)                                                                                                                                                                                                                                                                
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                
  File "<stdin>", line 16, in <module>                                                                                                                                                                                                                                                                                      
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 402, in call                                                                                                                                                                                                                                     
    return self.wait(c, callback=callback, job=job, timeout=timeout)                                                                                                                                                                                                                                                        
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                        
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 428, in wait                                                                                                                                                                                                                                     
    return jobobj.result()                                                                                                                                                                                                                                                                                                  
           ^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                  
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 166, in result                                                                                                                                                                                                                                   
    raise ValidationErrors(job['exc_info']['extra'])                                                                                                                                                                                                                                                                        
truenas_api_client.exc.ValidationErrors: [EINVAL] certificate_create.passphrase: String should have at least 1 character                                                                                                                                                                                                    
Trying to upload new certificate...                                                                                                                                                                                                                                                          
New certificate ID:                                                                                                                                                                                                                                                                          
Replace FTP certificate...      

This error wasn't even checked and continued silently without creating mentioned "acme_20251101_202724" (it failed later on ftp step).
Guessing by message, maybe certifcate.create call in

acme.sh/deploy/truenas_ws.sh

Line 74 in a1f4f9d

ret = c.call("certificate.create", {"name": "$3", "create_type": "CERTIFICATE_CREATE_IMPORTED", "certificate": fullchain, "privatekey": privatekey, "passphrase": ""}, job=True)

should omit empty passphrase argument? Couldn't check that, at this point I switched back to old "truenas" hook as my cert was dangerously close to expiring.

[gorucci]

Still an issue for me.

I use truenas 25.04 (and the old truenas deploy hook doesn't work). I have a docker container neilpang/acme.sh:dev. I ran acme.sh --upgrade just in case.

Running acme.sh --insecure --deploy --debug 2 -d "${DEPLOY_TRUENAS_HOSTNAME}" --deploy-hook truenas_ws gives this error:

<...>
[Sun Nov  2 05:50:43 UTC 2025] _ws_uri='ws://<truenas.mydomain.tld>/websocket'
[Sun Nov  2 05:50:43 UTC 2025] DEPLOY_TRUENAS_APIKEY='[hidden](please add '--output-insecure' to see this value)'
[Sun Nov  2 05:50:43 UTC 2025] Environment variables: OK
[Sun Nov  2 05:50:43 UTC 2025] Checking TrueNAS health...
[Sun Nov  2 05:50:43 UTC 2025] _ws_call arg1='system.ready'
[Sun Nov  2 05:50:43 UTC 2025] _ws_call arg2
[Sun Nov  2 05:50:43 UTC 2025] _ws_call arg3
/usr/local/bin/acme.sh: line 48: midclt: not found
[Sun Nov  2 05:50:43 UTC 2025] _ws_response
[Sun Nov  2 05:50:43 UTC 2025] TrueNAS is not ready.
[Sun Nov  2 05:50:43 UTC 2025] Please check environment variables DEPLOY_TRUENAS_APIKEY, DEPLOY_TRUENAS_HOSTNAME and DEPLOY_TRUENAS_PROTOCOL.
[Sun Nov  2 05:50:43 UTC 2025] Verify API key.
[Sun Nov  2 05:50:43 UTC 2025] Error deploying for domain: <truenas.mydomain.tld>
[Sun Nov  2 05:50:43 UTC 2025] Error encountered while deploying.

My truenas still has a valid cert, and I've tried with DEPLOY_TRUENAS_PROTOCOL=wss as well. No success either.

[JkktBkkt]

@gorucci

/usr/local/bin/acme.sh: line 48: midclt: not found

That specific part indicates something's wrong with your system, make sure your PATH variable used by acme.sh has /usr/bin in it and that /usr/bin/midclt is callable

[gorucci]

@gorucci

/usr/local/bin/acme.sh: line 48: midclt: not found

That specific part indicates something's wrong with your system, make sure your PATH variable used by acme.sh has /usr/bin in it and that /usr/bin/midclt is callable

AIUI, midclt is truenas specific binary, and it isn't included in the acme.sh docker image. I run acme.sh on a separate machine (if it does matter). Just in case, I've checked -- midclt is present on the truenas itself.

[Horlogrium]

Yeah this specific error was fixed, there is no check for literal "scale" or "core" as of a1f4f9d. Hook still doesn't work anyway for a number of another reasons, in order of encountered and worked around: recent changes in 25.10 require tls for websocket connections (export DEPLOY_TRUENAS_PROTOCOL=wss can fix it); error "ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'localhost'. (_ssl.c:1006)" is thrown by midclt when DEPLOY_TRUENAS_HOSTNAME set to "localhost" (setting it to target domain also can fix it, though I cannot imagine how this will work for first deploy with self-signed preinstalled cert, midclt has no cli switch to ignore tls checks); in the end it could not upload certificate saying:

Current WebUI certificate ID: 8                                                                                                                                                                                                                                                              
Current WebUI certificate name: Letsencrypt_2025-09-01_091243                                                                                                                                                                                                                                
Upload new certificate...                                                                                                                                                                                                                                                                    
New WebUI certificate name: acme_20251101_202724                                                                                                                                                                                                                                             
_certname='acme_20251101_202724'                                                                                                                                                                                                                                                             
Traceback (most recent call last):                                                                                                                                                                                                                                                                                          
  File "<stdin>", line 5, in <module>                                                                                                                                                                                                                                                                                       
  File "/usr/lib/python3/dist-packages/truenas_api_client/__init__.py", line 117, in __exit__                                                                                                                                                                                                                               
    return self.__client.__exit__(exc_type, exc_val, exc_tb)                                                                                                                                                                                                                                                                
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                
  File "<stdin>", line 16, in <module>                                                                                                                                                                                                                                                                                      
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 402, in call                                                                                                                                                                                                                                     
    return self.wait(c, callback=callback, job=job, timeout=timeout)                                                                                                                                                                                                                                                        
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                        
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 428, in wait                                                                                                                                                                                                                                     
    return jobobj.result()                                                                                                                                                                                                                                                                                                  
           ^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                  
  File "/usr/lib/python3/dist-packages/truenas_api_client/legacy.py", line 166, in result                                                                                                                                                                                                                                   
    raise ValidationErrors(job['exc_info']['extra'])                                                                                                                                                                                                                                                                        
truenas_api_client.exc.ValidationErrors: [EINVAL] certificate_create.passphrase: String should have at least 1 character                                                                                                                                                                                                    
Trying to upload new certificate...                                                                                                                                                                                                                                                          
New certificate ID:                                                                                                                                                                                                                                                                          
Replace FTP certificate...      

This error wasn't even checked and continued silently without creating mentioned "acme_20251101_202724" (it failed later on ftp step). Guessing by message, maybe certifcate.create call in

acme.sh/deploy/truenas_ws.sh

Line 74 in a1f4f9d
ret = c.call("certificate.create", {"name": "$3", "create_type": "CERTIFICATE_CREATE_IMPORTED", "certificate": fullchain, "privatekey": privatekey, "passphrase": ""}, job=True)
should omit empty passphrase argument? Couldn't check that, at this point I switched back to old "truenas" hook as my cert was dangerously close to expiring.

I didn't saw your post so i opened an other issue : #6592
I don't use passphrase so i removed it in the deploy script, but it should be 1 character long now.
Maybe we should omit it when empty or make it null ?

[Horlogrium]

@gorucci

/usr/local/bin/acme.sh: line 48: midclt: not found

That specific part indicates something's wrong with your system, make sure your PATH variable used by acme.sh has /usr/bin in it and that /usr/bin/midclt is callable

AIUI, midclt is truenas specific binary, and it isn't included in the acme.sh docker image. I run acme.sh on a separate machine (if it does matter). Just in case, I've checked -- midclt is present on the truenas itself.

I think you are forced to use acme.sh on truenas itself since the script use midctl now.
Before that it used an API but sometimes sending the certificates was a too long message for the API and truenas said that they won't fix it.

[rglidden]

@kapsh

setting it to target domain also can fix it, though I cannot imagine how this will work for first deploy with self-signed preinstalled cert, midclt has no cli switch to ignore tls checks

The approach I took was to manually deploy the correct cert once. Once it's been bootstrapped, the deploy script can handle all the updates if configured correctly. Not ideal, but it works.

@Horlogrium

I think you are forced to use acme.sh on truenas itself since the script use midctl now.

You can install midctl on another machine as part of the TrueNAS API Client. This requires Python.

I didn't saw your post so i opened an other issue : #6592

I opened #6594 to try to get this fixed.

@gorucci

AIUI, midclt is truenas specific binary, and it isn't included in the acme.sh docker image.

This is correct. AFAIK the midctl is not included in the Docker image. You would have to build your own image or install the client in your running container from the link above.

[gorucci]

I think you are forced to use acme.sh on truenas itself since the script use midctl now.
Before that it used an API but sometimes sending the certificates was a too long message for the API and truenas said that they won't fix it.

This is correct. AFAIK the midctl is not included in the Docker image. You would have to build your own image or install the client in your running container from the link above.

So, effectively, truenas_ws.sh uses the shiny brand new ws api for deploying the cert... to localhost. That doesn't make much sense to me. At least this requirement should be stated in the header comment of the hook.

@Horlogrium, @rglidden, thanks for your comments.