Inconsistent API result structure between local and local_batch

[moio]

Description of Issue/Question

Calling the same function from the API might result in having a retcode field or not depending on whether the local or local_batch client is using, eg.:

Running a state.apply with a file.absent in it, via the local client:

master:~ # curl -X POST http://localhost:9080/run --data @request.json -H "Content-Type: application/json"

{
  "return": [
    {
      "minion-8.tf.local": {
        "file_|-mgrchannels_repo_clean_all_|-/etc/zypp/repos.d/susemanager:channels.repo_|-absent": {
          "comment": "File /etc/zypp/repos.d/susemanager:channels.repo is not present",
          "pchanges": {},
          "name": "/etc/zypp/repos.d/susemanager:channels.repo",
          "start_time": "14:15:43.250718",
          "result": true,
          "duration": 0.346,
          "__run_num__": 0,
          "__sls__": "cleanup_minion",
          "changes": {},
          "__id__": "mgrchannels_repo_clean_all"
        }
      }
    }
  ]
}

Note how, in each minion, the result is a mapping from an identifier string to an object.

Same call via the local_batch client:

master:~ # curl -X POST http://localhost:9080/run --data @request.json -H "Content-Type: application/json"

{
  "return": [
    {
      "minion-8.tf.local": {
        "file_|-mgrchannels_repo_clean_all_|-/etc/zypp/repos.d/susemanager:channels.repo_|-absent": {
          "comment": "File /etc/zypp/repos.d/susemanager:channels.repo is not present",
          "pchanges": {},
          "name": "/etc/zypp/repos.d/susemanager:channels.repo",
          "start_time": "14:17:22.903034",
          "result": true,
          "duration": 0.41,
          "__run_num__": 0,
          "__sls__": "cleanup_minion",
          "changes": {},
          "__id__": "mgrchannels_repo_clean_all"
        },
        "retcode": 0
      }
    }
  ]
}

Note how the result also contains "special" key, retcode, which maps to a single number.

This makes it difficult to write exact parsers for the JSON structure, as we experienced in the salt-netapi-client project, where such a structure is translated into a Map<String, ApplyResult> object - mapping fails when retcode is found as it maps to a number and not an object.

Part of the reason why this is challenging is due to the fact retcode is not added to some "external" structure (in the example above: in the "return" map), but together with legitimate minion results. For other calls, such as cmd.run_all, this even conflicts with an existing returned key retcode:

{
  "return": [
    {
      "minion-8.tf.local": {
        "pid": 15246,
        "retcode": 0,
        "stderr": "",
        "stdout": "bin\nsalt"
      }
    }
  ]
}

It is also challenging because retcode is only added:

• if the local_batch client is used
• if the result is an object
• if the result didn't include its own retcode already

Code responsible for this behavior is located in batch.py was added by #38668, specifically it is located in these lines:

https://github.com/saltstack/salt/blob/develop/salt/cli/batch.py#L310-L311

Setup

In /etc/salt/master.d/api.conf:

rest_cherrypy:
  port: 9080
  host: 0.0.0.0
  collect_stats: false
  disable_ssl: true
  ssl_crt: /etc/pki/tls/certs/spacewalk.crt
  ssl_key: /etc/pki/tls/private/spacewalk.key

Sample request:

[                                                                                                                                                                                                 {
        "tgt":"MY_MINION",
        "metadata":{
            "batch-mode":true
        },
        "batch-size":"200",
        "eauth":"auto",
        "kwarg":{
        },
        "password":"",
        "tgt_type":"glob",
        "client":"local_batch",
        "batch_presence_ping_timeout":4,
        "batch_presence_ping_gather_job_timeout":1,
        "fun":"state.apply",
        "arg":[
            "cleanup_minion"
        ],
        "username":"admin"
    }
]

Steps to Reproduce Issue

To run the request: curl -X POST http://localhost:9080/run --data @request.json -H "Content-Type: application/json"

Versions Report

Tested on salt 2018.3.0 (Oxygen) and 2019.2

[moio]

CC @brejoc @lucidd

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[moio]

Not stale, please reopen

[stale]

Thank you for updating this issue. It is no longer marked as stale.

[MKLeb]

Hi @moio, I have tried to reproduce this and I couldn't. Have you been able to confirm this on the newest version of salt (3005.1)? If not, I am going to close this as fixed.

EDIT: I think I actually fixed this in a PR a while back... #61519

[moio]

Glad to hear the issue has been resolved! Thanks