Auto generate actions

[lampwins]

Pack overhaul to now use an action auto-generation script. The script in /bin uses the NetBox OpenAPI (Swagger) spec to generate all pack actions (per API endpoint).

cc: @namachieli

[namachieli]

Installed this on a test instance of stackstorm (2.9.0-7) and pointed at a test instance of netbox (2.4.6)

• GET, POST, and PUT appear to be working well

  • I tried creating new objects (device, device_type, site, providers, circuits, racks)
  • I then confirmed that GET and PUT were working as well for those objects

• PATCH

  • PATCH required fields are identical to PUT required fields, even though PATCH should only require the id of the object being patched Fixed
    • This is more of a problem with the API being the source for the generated .yaml actions, and that reference marking items are required, even though they are not for PATCH.
    • This is likely something that needs to be updated in the Netbox API and then these actions regenerated so that PATCH actually behaves like PATCH and not PUT

• DELETE

  • This is not working for all objects I tried creating above Fixed
    • I confirmed the API key I created was for the local administrative user and should have ALL permissions
    • Action output: (Same/similar for all objects)

{
  "result": "None",
  "exit_code": 1,
  "stderr": "st2.actions.python.NetboxHTTPAction: DEBUG    Calling base delete with kwargs: {u'id': 2}\n/opt/stackstorm/st2/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\nTraceback (most recent call last):\n  File \"/opt/stackstorm/runners/python_runner/python_runner/python_action_wrapper.py\", line 320, in <module>\n    obj.run()\n  File \"/opt/stackstorm/runners/python_runner/python_runner/python_action_wrapper.py\", line 179, in run\n    output = action.run(**self._parameters)\n  File \"/opt/stackstorm/packs/netbox/actions/run.py\", line 38, in run\n    result = self.make_request(endpoint_uri, http_verb, **kwargs)\n  File \"/opt/stackstorm/packs/netbox/actions/lib/action.py\", line 69, in make_request\n    return {'raw': r.json()}\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/requests/models.py\", line 885, in json\n    return complexjson.loads(self.text, **kwargs)\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/simplejson/__init__.py\", line 518, in loads\n    return _default_decoder.decode(s)\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/simplejson/decoder.py\", line 370, in decode\n    obj, end = self.raw_decode(s)\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/simplejson/decoder.py\", line 400, in raw_decode\n    return self.scan_once(s, idx=_w(s, idx).end())\nsimplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n",
  "stdout": ""
}

Otherwise this is looking good. Thank you for working on this!

[namachieli]

Tested DELETE today on a fresh 2.9.1-3 st2 instance.

I'm getting the same behavior, but i noticed something else.

When I run a delete action against an ID that does not exist, the action succeeds with a message of Not Found. So perhaps the DELETE action is having issues processing data received when successfully deleting a resource on netbox.

Success output:

{
  "result": {
    "raw": {
      "detail": "Not found."
    }
  },
  "exit_code": 0,
  "stderr": "",
  "stdout": ""
}

[namachieli]

Tested DELETE on device, Rack, provider, site, circuit.

All completed successfully and the resource was deleted.

lgtm

[lampwins]

@namachieli thank you very much for your continued testing efforts!

@LindsayHill would you mind giving this a once-over? I am interested in your thoughts on conveying the backward incompatible changes (I have bumped the pack version 1.0.0 as such). The interesting stuff is in /bin and then the readme/pack.yaml/etc.

[LindsayHill]

Change looks good. Only concern I have is about number of actions. There's work going on to speed up action registration, but things still get a bit ugly with large packs.

Hopefully they won't be adding large amounts of new endpoints to the netbox API

[lampwins]

Thanks! We do have some new things in the pipeline for NetBox obviously, but I don't see us ever getting as bad as the old aws pack :) It is a good point to be conscious of moving forward.