juniper_junos_config - Unexpected XML tag returned on grabbing a config in set commands

[eregimbald]

Ansible: 2.4.2.0
Python: 2.7.12
Junos-eznc: 2.1.7
Junos-netconify: 1.0.2
jxmlease: 1.0.1

I have a playbook that grabs the configuration from a list of devices and then parses them for a specific keyword. On certain devices I get the error "Unexpected XML tag returned".

Here's a partial -vvv output. The full version also includes the device's config in XML but I'd prefer to share this through email.

https://hastebin.com/metugibivo.xml

[stacywsmith]

@eregimbald It would be helpful to see the full unedited version of the output. Could you please share that with me via private email to stacys at juniper dot net?

[stacywsmith]

Private email confirmed that the device returned the configuration in "xml" format even though "set" format was requested. In addition, the device was confirmed to be running Junos 12.3.

According to this URL, downloading the configuration in "set" format is only supported in Junos versions >= 15.1:

The documentation for the "format" argument states:

Specifies the format of the configuration retrieved, if retrieve is not none.
Specifies the format of the configuration to be loaded, if load is not none.
The specified format must be supported by the target Junos device.

I'm afraid it's not possible to retrieve the config in set format on this device that's running Junos 12.3.

[eregimbald]

Thanks, sorry I missed that.

…

On Tue, Jan 16, 2018 at 6:14 PM, Stacy W. Smith ***@***.***> wrote: Closed #326 <#326>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#326 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATI79C5QOhNUKkHHoHErcYEN54n0a5OOks5tLS1EgaJpZM4Rgeaa> .

-- Emmanuel Régimbald - Data Center Engineer 613-699-7761 | eregimbald@fuze.com 436 Hazeldean Road, Suite 202 | Kanata, ON, K2L 1T9

-- *Confidentiality Notice: The information contained in this e-mail and any attachments may be confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.*

[stacywsmith]

@eregimbald Here's a potential workaround.

The juniper_junos_command module lets you execute show configuration | display set which does work on pre 15.1 Junos versions.

If you need to save the config to a file, use the dest or dest_dir options. If you need to use the config in later tasks, use the stdout string or stdout_lines list in the response as shown here:

user@h0:~/$ cat pb.juniper_junos_command_config_set.yaml 
---
- name: 'Get configuration in set format using CLI'
  hosts: junos-all
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  tasks:
    - name: 'Get config in set format'
      juniper_junos_command:
        command: 'show configuration | display set'
      register: response
    - name: 'Print the first set line'
      debug:
        var: response.stdout_lines[1]

and running this playbook:

user@h0:~/$ ansible-playbook --user root -k --limit sw1 pb.juniper_junos_command_config_set.yaml
SSH password: 

PLAY [Get configuration in set format using CLI] ***********************************************************************

TASK [Get config in set format] ****************************************************************************************
ok: [sw1]

TASK [Print the first set line] ****************************************************************************************
ok: [sw1] => {
    "response.stdout_lines[1]": "set version 12.3R12.4"
}

PLAY RECAP *************************************************************************************************************
sw1                        : ok=2    changed=0    unreachable=0    failed=0   

[eregimbald]

@stacywsmith

This is interesting because the documentation for juniper_junos_command specifically states that "pipe modifiers" are not supported. I tried using this earlier this week and was hit with this type of error.

  eregimbald@eregimbald-ubuntu:/media/sf_ansibleshare/network-ansible$ ansible-playbook send-cli.yml
  Enter host or inventory group: sw-edge-bos-d
  Enter your command (string or list) [request snmp spoof-trap jnxSyslogTrap]: show configuration | grep hello

  PLAY [Send CLI command] ********************************************************************************************************************************************************************

  TASK [file] ********************************************************************************************************************************************************************************
  ok: [sw-edge-bos-d]

  TASK [file] ********************************************************************************************************************************************************************************
  ok: [sw-edge-bos-d]

  TASK [Junos CLI command] *******************************************************************************************************************************************************************
  fatal: [sw-edge-bos-d]: FAILED! => {"changed": false, "msg": "The pipe modifier (| grep hello) in the command (show configuration| grep hello) is not supported."}
          to retry, use: --limit @/media/sf_ansibleshare/network-ansible/send-cli.retry

  PLAY RECAP *********************************************************************************************************************************************************************************
  sw-edge-bos-d              : ok=2    changed=0    unreachable=0    failed=1

It seems that | display set is an exception.

That's fine, but instead of registering the results of the command, my playbook dumps the command return into a file. However when I use "| display set", even though the command is successful, Ansible can't create a file with a pipe (|) in its name because it's an invalid character.

eregimbald@eregimbald-ubuntu:/media/sf_ansibleshare/network-ansible$ ansible-playbook send-cli.yml
Enter host or inventory group: sw-edge-bos-d
Enter your command (string or list) [request snmp spoof-trap jnxSyslogTrap]: show configuration | display set

PLAY [Send CLI command] ********************************************************************************************************************************************************************

TASK [file] ********************************************************************************************************************************************************************************
ok: [sw-edge-bos-d]

TASK [file] ********************************************************************************************************************************************************************************
ok: [sw-edge-bos-d]

TASK [Junos CLI command] *******************************************************************************************************************************************************************
fatal: [sw-edge-bos-d]: FAILED! => {"changed": false, "msg": "Unable to save output. Failed to open the vars/send-cli/sw-edge-bos-d_show_configuration_|_display_set.text file."}
        to retry, use: --limit @/media/sf_ansibleshare/network-ansible/send-cli.retry

PLAY RECAP *********************************************************************************************************************************************************************************
sw-edge-bos-d              : ok=2    changed=0    unreachable=0    failed=1

[eregimbald]

The playbook in question:

---
- name: Send CLI command
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  vars_prompt:
    - name: "target"
      prompt: "Enter host or inventory group"
      private: no
    - name: command
      prompt: Enter your command (string or list)
      private: no
      default: "request snmp spoof-trap jnxSyslogTrap"

  hosts: "{{target}}"

  tasks:
    - file: path=vars/send-cli/ state=directory
    - file: path=logs/send-cli/ state=directory
    - name: Junos CLI command
      juniper_junos_command:
        host: '{{ inventory_hostname }}'
        user: '{{ user }}'
        commands: '{{ command }}'
        logfile: logs/send-cli/{{ inventory_hostname }}.log
        dest_dir: vars/send-cli/  

[stacywsmith]

This is interesting because the documentation for juniper_junos_command specifically states that "pipe modifiers" are not supported.

This is generally true, and it's not a limitation of the Ansible module, but a limitation of Junos. As explained in the documentation, this module doesn't actually execute a command at the CLI. Instead, it uses the <command> RPC over a NETCONF session. The <command> RPC itself doesn't support most pipe modifiers. The only exception is | display <format>. However, most of the time you don't even need to do | display <format>, since the format: <format> argument to this module is generally equivalent to | display <format>.

The only exception is specifically for | display set and the only operational-mode command which accepts | display set is show configuration. This workaround was specifically allowed to overcome the limitation of not being able to retrieve the config in set format on pre 15.1 releases. If it's of interest the code is here

However when I use "| display set", even though the command is successful, Ansible can't create a file with a pipe (|) in its name because it's an invalid character.

We should probably fix that issue by translating a | into _ when creating the filename...

In the meantime, you could use the older dest option to specify the exact filename. Something like:

---
- name: 'Get configuration in set format using CLI'
  hosts: junos-all
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  tasks:
    - name: 'Get config in set format'
      juniper_junos_command:
        command: 'show configuration | display set'
        dest: './output/{{ inventory_hostname }}.config.set'

[eregimbald]

Makes sense, thanks.

It seems like the older dest option has more functionality than dest-dir, why not allow dest-dir to include a specific filename?

[eregimbald]

Should I create an issue for the pipe in the filename?

…

On Wed, Jan 17, 2018 at 2:08 PM, Stacy W. Smith ***@***.***> wrote: This is interesting because the documentation for juniper_junos_command specifically states that "pipe modifiers" are not supported. This is generally true, and it's not a limitation of the Ansible module, but a limitation of Junos. As explained in the documentation, this module doesn't actually execute a command at the CLI. Instead, it uses the <command> RPC over a NETCONF session. The <command> RPC itself doesn't support most pipe modifiers. The only exception is | display <format>. However, most of the time you don't even need to do | display <format>, since the format: <format> argument to this module is generally equivalent to | display <format>. The only exception is specifically for | display set and the only operational-mode command which accepts | display set is show configuration. This workaround was specifically allowed to overcome the limitation of not being able to retrieve the config in set format on pre 15.1 releases. If it's of interest the code is here <https://github.com/Juniper/ansible-junos-stdlib/blob/master/library/juniper_junos_command.py#L399-L416> However when I use "| display set", even though the command is successful, Ansible can't create a file with a pipe (|) in its name because it's an invalid character. We should probably fix that issue by translating a | into _ when creating the filename... In the meantime, you could use the older dest option to specify the exact filename. Something like: --- - name: 'Get configuration in set format using CLI' hosts: junos-all connection: local gather_facts: no roles: - Juniper.junos tasks: - name: 'Get config in set format' juniper_junos_command: command: 'show configuration | display set' dest: './output/{{ inventory_hostname }}.config.set' — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#326 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATI79KjKPlKhwnq8chUhxqVB1AbBmLsYks5tLkUggaJpZM4Rgeaa> .

-- Emmanuel Régimbald - Data Center Engineer 613-699-7761 | eregimbald@fuze.com 436 Hazeldean Road, Suite 202 | Kanata, ON, K2L 1T9

-- *Confidentiality Notice: The information contained in this e-mail and any attachments may be confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.*

[stacywsmith]

A new issue for | in filename is not necessary. I have #328 in progress to address it.

[stacywsmith]

It seems like the older dest option has more functionality than dest-dir, why not allow dest-dir to include a specific filename?

In what way do you feel dest has more functionality? I think they currently compliment each other, but I do welcome feedback.

The dest argument requires the user to specify a specific filename. It's the users responsibility to ensure that the filename is unique within their playbook.

The dest_dir argument requires the user to specify a directory and assumes a filename of {{ inventory_hostname }}_command.format within that directory. It is the user's responsibility to ensure the directory + generated filename is unique within their playbook.

You can use variable substitution in the value of both the dest and dest_dir arguments.

The juniper_junos_command module now supports a list of commands like this:

---
- name: 'Execute multiple CLI commands and save the output'
  hosts: junos-all
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  tasks:
    - name: 'Execute CLI commands'
      juniper_junos_command:
        commands:
          - 'show version'
          - 'show interfaces terse'
          - 'show lldp neighbor'
      register: response
    - name: 'Print the response'
      debug:
        var: response

The difference between dest and dest_dir is important when multiple commands are specified.

This playbook with the dest argument will cause the output of all three commands to appear in a single file per {{ inventory_host }} (the output of each command is appended within the same file):

---
- name: 'Execute multiple CLI commands and save the output'
  hosts: junos-all
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  tasks:
    - name: 'Execute CLI commands'
      juniper_junos_command:
        commands:
          - 'show version'
          - 'show interfaces terse'
          - 'show lldp neighbor'
        dest: './output/{{ inventory_hostname }}.showcmds.txt'
      register: response
    - name: 'Print the response'
      debug:
        var: response

This playbook with the dest_dir argument will cause the output of each command to appear in a separate file within the ./output/ directory:

---
- name: 'Execute multiple CLI commands and save the output'
  hosts: junos-all
  connection: local
  gather_facts: no
  roles:
    - Juniper.junos

  tasks:
    - name: 'Execute CLI commands'
      juniper_junos_command:
        commands:
          - 'show version'
          - 'show interfaces terse'
          - 'show lldp neighbor'
        dest_dir: './output/'
      register: response
    - name: 'Print the response'
      debug:
        var: response

The idea was to allow the user to get the output from multiple commands in either a single file or in multiple files depending on which they preferred for a particular situation.

Yes, the current functionality does not allow the user to get the output of multiple commands in multiple files AND control the specific filename. In order to have supported that capability, it would have either required dest to accept a list value (one filename per command) or it would have required an option that accepted some sort of "templated" filename. I just felt like both of those options were too complicated for the benefit.

Again, I welcome your feedback if you've got suggestions on improving this behavior.

[eregimbald]

I had not considered the outcome of using multiple commands. I'm reluctant to use the "dest" option because the documentation states that it's deprecated, so I can't help but wonder if one day it will be completely removed and I'll have to retroactively update my playbooks. I try to make everything work with the new way of doing things. If you plan on keeping the old "dest" option indefinitely perhaps clarifying the documentation to illustrate the pros and cons of both options could be a good idea. By the way, I'm very thankful for the work your team has been doing. I was using a different set of modules until I discovered this new generation of juniper_junos modules and they work very well for what I'm trying to accomplish.

…

On Wed, Jan 17, 2018 at 4:16 PM, Stacy W. Smith ***@***.***> wrote: It seems like the older dest option has more functionality than dest-dir, why not allow dest-dir to include a specific filename? In what way do you feel dest has more functionality? I think they currently compliment each other, but I do welcome feedback. The dest argument requires the user to specify a specific filename. It's the users responsibility to ensure that the filename is unique within their playbook. The dest_dir argument requires the user to specify a directory and assumes a filename of {{ inventory_hostname }}_command.format within that directory. It is the user's responsibility to ensure the directory + generated filename is unique within their playbook. You can use variable substitution in the value of both the dest and dest_dir arguments. The juniper_junos_command module now supports a list of commands like this: --- - name: 'Execute multiple CLI commands and save the output' hosts: junos-all connection: local gather_facts: no roles: - Juniper.junos tasks: - name: 'Execute CLI commands' juniper_junos_command: commands: - 'show version' - 'show interfaces terse' - 'show lldp neighbor' register: response - name: 'Print the response' debug: var: response The difference between dest and dest_dir is important when multiple commands are specified. This playbook with the dest argument will cause the output of all three commands to appear in a single file per {{ inventory_host }} (the output of each command is appended within the same file): --- - name: 'Execute multiple CLI commands and save the output' hosts: junos-all connection: local gather_facts: no roles: - Juniper.junos tasks: - name: 'Execute CLI commands' juniper_junos_command: commands: - 'show version' - 'show interfaces terse' - 'show lldp neighbor' dest: './output/{{ inventory_hostname }}.showcmds.txt' register: response - name: 'Print the response' debug: var: response This playbook with the dest_dir argument will cause the output of each command to appear in a separate file within the ./output/ directory: --- - name: 'Execute multiple CLI commands and save the output' hosts: junos-all connection: local gather_facts: no roles: - Juniper.junos tasks: - name: 'Execute CLI commands' juniper_junos_command: commands: - 'show version' - 'show interfaces terse' - 'show lldp neighbor' dest_dir: './output/' register: response - name: 'Print the response' debug: var: response The idea was to allow the user to get the output from multiple commands in either a single file or in multiple files depending on which they preferred for a particular situation. Yes, the current functionality does not allow the user to get the output of multiple commands in multiple files AND control the specific filename. In order to have supported that capability, it would have either required dest to accept a list value (one filename per command) or it would have required an option that accepted some sort of "templated" filename. I just felt like both of those options were too complicated for the benefit. Again, I welcome your feedback if you've got suggestions on improving this behavior. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#326 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATI79CAKDHJjWHGUCfsOJ8b39GQqtRu8ks5tLmMcgaJpZM4Rgeaa> .

-- Emmanuel Régimbald - Data Center Engineer 613-699-7761 | eregimbald@fuze.com 436 Hazeldean Road, Suite 202 | Kanata, ON, K2L 1T9

-- *Confidentiality Notice: The information contained in this e-mail and any attachments may be confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.*

[stacywsmith]

I'm reluctant to use the "dest" option because the documentation states that it's
deprecated, so I can't help but wonder if one day it will be completely
removed and I'll have to retroactively update my playbooks. I try to make
everything work with the new way of doing things. If you plan on keeping
the old "dest" option indefinitely perhaps clarifying the documentation to
illustrate the pros and cons of both options could be a good idea.

I'm sorry about the documentation in this area. The dest option should not be deprecated. As I just explained, there are valid use cases for both dest and dest_dir. I will improve the documentation to explain the pros and cons and remove the statement that says dest is deprecated.

By the way, I'm very thankful for the work your team has been doing. I was
using a different set of modules until I discovered this new generation of
juniper_junos modules and they work very well for what I'm trying to
accomplish.

Thanks for the positive feedback. I truly appreciate it.