Skip to content

Commit 06c32d8

Browse files
kaelemcssassoStefano Sassojcpvdmhellt
authored
Implement Scrapli 'core' and platform migration (#297)
* AOS-CX: Update CPU and MEM values for newer versions (#294) Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com> * backdoor to reset VR or specific VMs (#285) * backdoor to reset VR * option to reset specific VMs * give ocnos some time to boot in the login routine (#295) * Add vrnetlab base image * Add `cidfile` to gitignore * Implement Scrapli - Implement scrapli for telnet console and qemu monitor - Add scrapli for core funcs (wait_write, read_until, expect) - Add conditional use of scrapli via 'use_scrapli' var. Default is disabled - Add colours to logging - Log env vars - Log if transparent mgmt intf is in use - Log if scrapli is in use - Log overlay image creation - Log defined SMP and RAM * cat8kv: Migrate to Scrapli - Use Scrapli IOSXEDriver for config - Update install VM var name to 'cat8kv' from 'csr' - Fix installer class init so overlay image is only created once * cat9kv: Migrate to Scrapli - Remove license check - Send bootstrap config via day0/CVAC config (mounted file to cdrom) - Send startup config via Scrapli IOSXEDriver * csr1kv: Migrate to Scrapli - Use Scrapli IOSXEDriver for sending bootstrap and startup configs * xrv: Migrate to Scrapli - Use Scrapli IOSXRDriver to send bootstrap and startup configs * xrv: Add convert-image target in Makefile - Converts the qcow2 image into required vmdk format for vrnetlab via qemu-img. * xrv9k: Migrate to Scrapli - Use Scrapli IOSXRDriver for bootstrap and startup configs - Change class names to 'XRv9k' instead of 'XRv' - Explicitly wait for SDR baking to complete in install process - Remove call home/LC check * xrv: Remove env var printing * n9kv: Migrate to Scrapli - Use NXOSDriver for bootstrap and startup configs * nxos: Migrate to Scrapli - Use NXOSDriver for bootstrap and startup configs * vios: Migrate to Scrapli - Use IOSXEDriver for bootstrap and startup configs * vrnetlab: Support scrapli qemu monitor option for VM reset * vrnetlab: Move logging colour config outside of class init method * cat8kv: Fix logger warning (log.warning -> logger.warning) * vrnetlab: Remove scrpali logging import * Cisco devices: Add/tweak configuration saving: - vios, csr, cat8kv, cat9kv -- add configuration saving - XRv, XRv9k -- log configuration saving * xrv, xrv9k: Return to root at end of bootstrap cfg * vrnetlab: add bool formatter func * sros: Migrate to Scrapli - Use scrapli community 'nokia_sros' platform - Remove wait_write clean_buffer override - Check if tftpboot conifg exists *before* opening Scrapli connection - Log command outputs with 'DEBUG_SCRAPLI' env var (defaults to false) * Use kaelemc/scrapli_community in base image * Disable eager mode for config saving on Cisco devices * cat8kv: Migrate to CVAC configuration * cat8kv: add log message and block while generating cfg ISO * cat9kv: Migrate startup config to CVAC * Remove erroneous Scrapli Community submodule * sros: fix typos for classic CLI * csr: Migrate to CVAC * Switch back to scrapli/scrapli-community * added uv lock/venv and env file for pylance resolve sequence (#303) * added uv dep for scrapli * use ruff formatting * update base image with pinned scrapli community * close sros driver connection to invoke on_exit commands (quit-config) * added local deps * use uv in the base image * ruff formatting * use single const for scrapli timeout * Close the commandeered connection so the on close actions are run * Connection error log type from info->error * extracting image edit * persist bof and config after bootstrap config is applied and close sros con regardless if config was provided or not * Configure scrapli variant if startup config is classic If the startup-configuration provided is classic then the default configuration engine will be set to classic mode. In this case the scrapli device variant should also be set to classic so the scrapli magic can do it's thing with the correct prompt matching. * Use a global var to determine when to send classic configs As classic startup configurations are now supported for MD-CLI defaulting versions, the classic CLI will mean the default config engine is classic on node boot. In this commit all logic that determined when to send/not send config for classic versions is now replaced with a single 'classic_cfg' global variable. Most of the logic across the code had repeated statements checking if the version was <= 22 or magc. 'classic_cfg' is set to True in this case. Else it is False. * Use explicit `quit-config` and move persistBofAndConfig back to end of bootstrap. * Don't enforce MD-CLI on versions older than 19.x --------- Co-authored-by: Stefano Sasso <852093+ssasso@users.noreply.github.com> Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com> Co-authored-by: João Machado <63718541+jcpvdm@users.noreply.github.com> Co-authored-by: Roman Dodin <dodin.roman@gmail.com>
1 parent d139c83 commit 06c32d8

32 files changed

+1314
-1152
lines changed

‎.env

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
PYTHONPATH=".:./common"

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ cisco*.bin
77
*.xz
88
*.vmdk
99
*.iso
10+
*cidfile
1011

1112
.DS_Store
1213
*/.DS_Store

‎aoscx/README.md

+4-2
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,13 @@ docker tag vrnetlab/vr-aoscx:20210610000730 vrnetlab/vr-aoscx:10.07.0010
1616
Tested booting and responding to SSH:
1717

1818
* `ArubaOS-CX_10_12_0006.ova` (`arubaoscx-disk-image-genericx86-p4-20230531220439.vmdk`)
19+
* `ArubaOS-CX_10_13_0005.ova` (`arubaoscx-disk-image-genericx86-p4-20231110145644.vmdk`)
20+
* `ArubaOS-CX_10_14_1000.ova` (`arubaoscx-disk-image-genericx86-p4-20240731173624.vmdk`)
1921

2022
## System requirements
2123

22-
CPU: 2 core
24+
CPU: 4 core
2325

24-
RAM: 4GB
26+
RAM: 8GB
2527

2628
Disk: <1GB

‎aoscx/docker/launch.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ def __init__(self, hostname, username, password, conn_mode):
4747
logging.getLogger().info("Disk image was not found")
4848
exit(1)
4949
super(AOSCX_vm, self).__init__(
50-
username, password, disk_image=disk_image, ram=4096, cpu="host,level=9", smp="2"
50+
username, password, disk_image=disk_image, ram=8192, cpu="host,level=9", smp="4"
5151
)
5252
self.hostname = hostname
5353
self.conn_mode = conn_mode

‎build-base-image.sh

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/usr/bin/env bash
2+
# this script builds the vrnetlab base container image
3+
# that is used in the dockerfiles of the NOS images
4+
5+
set -e
6+
7+
if [ -z "$1" ]; then
8+
echo "Usage: $0 <version>"
9+
exit 1
10+
fi
11+
12+
sudo docker build -t ghcr.io/srl-labs/vrnetlab-base:$1 \
13+
-f vrnetlab-base.dockerfile .

‎c8000v/docker/Dockerfile

+1-17
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,4 @@
1-
FROM public.ecr.aws/docker/library/debian:bookworm-slim
2-
3-
ENV DEBIAN_FRONTEND=noninteractive
4-
5-
RUN apt-get update -qy \
6-
&& apt-get install -y \
7-
bridge-utils \
8-
iproute2 \
9-
socat \
10-
qemu-kvm \
11-
tcpdump \
12-
inetutils-ping \
13-
ssh \
14-
telnet \
15-
procps \
16-
genisoimage \
17-
&& rm -rf /var/lib/apt/lists/*
1+
FROM ghcr.io/srl-labs/vrnetlab-base:0.1.0
182

193
ARG VERSION
204
ENV VERSION=${VERSION}

‎c8000v/docker/launch.py

+124-132
Original file line numberDiff line numberDiff line change
@@ -52,38 +52,116 @@ def __init__(self, hostname, username, password, conn_mode, install_mode=False):
5252
logger.info("License found")
5353
self.license = True
5454

55-
super().__init__(username, password, disk_image=disk_image, ram=4096)
55+
super().__init__(
56+
username, password, disk_image=disk_image, ram=4096, use_scrapli=True
57+
)
5658
self.install_mode = install_mode
5759
self.hostname = hostname
5860
self.conn_mode = conn_mode
5961
self.num_nics = 9
6062
self.nic_type = "virtio-net-pci"
63+
self.image_name = "config.iso"
6164

6265
if self.install_mode:
63-
logger.trace("install mode")
64-
self.image_name = "config.iso"
65-
self.create_boot_image()
66-
67-
self.qemu_args.extend(["-cdrom", "/" + self.image_name])
68-
69-
def create_boot_image(self):
70-
"""Creates a iso image with a bootstrap configuration"""
71-
72-
with open("/iosxe_config.txt", "w") as cfg_file:
73-
if self.license:
74-
cfg_file.write("do clock set 13:33:37 1 Jan 2010\r\n")
75-
cfg_file.write("interface GigabitEthernet1\r\n")
76-
cfg_file.write("ip address 10.0.0.15 255.255.255.0\r\n")
77-
cfg_file.write("no shut\r\n")
78-
cfg_file.write("exit\r\n")
79-
cfg_file.write("license accept end user agreement\r\n")
80-
cfg_file.write("yes\r\n")
81-
cfg_file.write("do license install tftp://10.0.0.2/license.lic\r\n\r\n")
82-
cfg_file.write("license boot level network-premier addon dna-premier\r\n")
83-
cfg_file.write("platform console serial\r\n\r\n")
84-
cfg_file.write("do clear platform software vnic-if nvtable\r\n")
85-
cfg_file.write("do wr\r\n")
86-
cfg_file.write("do reload\r\n")
66+
self.logger.debug("Install mode")
67+
self.create_config_image(self.gen_install_config())
68+
else:
69+
cfg = self.gen_bootstrap_config()
70+
if os.path.exists(STARTUP_CONFIG_FILE):
71+
self.logger.info("Startup configuration file found")
72+
with open(STARTUP_CONFIG_FILE, "r") as startup_config:
73+
cfg += startup_config.read()
74+
else:
75+
self.logger.warning("User provided startup configuration is not found.")
76+
self.create_config_image(cfg)
77+
78+
self.qemu_args.extend(["-cdrom", "/" + self.image_name])
79+
80+
def gen_install_config(self) -> str:
81+
"""
82+
Returns the configuration to load in install mode
83+
"""
84+
85+
config = ""
86+
87+
if self.license:
88+
config += """do clock set 13:33:37 1 Jan 2010
89+
interface GigabitEthernet1
90+
ip address 10.0.0.15 255.255.255.0
91+
no shut
92+
exit
93+
license accept end user agreement
94+
yes
95+
do license install tftp://10.0.0.2/license.lic
96+
"""
97+
98+
config += """
99+
license boot level network-premier addon dna-premier
100+
platform console serial
101+
do clear platform software vnic-if nvtable
102+
do wr
103+
do reload
104+
"""
105+
106+
return config
107+
108+
def gen_bootstrap_config(self) -> str:
109+
"""
110+
Returns the system bootstrap configuration
111+
"""
112+
113+
v4_mgmt_address = vrnetlab.cidr_to_ddn(self.mgmt_address_ipv4)
114+
115+
return f"""hostname {self.hostname}
116+
username {self.username} privilege 15 password {self.password}
117+
ip domain name example.com
118+
!
119+
crypto key generate rsa modulus 2048
120+
!
121+
line con 0
122+
logging synchronous
123+
!
124+
line vty 0 4
125+
logging synchronous
126+
login local
127+
transport input all
128+
!
129+
ipv6 unicast-routing
130+
!
131+
vrf definition clab-mgmt
132+
description Containerlab management VRF (DO NOT DELETE)
133+
address-family ipv4
134+
exit
135+
address-family ipv6
136+
exit
137+
exit
138+
!
139+
ip route vrf clab-mgmt 0.0.0.0 0.0.0.0 {self.mgmt_gw_ipv4}
140+
ipv6 route vrf clab-mgmt ::/0 {self.mgmt_gw_ipv6}
141+
!
142+
interface GigabitEthernet 1
143+
description Containerlab management interface
144+
vrf forwarding clab-mgmt
145+
ip address {v4_mgmt_address[0]} {v4_mgmt_address[1]}
146+
ipv6 address {self.mgmt_address_ipv6}
147+
no shut
148+
exit
149+
!
150+
restconf
151+
netconf-yang
152+
netconf max-sessions 16
153+
netconf detailed-error
154+
!
155+
ip ssh server algorithm mac hmac-sha2-512
156+
ip ssh maxstartups 128
157+
!
158+
"""
159+
160+
def create_config_image(self, config):
161+
"""Creates a iso image with a installation configuration"""
162+
163+
with open("/iosxe_config.txt", "w") as cfg:
164+
cfg.write(config)
87165

88166
genisoimage_args = [
89167
"genisoimage",
@@ -93,7 +171,8 @@ def create_boot_image(self):
93171
"/iosxe_config.txt",
94172
]
95173

96-
subprocess.Popen(genisoimage_args)
174+
self.logger.debug("Generating boot ISO")
175+
subprocess.Popen(genisoimage_args).wait()
97176

98177
def bootstrap_spin(self):
99178
"""This function should be called periodically to do work."""
@@ -104,127 +183,40 @@ def bootstrap_spin(self):
104183
self.start()
105184
return
106185

107-
(ridx, match, res) = self.tn.expect(
108-
[b"Press RETURN to get started!", b"IOSXEBOOT-4-FACTORY_RESET"], 1
186+
(ridx, match, res) = self.con_expect(
187+
[b"CVAC-4-CONFIG_DONE", b"IOSXEBOOT-4-FACTORY_RESET"]
109188
)
110189
if match: # got a match!
111-
if ridx == 0: # login
112-
self.logger.debug("matched, Press RETURN to get started.")
113-
if self.install_mode:
114-
self.logger.debug("Now we wait for the device to reload")
115-
else:
116-
self.wait_write("", wait=None)
117-
118-
# run main config!
119-
self.bootstrap_config()
120-
# add startup config if present
121-
self.startup_config()
122-
# close telnet connection
123-
self.tn.close()
124-
# startup time?
125-
startup_time = datetime.datetime.now() - self.start_time
126-
self.logger.info("Startup complete in: %s", startup_time)
127-
# mark as running
128-
self.running = True
129-
return
190+
if ridx == 0 and not self.install_mode: # configuration applied
191+
self.logger.info("CVAC Configuration has been applied.")
192+
# close telnet connection
193+
self.scrapli_tn.close()
194+
# startup time?
195+
startup_time = datetime.datetime.now() - self.start_time
196+
self.logger.info("Startup complete in: %s", startup_time)
197+
# mark as running
198+
self.running = True
199+
return
130200
elif ridx == 1: # IOSXEBOOT-4-FACTORY_RESET
131201
if self.install_mode:
132202
install_time = datetime.datetime.now() - self.start_time
133203
self.logger.info("Install complete in: %s", install_time)
134204
self.running = True
135205
return
136206
else:
137-
self.log.warning("Unexpected reload while running")
207+
self.logger.warning("Unexpected reload while running")
138208

139209
# no match, if we saw some output from the router it's probably
140210
# booting, so let's give it some more time
141211
if res != b"":
142-
self.logger.trace("OUTPUT: %s", res.decode())
212+
self.write_to_stdout(res)
143213
# reset spins if we saw some output
144214
self.spins = 0
145215

146216
self.spins += 1
147217

148218
return
149219

150-
def bootstrap_config(self):
151-
"""Do the actual bootstrap config"""
152-
self.logger.info("applying bootstrap configuration")
153-
154-
v4_mgmt_address = vrnetlab.cidr_to_ddn(self.mgmt_address_ipv4)
155-
156-
self.wait_write("", None)
157-
self.wait_write("enable", wait=">")
158-
self.wait_write("configure terminal", wait=">")
159-
160-
self.wait_write(f"hostname {self.hostname}")
161-
self.wait_write(
162-
"username %s privilege 15 password %s" % (self.username, self.password)
163-
)
164-
if int(self.version.split(".")[0]) >= 16:
165-
self.wait_write("ip domain name example.com")
166-
else:
167-
self.wait_write("ip domain-name example.com")
168-
self.wait_write("crypto key generate rsa modulus 2048")
169-
170-
self.wait_write("ipv6 unicast-routing")
171-
172-
self.wait_write("vrf definition clab-mgmt")
173-
self.wait_write("description Containerlab management VRF (DO NOT DELETE)")
174-
self.wait_write("address-family ipv4")
175-
self.wait_write("exit")
176-
self.wait_write("address-family ipv6")
177-
self.wait_write("exit")
178-
self.wait_write("exit")
179-
180-
self.wait_write(f"ip route vrf clab-mgmt 0.0.0.0 0.0.0.0 {self.mgmt_gw_ipv4}")
181-
self.wait_write(f"ipv6 route vrf clab-mgmt ::/0 {self.mgmt_gw_ipv6}")
182-
183-
self.wait_write("interface GigabitEthernet1")
184-
self.wait_write("vrf forwarding clab-mgmt")
185-
self.wait_write(f"ip address {v4_mgmt_address[0]} {v4_mgmt_address[1]}")
186-
self.wait_write(f"ipv6 address {self.mgmt_address_ipv6}")
187-
self.wait_write("no shut")
188-
self.wait_write("exit")
189-
self.wait_write("restconf")
190-
self.wait_write("netconf-yang")
191-
self.wait_write("netconf max-sessions 16")
192-
# I did not find any documentation about this, but is seems like a good idea!?
193-
self.wait_write("netconf detailed-error")
194-
self.wait_write("ip ssh server algorithm mac hmac-sha2-512")
195-
self.wait_write("ip ssh maxstartups 128")
196-
197-
self.wait_write("line vty 0 4")
198-
self.wait_write("login local")
199-
self.wait_write("transport input all")
200-
self.wait_write("end")
201-
self.wait_write("copy running-config startup-config")
202-
self.wait_write("\r", "Destination")
203-
204-
def startup_config(self):
205-
"""Load additional config provided by user."""
206-
207-
if not os.path.exists(STARTUP_CONFIG_FILE):
208-
self.logger.trace(f"Startup config file {STARTUP_CONFIG_FILE} is not found")
209-
return
210-
211-
self.logger.trace(f"Startup config file {STARTUP_CONFIG_FILE} exists")
212-
with open(STARTUP_CONFIG_FILE) as file:
213-
config_lines = file.readlines()
214-
config_lines = [line.rstrip() for line in config_lines]
215-
self.logger.trace(f"Parsed startup config file {STARTUP_CONFIG_FILE}")
216-
217-
self.logger.info(f"Writing lines from {STARTUP_CONFIG_FILE}")
218-
219-
self.wait_write("configure terminal")
220-
# Apply lines from file
221-
for line in config_lines:
222-
self.wait_write(line)
223-
# End and Save
224-
self.wait_write("end")
225-
self.wait_write("copy running-config startup-config")
226-
self.wait_write("\r", "Destination")
227-
228220

229221
class C8000v(vrnetlab.VR):
230222
def __init__(self, hostname, username, password, conn_mode):
@@ -240,17 +232,17 @@ class C8000v_installer(C8000v):
240232
"""
241233

242234
def __init__(self, hostname, username, password, conn_mode):
243-
super(C8000v_installer, self).__init__(hostname, username, password, conn_mode)
235+
super(C8000v, self).__init__(username, password)
244236
self.vms = [
245237
C8000v_vm(hostname, username, password, conn_mode, install_mode=True)
246238
]
247239

248240
def install(self):
249241
self.logger.info("Installing C8000v")
250-
csr = self.vms[0]
251-
while not csr.running:
252-
csr.work()
253-
csr.stop()
242+
cat8kv = self.vms[0]
243+
while not cat8kv.running:
244+
cat8kv.work()
245+
cat8kv.stop()
254246
self.logger.info("Installation complete")
255247

256248

0 commit comments

Comments
 (0)