bug: running quickget --list on macOS results grep errors

[rowdy-fms]

I confirm this bug has not already been reported

• I have searched the issues and this bug has not been reported previously

Describe the bug
When running quickget --list on macOS (Sonoma, on an M2 Mac Studio), it responds with error grep: invalid option -- P twice, then produces a long list of available operating systems and versions. Most of the entries in the list are correct, but some are nonsensical, sparkylinux in particular.

To Reproduce
Steps to reproduce the behavior:

1. Run quickget --list on macOS.
2. Observe the two grep errors, and the list of nonsensical versions for sparkylinux.

Expected behavior
A complete list of available operating systems with no grep errors and no nonsensical output.

Quickget output
Run quickemu or quickemu and include the output of the failure below:

quickget output

rowdy@silicon:quickemu$ ./quickget --list
grep: invalid option -- P
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
  [-e pattern] [-f file] [--binary-files=value] [--color=when]
  [--context[=num]] [--directories=action] [--label] [--line-buffered]
  [--null] [pattern] [file ...]
grep: invalid option -- P
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
  [-e pattern] [-f file] [--binary-files=value] [--color=when]
  [--context[=num]] [--directories=action] [--label] [--line-buffered]
  [--null] [pattern] [file ...]
OS Release Option
alma 8 boot
alma 8 dvd
alma 8 minimal
alma 9 boot
alma 9 dvd
alma 9 minimal
alpine v3.11 
alpine v3.12 
<snip lots of valid entries>
sparkylinux /> <meta
sparkylinux /> <meta
sparkylinux /> <meta
sparkylinux /> <meta
sparkylinux /> content="7
sparkylinux /> content="7
sparkylinux /> content="7
sparkylinux /> content="7
sparkylinux /> kde
sparkylinux /> kde
sparkylinux /> lxqt
sparkylinux /> lxqt
sparkylinux /> mate
sparkylinux /> mate
sparkylinux /> minimalcli
sparkylinux /> minimalcli
sparkylinux /> minimalgui
sparkylinux /> minimalgui
sparkylinux /> name="twitter:description"
sparkylinux /> name="twitter:description"
sparkylinux /> property="og:description"
sparkylinux /> property="og:description"

The rest of the list, from spirallinux (which appears after sparkylinux) onwards, appears to be valid. There are around 1536 lines of sparkylinux nonsense, similar to the above.

System information
Run quickreport and include the output here; if you can't run quickreport,
please provide the output of the following:

Quickreport output

----------------------------------
      Quickemu 4.9.6
----------------------------------
Distro:	macOS 14.5
Kernel:	Darwin 23.5.0 arm64
Memory:	64G
GPU:
-Apple M2 Max
----------------------------------
          curl 8.6.0
----------------------------------
Libraries: libcurl/8.6.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features:  alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets
----------------------------------
          QEMU 9.0.2
----------------------------------
Available CPUs:
a64fx
arm1026
arm1136
arm1136-r2
arm1176
arm11mpcore
arm926
arm946
cortex-a15
cortex-a35
cortex-a53
cortex-a55
cortex-a57
cortex-a7
cortex-a710
cortex-a72
cortex-a76
cortex-a8
cortex-a9
cortex-m0
cortex-m3
cortex-m33
cortex-m4
cortex-m55
cortex-m7
cortex-r5
cortex-r52
cortex-r5f
host
max
neoverse-n1
neoverse-n2
neoverse-v1
pxa250
pxa255
pxa260
pxa261
pxa262
pxa270-a0
pxa270-a1
pxa270
pxa270-b0
pxa270-b1
pxa270-c0
pxa270-c5
sa1100
sa1110
ti925t
----------------------------------
             CPU
----------------------------------
Apple M2 Max

Screenshots
n/a

Additional context
n/a

[lj3954]

Quickget uses pattern matching to find operating system releases, editions, and URLs, when such data can't be accessed via APIs. SparkyLinux appears to have modified their download page since quickget's release/edition detection for it was last updated.

[flexiondotorg]

Sparky Linux listings are changing frequently. I am inclined to unlist Sparky.

[lj3954]

Sparky Linux listings are changing frequently. I am inclined to unlist Sparky.

You're trying to parse data from a user facing page, one which is likely to change quite often. The direct mirror would likely be a better option. http://us2.repo.sparkylinux.org/iso/stable/

[philclifford]

#1406 should take care of Sparky errors so that just leaves the grep -- invalid option

[lj3954]

macOS ships BSD grep, which doesn't support perl expressions. All instances of grep -P must be replaced with grep -E to fix this.

[philclifford]

$ grep -n -e grep ../quickget | grep -e '-P'
845:    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -P '(?<=<Key>nixos-)[0-9]+.[0-9]+(?=</Key>)' | sort -nr | head -n +2)
882:    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" | grep -o -P '(?<=href=")[0-9].*(?=/")' | sort -nr | head -n 1)

[rowdy-fms]

I've tried simply replacing -P with -E (in both places), but that resulted in the following error appearing twice:

grep: repetition-operator operand invalid

My regex-fu is not sufficiently developed to resolve that error.

[lj3954]

I've tried simply replacing -P with -E (in both places), but that resulted in the following error appearing twice:

grep: repetition-operator operand invalid

My regex-fu is not sufficiently developed to resolve that error.

You can't simply replace the flag, the regex syntax is different.

[philclifford]

@rowdy-fms These tweaks could doubtless be improved, but they work and are fairly readable:

diff --git a/quickget b/quickget
index ab9cd3c174..4118932b98 100755
--- a/quickget
+++ b/quickget
@@ -842,7 +842,7 @@ function releases_nitrux() {
 function releases_nixos() {
     # Lists unstable plus the two most recent releases
     #shellcheck disable=SC2046
-    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -P '(?<=<Key>nixos-)[0-9]+.[0-9]+(?=</Key>)' | sort -nr | head -n +2)
+    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -E 'nixos-[[:digit:]]+\.[[:digit:]]+' | cut -d- -f2 | sort -nru | head -n +2)
 }
 
 function editions_nixos() {
@@ -879,7 +879,7 @@ function releases_oraclelinux() {
 
 function releases_parrotsec() {
     #shellcheck disable=SC2046,SC2005
-    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" | grep -o -P '(?<=href=")[0-9].*(?=/")' | sort -nr | head -n 1)
+    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" |  grep -o -E 'href="[[:digit:]]\.[[:digit:]]+' | sort -nr | head -n 3 | cut -d\" -f 2 )
 }
 
 function editions_parrotsec() {