Ubuntu failed to scan vulnerable packages

[william20111]

[Oct  6 14:27:33] DEBUG [x-x-x-x] SSHResult: servername: x-x-x-x, cmd: set -o pipefail; sudo -S LANG=en_US.UTF-8 apt-get upgrade --dry-run, exitstatus: 0, stdout: ^MReading package lists... 0%^MReading package lists... 100%^MReading package lists... Done
^MBuilding dependency tree... 0%^MBuilding dependency tree... 0%^MBuilding dependency tree... 50%^MBuilding dependency tree... 50%^MBuilding dependency tree       
^MReading state information... 0%^MReading state information... 0%^MReading state information... Done
^MCalculating upgrade... 0%^MCalculating upgrade... 50%^MCalculating upgrade... Done
The following packages were automatically installed and are no longer required:
  linux-headers-3.19.0-65 linux-headers-3.19.0-65-generic linux-headers-4.4.0-34 linux-headers-4.4.0-34-generic linux-image-3.19.0-65-generic linux-image-3.19.0-66-generic linux-image-4.4.0-34-generic linux-image-extra-3.19.0-65-generic
  linux-image-extra-4.4.0-34-generic linux-image-generic-lts-vivid
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  initramfs-tools initramfs-tools-bin initramfs-tools-core klibc-utils libklibc ntp ntpdate
7 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.
Inst ntp [1:4.2.8p4+dfsg-3ubuntu5.1] (1:4.2.8p4+dfsg-3ubuntu5.3 Ubuntu:16.04/xenial-updates, Ubuntu:16.04/xenial-security [amd64])
Inst ntpdate [1:4.2.8p4+dfsg-3ubuntu5.1] (1:4.2.8p4+dfsg-3ubuntu5.3 Ubuntu:16.04/xenial-updates, Ubuntu:16.04/xenial-security [amd64])
Inst klibc-utils [2.0.4-8ubuntu1.16.04.1] (2.0.4-8ubuntu1.16.04.2 Ubuntu:16.04/xenial-updates [amd64]) []
Inst initramfs-tools [0.122ubuntu8.1] (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [all]) []
Inst initramfs-tools-core [0.122ubuntu8.1] (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [all]) []
Inst initramfs-tools-bin [0.122ubuntu8.1] (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [amd64]) []
Inst libklibc [2.0.4-8ubuntu1.16.04.1] (2.0.4-8ubuntu1.16.04.2 Ubuntu:16.04/xenial-updates [amd64])
Conf ntp (1:4.2.8p4+dfsg-3ubuntu5.3 Ubuntu:16.04/xenial-updates, Ubuntu:16.04/xenial-security [amd64])
Conf ntpdate (1:4.2.8p4+dfsg-3ubuntu5.3 Ubuntu:16.04/xenial-updates, Ubuntu:16.04/xenial-security [amd64])
Conf libklibc (2.0.4-8ubuntu1.16.04.2 Ubuntu:16.04/xenial-updates [amd64])
Conf klibc-utils (2.0.4-8ubuntu1.16.04.2 Ubuntu:16.04/xenial-updates [amd64])
Conf initramfs-tools-bin (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [amd64])
Conf initramfs-tools-core (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [all])
Conf initramfs-tools (0.122ubuntu8.3 Ubuntu:16.04/xenial-updates [all])
, stderr: , err: %!s(<nil>)
[Oct  6 14:27:33] ERROR [x-x-x-x] Failed to scan vulnerable packages

I have tested on both 16.04 and 14.04 with the same results. I can provide any other information required.

Cheers
Will

[william20111]

Also on the cli this is the final error message.

[Oct 6 14:27:33] ERROR [localhost] Failed to scan. err: user@x.x.x.x:22: Failed to scan upgradable packages

[william20111]

func (o *debian) parseAptGetUpgrade(stdout string) (upgradableNames []string, err error) {
    startRe := regexp.MustCompile(`The following packages will be upgraded:`)
    stopRe := regexp.MustCompile(`^(\d+) upgraded.*`)
    startLineFound, stopLineFound := false, false

    lines := strings.Split(stdout, "\n")
    for _, line := range lines {
        if !startLineFound {
            if matche := startRe.MatchString(line); matche {
                startLineFound = true
            }

            continue
        }
        result := stopRe.FindStringSubmatch(line)
        if len(result) == 2 {
            numUpgradablePacks, err := strconv.Atoi(result[1])
            if err != nil {
                return nil, fmt.Errorf(
                    "Failed to scan upgradable packages number. line: %s", line)
            }
            if numUpgradablePacks != len(upgradableNames) {
                return nil, fmt.Errorf(
                    "Failed to scan upgradable packages, expected: %s, detected: %d",
                    result[1], len(upgradableNames))
            }
            stopLineFound = true
            o.log.Debugf("Found the stop line. line: %s", line)
            break
        }
        upgradableNames = append(upgradableNames, strings.Fields(line)...)
    }
    if !startLineFound {
        // no upgrades
        return
    }
    if !stopLineFound {
        // There are upgrades, but not found the stop line.
        return nil, fmt.Errorf("Failed to scan upgradable packages")
    }
    return
}

Bug is in the stopline regex.

I have a bunch of servers on different apt versions. The output is slightly different on the stop line. so on a kali linux VM that i had lying around it has version 1.3 of apt and that works fine and finds the stop line. Using an LTS version of ubuntu on 14.04 & 16.04 they both failed because the versions of apt are different and the stop line outputs slightly differently.

stopRe := regexp.MustCompile(`^(\d+) to upgrade.*`)

Above is a working regex for myself. because the output looks like this for example

10 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.

I think more recent versions of apt output like this.

36 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

[william20111]

I will put a pull request in for a fix with older versions of apt.

Cheers
Will

[kotakanbe]

Thanks:)
I will check it out later.

[kotakanbe]

I tried to reproduce the bug, but I couldn't.
I checked the output of apt-get upgrade of apt v1.1, 1.2 and 1.3, but all are the same.

• apt 1.3 on Linux kali 4.6.0-kali1-amd64

root@kali:~# apt -v
apt 1.3~rc4 (amd64)

# LANGUAGE=en_US.UTF-8 apt-get upgrade --dry-run
248 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.

• apt 1.2.6 on Ubuntu 16.04

root@be5ade205d00:/# apt -v
apt 1.2.6 (amd64)

# LANGUAGE=en_US.UTF-8 apt-get upgrade --dry-run
99 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.

• apt 1.1.10 on Ubuntu 16.04

root@6e1cc335bd92:/# apt -v
apt 1.1.10 (amd64)

# LANGUAGE=en_US.UTF-8 apt-get upgrade --dry-run
128 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.

[kotakanbe]

Has anyone run into the same problem?

[william20111]

apt 1.0.1ubuntu2
14.04.4 LTS

140 to upgrade, 0 to newly install, 0 to remove and 6 not to upgrade.

[kotakanbe]

Thanks.
I will try it out tomorrow.

[kotakanbe]

• apt-get 0.8.16 on Ubuntu12.04

root@6a22b2e69970:/# apt-get -v
apt 0.8.16~exp12ubuntu10.27 for amd64 compiled on May 18 2016 09:51:57

root@6a22b2e69970:/# apt-get upgrade --dry-run
26 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

• apt-get 1.0.1 on Ubuntu 14.04

root@fdf4c36e44a9:/# apt -v
apt 1.0.1ubuntu2 for amd64 compiled on Oct 28 2014 20:55:14

root@fdf4c36e44a9:/# apt-get upgrade --dry-run
58 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[kotakanbe]

Hi, @william20111

I can not reproduce the bug...
(But I found the blog post)
http://landoflinux.com/linux_package_management_apt-get.html

So, I will promote the work in the test code.
I will check and comment to your P/R.
Thanks.

[william20111]

Thanks. Yes this is very confusing. Im going to test multiple apt releases to figure out why this is happening....

[wnoguchi]

I have same problem.
IP Addr is fake.
It seems to lookup distribution kind.
I use bastion server.

Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-35-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Tue Dec 13 03:16:50 JST 2016

  System load:  0.0               Processes:           124
  Usage of /:   8.7% of 90.43GB   Users logged in:     0
  Memory usage: 7%                IP address for eth0: 198.51.100.121
  Swap usage:   0%

  Graph this data and manage this system at:
    https://landscape.canonical.com/

62 packages can be updated.
0 updates are security updates.

New release '16.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

*** システムの再起動が必要です ***
Last login: Mon Dec 12 19:53:33 2016 from 198.51.100.123

vuls prepare output here

$ vuls prepare -ssh-external -debug example-target-host
INFO[0000] Start Preparing (config: /opt/vuls/config.toml) 
INFO[0000] Validating Config...                         
[Dec 13 14:59:48]  INFO [localhost] Detecting OS... 
[Dec 13 14:59:48]  INFO [localhost] Detecting OS of servers... 
[Dec 13 14:59:54] DEBUG [localhost] SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/debian_version, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:54] DEBUG [localhost] Not Debian like Linux. SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/debian_version, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:55] DEBUG [localhost] SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/fedora-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:56] DEBUG [localhost] SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/redhat-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:57] DEBUG [localhost] SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/system-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:57] DEBUG [localhost] Not RedHat like Linux. servername: example-target-host
[Dec 13 14:59:58] DEBUG [localhost] SSHResult: servername: example-target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no uname, exitstatus: 0, stdout: Linux
, stderr: Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 14:59:58] DEBUG [localhost] Not FreeBSD. servernam: example-target-host
[Dec 13 14:59:58] ERROR [localhost] (1/1) Failed: example-target-host, err: [Unknown OS Type]
[Dec 13 14:59:58] ERROR [localhost] Failed to init servers: No scannable servers

vuls scan output here

$ vuls prepare output here
  vuls scan \
    -report-json \
    -report-mail \
    -lang=ja \
    -cve-dictionary-dbpath=$PWD/cve.sqlite3 \
    -ssh-external \
    -debug example-target-host

INFO[0000] Start scanning                               
INFO[0000] config: /opt/vuls/config.toml                
INFO[0000] cve-dictionary: /opt/vuls/cve.sqlite3        
DEBU[0000] {
  "target-host": config.ServerInfo{
    ServerName:             "target-host",
    User:                   "vuls",
    Host:                   "test.example.com",
    Port:                   "22",
    KeyPath:                "/var/lib/jenkins/.ssh/id_rsa",
    KeyPassword:            "",
    CpeNames:               []string{},
    DependencyCheckXMLPath: "",
    Containers:             []string{},
    IgnoreCves:             []string{},
    Optional:               [][]interface {}{},
    Enablerepo:             "",
    LogMsgAnsiColor:        "\x1b[32m",
    Container:              config.Container{
      ContainerID: "",
      Name:        "",
      Type:        "",
    },
    Distro: config.Distro{
      Family:  "",
      Release: "",
    },
  },
} 
[Dec 13 15:05:41]  INFO [localhost] Validating Config...
DEBU[0000] get cve-dictionary from sqlite3              
[Dec 13 15:05:41]  INFO [localhost] Detecting Server/Contianer OS... 
[Dec 13 15:05:41]  INFO [localhost] Detecting OS of servers... 
[Dec 13 15:05:47] DEBUG [localhost] SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/debian_version, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:47] DEBUG [localhost] Not Debian like Linux. SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/debian_version, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:48] DEBUG [localhost] SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/fedora-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:49] DEBUG [localhost] SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/redhat-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:50] DEBUG [localhost] SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no set -o pipefail; ls /etc/system-release, exitstatus: 2, stdout: , stderr: sh: 1: set: Illegal option -o pipefail
Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:50] DEBUG [localhost] Not RedHat like Linux. servername: target-host
[Dec 13 15:05:51] DEBUG [localhost] SSHResult: servername: target-host, cmd: /bin/ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ControlMaster=no -o ControlPath=none vuls@test.example.com -p 22 -i /var/lib/jenkins/.ssh/id_rsa -o PasswordAuthentication=no uname, exitstatus: 0, stdout: Linux
, stderr: Killed by signal 1.
, err: %!s(<nil>)
[Dec 13 15:05:51] DEBUG [localhost] Not FreeBSD. servernam: target-host
[Dec 13 15:05:51] ERROR [localhost] (1/1) Failed: target-host, err: [Unknown OS Type]
[Dec 13 15:05:51] ERROR [localhost] Failed to init servers: No scannable servers

[wnoguchi]

oh, sorry...
my problem caused by ssh user shell is sh (not bash).
thus, my problem may not related to this issue...

sh: 1: set: Illegal option -o pipefail

/bin/sh has no pipefail option support.

$ echo $SHELL
/bin/sh

then change shell to bash, works fine.

sudo chsh -s /bin/bash maintenance

$ echo $SHELL
/bin/bash

works fine.

Thanks.