OS detection get unsuccessful in first run

[thinksabin]

What did you do? (required. The issue will be closed when not provided.)

vuls scan ubuntu

What did you expect to happen?

Complete scan successfully in first attempt (run)

What happened instead?

Couldn't determine OS of server. the step timeout. Or, we can cancel whole thing with ctrl+c and then again run same command "vuls scan ubuntu" and everything is fine 2nd time.

• Current Output
  freeze in line " INFO [localhost] Detecting OS of servers... "
  Please re-run the command using -debug and provide the output below.

when used --debug, the working is normal and fine sometimes. log below:

$ vuls scan -debug ubuntu
[Aug 21 09:11:01] INFO [localhost] Start scanning
[Aug 21 09:11:01] INFO [localhost] config: /home/ubuntu/config.toml
[Aug 21 09:11:01] DEBUG [localhost] {
"logpoint": config.ServerInfo{
ServerName: “ubuntu”,
User: "vuls",
Host: “xyz”,
Port: "22",
KeyPath: "/home/ubuntu/vuls_keys/id_rsa_vuls_az",
KeyPassword: "",
CpeNames: []string{},
DependencyCheckXMLPath: "",
Containers: config.Containers{
Type: "",
Includes: []string{},
Excludes: []string{},
},
IgnoreCves: []string{},
Optional: [][]interface {}{},
Enablerepo: []string{},
Type: "",
LogMsgAnsiColor: "\x1b[33m",
Container: config.Container{
ContainerID: "",
Name: "",
Image: "",
},
Distro: config.Distro{
Family: "",
Release: "",
},
IPv4Addrs: []string{},
IPv6Addrs: []string{},
},
}
[Aug 21 09:11:01] INFO [localhost] Validating config...
[Aug 21 09:11:01] INFO [localhost] Detecting Server/Container OS...
[Aug 21 09:11:01] INFO [localhost] Detecting OS of servers...
[Aug 21 09:11:01] DEBUG [localhost] Executing... ls /etc/debian_version
[Aug 21 09:16:01] ERROR [localhost] Timed out while detecting servers
[Aug 21 09:16:01] ERROR [localhost] (1/1) Timed out: ubuntu
[Aug 21 09:16:01] ERROR [localhost] Failed to init servers: No scannable servers

Steps to reproduce the behaviour

uncertain.
vuls scan ubuntu

Configuration (MUST fill this out):

• Go version (go version):
  go version go1.10.3 linux/amd64
• Go environment (go env):

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ubuntu/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build834591688=/tmp/go-build -gno-record-gcc-switches"

• Vuls environment:

Hash : ____ vuls v0.4.2 fe582ac

To check the commit hash of HEAD
$ vuls -v

or

$ cd $GOPATH/src/github.com/future-architect/vuls
$ git rev-parse --short HEAD

• config.toml:

• command:

vuls scan ubuntu

[thinksabin]

Even tried killing the vuls process and re-running the vuls scan command. Not working.

[kotakanbe]

Hi, @thinksabin

You have to SSH to the scan target server manually before scanning.
https://vuls.io/docs/en/tutorial-remote-scan.html#step1-launch-new-ubuntu-linux

or see https://vuls.io/docs/en/tips.html#how-to-scan-without-ssh-host-key-check

How to scan without SSH host key check
scan with -ssh-config option. see #417, #660

[thinksabin]

i did ssh manually into the target server before facing issue.
The case is sometimes vuls work sometime not. Here is the output of the vuls configtest where I had to cancel the first execution (^C) and re-run vuls.

vuls configtest ubuntu
[Aug 22 04:16:34] INFO [localhost] Validating config...
[Aug 22 04:16:34] INFO [localhost] Detecting Server/Container OS...
[Aug 22 04:16:34] INFO [localhost] Detecting OS of servers...
^C
ubuntu@server123:~$ vuls configtest ubuntu
[Aug 22 04:20:08] INFO [localhost] Validating config...
[Aug 22 04:20:08] INFO [localhost] Detecting Server/Container OS...
[Aug 22 04:20:08] INFO [localhost] Detecting OS of servers...
[Aug 22 04:20:08] INFO [localhost] (1/1) Detected: ubuntu: ubuntu 16.04
[Aug 22 04:20:08] INFO [localhost] Detecting OS of containers...
[Aug 22 04:20:08] INFO [localhost] Checking dependencies...
[Aug 22 04:20:08] INFO [ubuntu] Dependencies... No need
[Aug 22 04:20:08] INFO [ubuntu] Checking sudo settings...
[Aug 22 04:20:08] INFO [ubuntu] sudo ... No need
[Aug 22 04:20:08] INFO [localhost] Scannable servers are below...
ubuntu

I wonder why it freezes the first time but run well in second attempt.

[kotakanbe]

-debug and -vvv scan option shows the detail of ssh handshake.
please scan with -debug -vvv option.

[thinksabin]

did ' git fetch ' and got issue

#703

[thinksabin]

The output of the -debug -vvv is like this:

[Aug 29 12:08:13] DEBUG [localhost] {
"ubuntu": config.ServerInfo{
ServerName: “ubuntu”,
User: "vuls",
Host: “ubuntu”,
Port: "22",
KeyPath: "/home/ubuntu/vuls_keys/id_rsa_vuls_az",
KeyPassword: "",
CpeNames: []string{},
DependencyCheckXMLPath: "",
Containers: config.Containers{
Type: "",
Includes: []string{},
Excludes: []string{},
},
IgnoreCves: []string{},
Optional: [][]interface {}{},
Enablerepo: []string{},
Type: "",
LogMsgAnsiColor: "\x1b[32m",
Container: config.Container{
ContainerID: "",
Name: "",
Image: "",
},
Distro: config.Distro{
Family: "",
Release: "",
},
IPv4Addrs: []string{},
IPv6Addrs: []string{},
},
}
[Aug 29 12:08:13] INFO [localhost] Validating config...
[Aug 29 12:08:13] INFO [localhost] Detecting Server/Container OS...
[Aug 29 12:08:13] INFO [localhost] Detecting OS of servers...
[Aug 29 12:08:13] DEBUG [localhost] Executing... ls /etc/debian_version
[Aug 29 12:13:13] ERROR [localhost] Timed out while detecting servers
[Aug 29 12:13:13] ERROR [localhost] (1/1) Timed out: ubuntu
[Aug 29 12:13:13] ERROR [localhost] Failed to init servers: No scannable servers

[thinksabin]

@kotakanbe anything on this?

[kotakanbe]

@thinksabin

Finally, I reproduced the issue now !!!!!!!!!
I need some time to fix.

• How to reproduce

bash-3.2$ ls $HOME/.vuls

bash-3.2$ ./vuls scan -vvv deb8
[Sep 10 21:25:56]  INFO [localhost] Start scanning
[Sep 10 21:25:56]  INFO [localhost] config: /Users/kanbe/go/src/github.com/future-architect/vuls/config.toml
[Sep 10 21:25:56]  INFO [localhost] Validating config...
[Sep 10 21:25:56]  INFO [localhost] Detecting Server/Container OS...
[Sep 10 21:25:56]  INFO [localhost] Detecting OS of servers...
^C

Stalled for the first time.

SSH controlmaster file was created.

bash-3.2$ ls $HOME/.vuls
controlmaster-root-deb8.6012

bash-3.2$
bash-3.2$ ./vuls scan -vvv deb8
[Sep 10 21:26:07]  INFO [localhost] Start scanning
[Sep 10 21:26:07]  INFO [localhost] config: /Users/kanbe/go/src/github.com/future-architect/vuls/config.toml
[Sep 10 21:26:07]  INFO [localhost] Validating config...
[Sep 10 21:26:07]  INFO [localhost] Detecting Server/Container OS...
[Sep 10 21:26:07]  INFO [localhost] Detecting OS of servers...
[Sep 10 21:26:08]  INFO [localhost] (1/1) Detected: deb8: debian 8.11
[Sep 10 21:26:08]  INFO [localhost] Detecting OS of containers...
[Sep 10 21:26:08]  INFO [localhost] Checking Scan Modes...
[Sep 10 21:26:08]  INFO [localhost] Detecting Platforms...
[Sep 10 21:26:09]  INFO [localhost] (1/1) deb8 is running on other
[Sep 10 21:26:09]  INFO [localhost] Scanning vulnerabilities...
[Sep 10 21:26:09]  INFO [localhost] Scanning vulnerable OS packages...
[Sep 10 21:26:09]  INFO [deb8] Scanning in fast-root mode
[Sep 10 21:26:09]  INFO [deb8] apt-get update...
...

Success.

[thinksabin]

Thats great. what is the issue here? I couldn't even get it how is this happening.

I'm going to put vuls in the Continuous integration and this issue is really creating trouble on automation. Hope you will fix it soon. :)

[kotakanbe]

@thinksabin Sorry for the late reply.

I fixed this issue.
Would you please test on your reproduce-able environment.
#753