VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" #450

eaglerainbow · 2023-05-21T09:39:30Z

Hi folks,

vanilla installation at

# git rev-parse HEAD
e03dfe70e4ff74edf2e01e810cd69bc6f39adcaa

running on

# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

currently fails for me with

# ./create-director-vm.sh
Deployment manifest: '/opt/cf/bosh/bosh-deployment/bosh.yml'
Deployment state: './state.json'

Started validating
  Downloading release 'bosh'... Finished (00:00:05)
  Validating release 'bosh'... Finished (00:00:00)
  Downloading release 'bpm'... Finished (00:00:05)
  Validating release 'bpm'... Finished (00:00:00)
  Downloading release 'bosh-virtualbox-cpi'... Finished (00:00:03)
  Validating release 'bosh-virtualbox-cpi'... Finished (00:00:03)
  Downloading release 'garden-runc'... Finished (00:00:16)
  Validating release 'garden-runc'... Finished (00:00:02)
  Downloading release 'bosh-warden-cpi'... Finished (00:00:06)
  Validating release 'bosh-warden-cpi'... Finished (00:00:00)
  Downloading release 'os-conf'... Finished (00:00:00)
  Validating release 'os-conf'... Finished (00:00:00)
  Downloading release 'uaa'... Finished (00:00:13)
  Validating release 'uaa'... Finished (00:00:01)
  Downloading release 'credhub'... Finished (00:00:06)
  Validating release 'credhub'... Finished (00:00:00)
  Validating cpi release... Finished (00:00:00)
  Validating deployment manifest... Finished (00:00:00)
  Downloading stemcell... Finished (00:00:18)
  Validating stemcell... Finished (00:00:06)
Finished validating (00:01:50)

Started installing CPI
  Compiling package 'golang-1-linux/79f531850e62e3801f1dfa4acd11c421aebe653cd4316f6e49061818071bb617'... Finished (00:00:30)
  Compiling package 'golang-1-darwin/42353203720fdc244cc16f881167bb0363f38ff6f991247c4356e38bc6b5fb81'... Finished (00:00:25)
  Compiling package 'virtualbox_cpi/10a941d387c86d06f0eb3c82f29fb1228489782ba44f8961d3acd264bf2d5198'... Finished (00:00:20)
  Installing packages... Finished (00:00:07)
  Rendering job templates... Finished (00:00:00)
  Installing job 'virtualbox_cpi'... Finished (00:00:00)
Finished installing CPI (00:01:25)

Cleaning up rendered CPI jobs... Finished (00:00:00)

Error getting CPI info:
  Executing external CPI command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi':
    Running command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi', stdout: '', stderr: '[File System] 2023/05/21 09:24:02 DEBUG - Reading file /root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/config/cpi.json
[File System] 2023/05/21 09:24:02 DEBUG - Read content
********************
{"Host":"","Username":"ubuntu","PrivateKey":"","BinPath":"VBoxManage","StoreDir":"~/.bosh_virtualbox_cpi","StorageController":"sata","AutoEnableNetworks":true,"Agent":{"NTP":["0.pool.ntp.org","1.pool.ntp.org"],"mbus":"https://mbus:**REDACTED***@0.0.0.0:6868"}}

********************
[main] 2023/05/21 09:24:02 ERROR - Loading config Validating configuration: Validating Agent configuration: Validating Blobstore configuration: Must provide non-empty Type
':
      exit status 1

Exit code 1

Content of ./create-director-vm.sh is:

../../bosh create-env /opt/cf/bosh/bosh-deployment/bosh.yml \
  --state ./state.json \
  -o /opt/cf/bosh/bosh-deployment/virtualbox/cpi.yml \
  -o /opt/cf/bosh/bosh-deployment/virtualbox/outbound-network.yml \
  -o /opt/cf/bosh/bosh-deployment/bosh-lite.yml \
  -o /opt/cf/bosh/bosh-deployment/bosh-lite-runc.yml \
  -o /opt/cf/bosh/bosh-deployment/uaa.yml \
  -o /opt/cf/bosh/bosh-deployment/credhub.yml \
  -o /opt/cf/bosh/bosh-deployment/jumpbox-user.yml \
  --vars-store ./creds.yml \
  -v director_name=bosh-lite \
  -v internal_ip=192.168.56.6 \
  -v internal_gw=192.168.56.1 \
  -v internal_cidr=192.168.56.0/24 \
  -v outbound_network_name=NatNetwork

I am running on

vboxmanage --version
6.1.38_Ubuntur153438

following the steps as described on https://bosh.io/docs/bosh-lite/ (I am doing my first steps on bosh/cloudfoundry)

The text was updated successfully, but these errors were encountered:

eaglerainbow · 2023-05-21T09:47:36Z

Re-ran things with export BOSH_LOG_LEVEL=debug and got the following verbose log:

[CLI] 2023/05/21 09:44:24 ERROR - Error getting CPI info: Executing external CPI command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi': Running command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi', stdout: '', stderr: '[File System] 2023/05/21 09:44:24 DEBUG - Reading file /root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/config/cpi.json
[File System] 2023/05/21 09:44:24 DEBUG - Read content
********************
{"Host":"","Username":"ubuntu","PrivateKey":"","BinPath":"VBoxManage","StoreDir":"~/.bosh_virtualbox_cpi","StorageController":"sata","AutoEnableNetworks":true,"Agent":{"NTP":["0.pool.ntp.org","1.pool.ntp.org"],"mbus":"https://mbus:***REDACTED***@0.0.0.0:6868"}}

********************
[main] 2023/05/21 09:44:24 ERROR - Loading config Validating configuration: Validating Agent configuration: Validating Blobstore configuration: Must provide non-empty Type
': exit status 1
Error getting CPI info:
  Executing external CPI command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi':
    Running command: '/root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/bin/cpi', stdout: '', stderr: '[File System] 2023/05/21 09:44:24 DEBUG - Reading file /root/.bosh/installations/5fce9ad7-7811-4f5f-7118-e8389d7dc28f/jobs/virtualbox_cpi/config/cpi.json
[File System] 2023/05/21 09:44:24 DEBUG - Read content
********************
{"Host":"","Username":"ubuntu","PrivateKey":"","BinPath":"VBoxManage","StoreDir":"~/.bosh_virtualbox_cpi","StorageController":"sata","AutoEnableNetworks":true,"Agent":{"NTP":["0.pool.ntp.org","1.pool.ntp.org"],"mbus":"https://mbus:***REDACTED***@0.0.0.0:6868"}}

********************
[main] 2023/05/21 09:44:24 ERROR - Loading config Validating configuration: Validating Agent configuration: Validating Blobstore configuration: Must provide non-empty Type
':
      exit status 1
Exit code 1

ramonskie · 2023-05-22T09:56:51Z

this is similar to this issue cloudfoundry/bosh-aws-cpi-release#149

this pr cloudfoundry/bosh-aws-cpi-release#147 removed the blosttore settings from the agent
the blobstore settings should now be put under env: see bosh-deployment for example

bosh-deployment/bosh.yml

Lines 158 to 163 in c62c8bd

    
           - env: 
        
               bosh: 
        
                 blobstores: 
        
                 - options: 
        
                     blobstore_path: /var/vcap/micro_bosh/data/cache 
        
                   provider: local

and we should fix this in virtualbox cpi the same as we did with aws for example
https://github.com/cloudfoundry/bosh-aws-cpi-release/pull/147/files

ramonskie · 2023-05-22T10:11:14Z

created a pr for this just now cloudfoundry/bosh-virtualbox-cpi-release#27

rkoster · 2023-05-25T15:19:08Z

PR has been merged, asked for a release here: cloudfoundry/bosh-virtualbox-cpi-release#27 (comment)

eaglerainbow · 2023-05-25T21:25:18Z

Thanks for all your kind support!

eaglerainbow · 2023-06-04T07:52:34Z

So is this issue fixed? I'm facing the same issue on master branch.

No, at least I am still facing it as well.
As far as I get things, we are still at the state of cloudfoundry/bosh-virtualbox-cpi-release#27 (comment).

eaglerainbow · 2023-06-04T08:07:01Z

With

git rev-parse HEAD
73b961f8dc5c602541eabf3084aebe2f17b038cb

I now got bosh-virtualbox-cpi version 0.4.0:

...
[File System] 2023/06/04 07:57:23 DEBUG - Reading file /root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/tmp/bosh-release452306065/release.MF
[File System] 2023/06/04 07:57:23 DEBUG - Read content
********************
name: bosh-virtualbox-cpi
version: 0.4.0
commit_hash: bff95ae
uncommitted_changes: false
...

and later on I get again

[File System] 2023/06/04 08:00:10 DEBUG - Checking if file exists /root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/jobs/virtualbox_cpi/bin/cpi
[File System] 2023/06/04 08:00:10 DEBUG - Stat '/root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/jobs/virtualbox_cpi/bin/cpi'
[cloud] 2023/06/04 08:00:10 DEBUG - Init cloud with stemcell version: 3
[cloud] 2023/06/04 08:00:10 DEBUG - Info
[Cmd Runner] 2023/06/04 08:00:10 DEBUG - Running command '/root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/jobs/virtualbox_cpi/bin/cpi'
[Cmd Runner] 2023/06/04 08:00:10 DEBUG - Stdout: 
[Cmd Runner] 2023/06/04 08:00:10 DEBUG - Stderr: [File System] 2023/06/04 08:00:10 DEBUG - Reading file /root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/jobs/virtualbox_cpi/config/cpi.json
[File System] 2023/06/04 08:00:10 DEBUG - Read content
********************
{"Host":"","Username":"ubuntu","PrivateKey":"","BinPath":"VBoxManage","StoreDir":"~/.bosh_virtualbox_cpi","StorageController":"sata","AutoEnableNetworks":true,"Agent":{"NTP":["0.pool.ntp.org","1.pool.ntp.org"],"mbus":"https://mbus:***REDACTED***@0.0.0.0:6868"}}

********************
[main] 2023/06/04 08:00:10 ERROR - Loading config Validating configuration: Validating Agent configuration: Validating Blobstore configuration: Must provide non-empty Type
[Cmd Runner] 2023/06/04 08:00:10 DEBUG - Successful: false (1)
[cpiCmdRunner] 2023/06/04 08:00:10 DEBUG - Exit Code 1 when executing external CPI command '/root/.bosh/installations/8ad0b5df-30bb-47d6-6510-7b20045f7e51/jobs/virtualbox_cpi/bin/cpi'
STDIN: '{"method":"info","arguments":[],"context":{"director_uuid":"d2d71ee6-2f94-4e61-680c-87b1e2d1d06d","vm":{"stemcell":{"api_version":3}}},"api_version":1}'
STDOUT: ''

So I guess cloudfoundry/bosh-virtualbox-cpi-release#27 wasn't the cure?

ramonskie · 2023-06-05T14:23:17Z

i think its complaining about this line https://github.com/cloudfoundry/bosh-deployment/blob/master/bosh-lite.yml#L46

EDIT:
i can replicate it localy.
and the above line is not the issue unfortunately i will try to debug some more

ramonskie · 2023-06-06T11:15:19Z

i have solved the issue. but a few thinks need to happen.
first we need to merge cloudfoundry/bosh-cpi-go#4
and then we need to update the dependency in the virtualbox-cpi so that we use the latest bosh-cpi-go

ramonskie · 2023-06-09T11:21:26Z

virtualbox has now been release https://github.com/cloudfoundry/bosh-virtualbox-cpi-release/releases/tag/v0.4.1
the bosh-deployment ci will pick this up automaticly

eaglerainbow · 2023-06-09T18:31:31Z

Thanks! I will check asap and get back with feedback.

eaglerainbow · 2023-06-09T21:35:20Z

the bosh-deployment ci will pick this up automaticly

@ramonskie Why do you think so?
When I try to run it on

$  git rev-parse HEAD
73b961f8dc5c602541eabf3084aebe2f17b038cb

I still get

[File System] 2023/06/09 21:32:56 DEBUG - Reading file /root/.bosh/installations/7ca3f07a-504a-4611-6c93-9cafc3f05dcd/tmp/bosh-release4215622770/release.MF
[File System] 2023/06/09 21:32:56 DEBUG - Read content
********************
name: bosh-virtualbox-cpi
version: 0.4.0
commit_hash: bff95ae
uncommitted_changes: false

in the logs.

Or am I just "too fast" and need to wait for the bosh-deployment CI to complete first?
Where can I check how far the pipeline is?

eaglerainbow · 2023-06-10T07:41:12Z

Or am I just "too fast" and need to wait for the bosh-deployment CI to complete first?

Seems like I was simply too fast yesterday night.

$ git rev-parse HEAD
f0cb03c59c1d2428f4ab87a6e415dc0a9546be0b


[File System] 2023/06/10 07:32:45 DEBUG - Reading file /root/.bosh/installations/ac4ed348-ab2a-460e-4d29-44f40314fccb/tmp/bosh-release469373791/release.MF
[File System] 2023/06/10 07:32:45 DEBUG - Read content
********************
name: bosh-virtualbox-cpi
version: 0.4.1
commit_hash: fe0b631
uncommitted_changes: false

Looks much better now:

[cloud] 2023/06/10 07:37:22 INFO - CPI Info resulted in {[general-ovf vsphere-ovf] 2}
[Cmd Runner] 2023/06/10 07:37:22 DEBUG - Running command '/root/.bosh/installations/ac4ed348-ab2a-460e-4d29-44f40314fccb/jobs/virtualbox_cpi/bin/cpi'
[Cmd Runner] 2023/06/10 07:37:55 DEBUG - Stdout: {"result":"sc-2331012c-af6a-4b8b-4397-7e2250904495","error":null,"log":""}
[Cmd Runner] 2023/06/10 07:37:55 DEBUG - Stderr: [File System] 2023/06/10 07:37:22 DEBUG - Reading file /root/.bosh/installations/ac4ed348-ab2a-460e-4d29-44f40314fccb/jobs/virtualbox_cpi/config/cpi.json

Seems like this issue is fixed. Hence I am closing it.

tl&dr;

You need to use virtualbox_cpi:0.4.1 or higher.
This became available with git commit f0cb03c.

cf-foundation-community-automation bot moved this to Inbox in Foundational Infrastructure Working Group May 21, 2023

cf-foundation-community-automation bot added this to Foundational Infrastructure Working Group May 21, 2023

rkoster moved this from Inbox to Pending Merge | Prioritized in Foundational Infrastructure Working Group May 25, 2023

rkoster assigned ramonskie May 25, 2023

ramonskie mentioned this issue Jun 6, 2023

remove blobstore info from iaas metada endpoint cloudfoundry/bosh-cpi-go#4

Merged

This was referenced Jun 8, 2023

VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" cloudfoundry/bosh-virtualbox-cpi-release#28

Closed

bump bosh-cpi-go to fix #28 cloudfoundry/bosh-virtualbox-cpi-release#29

Merged

eaglerainbow closed this as completed Jun 10, 2023

github-project-automation bot moved this from Pending Merge | Prioritized to Done in Foundational Infrastructure Working Group Jun 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" #450

VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" #450

eaglerainbow commented May 21, 2023 •

edited

Loading

eaglerainbow commented May 21, 2023 •

edited

Loading

ramonskie commented May 22, 2023

ramonskie commented May 22, 2023

rkoster commented May 25, 2023

eaglerainbow commented May 25, 2023

eaglerainbow commented Jun 4, 2023

eaglerainbow commented Jun 4, 2023

ramonskie commented Jun 5, 2023 •

edited

Loading

ramonskie commented Jun 6, 2023

ramonskie commented Jun 9, 2023

eaglerainbow commented Jun 9, 2023

eaglerainbow commented Jun 9, 2023 •

edited

Loading

eaglerainbow commented Jun 10, 2023

VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" #450

VirtualBox install fails with "Validating Blobstore configuration: Must provide non-empty Type" #450

Comments

eaglerainbow commented May 21, 2023 • edited Loading

eaglerainbow commented May 21, 2023 • edited Loading

ramonskie commented May 22, 2023

ramonskie commented May 22, 2023

rkoster commented May 25, 2023

eaglerainbow commented May 25, 2023

eaglerainbow commented Jun 4, 2023

eaglerainbow commented Jun 4, 2023

ramonskie commented Jun 5, 2023 • edited Loading

ramonskie commented Jun 6, 2023

ramonskie commented Jun 9, 2023

eaglerainbow commented Jun 9, 2023

eaglerainbow commented Jun 9, 2023 • edited Loading

eaglerainbow commented Jun 10, 2023

eaglerainbow commented May 21, 2023 •

edited

Loading

eaglerainbow commented May 21, 2023 •

edited

Loading

ramonskie commented Jun 5, 2023 •

edited

Loading

eaglerainbow commented Jun 9, 2023 •

edited

Loading