template reinstall not ported to Qubes 4.0

[marmarek]

Qubes OS version (e.g., R3.2): R4.0

---

Steps to reproduce the behavior:

sudo qubes-dom0-update --action=reinstall qubes-template-fedora-25

Expected behavior:

Template got reinstalled. This include preserving its properties. Technically, only root volume should be overwritten, and private volume should be cleaned.

Actual behavior:

qvm-prefs: error: no such property: 'root_img'

And generally the code heavily depends on root.img being a file, and rpm replacing it.

[3hhh]

In general it would be nice to be able re-install or downgrade packages in dom0.

[tasket]

I'm installing R4 tonight and will look into this.

[marmarek]

Some hints:

1. If a backup is needed, use qvm-clone. It is cheap on LVM.
2. qvm-template-postprocess tool imports root.img (a file) into actual template root volume (LVM or else). Probably it needs to learn how to reinstall template (or maybe will just work in install mode?)

[tasket]

I'll be trying a fix hopefully this week.

[tasket]

Aspects of this issue that need to be addressed:

1. Should not try to delete template on import_data failure when template already existed. Easily addressed in qvm_template_postprocess.py
2. Resizing of root volume fails when the existing one had been expanded past the default size. Possibly a security as well as functional issue. Probably there should be a qubesd function that can initialize or zero-out a volume and make this accessible from qubesadmin. Or maybe volume should be deleted and re-created on each import, but I'm not sure where to put this.

[marmarek]

@tasket the second point is already tracked here: #2256 (comment)

[marmarek]

And should be addressed by QubesOS/qubes-core-admin#203 - import is performed on fresh temporary volume, renamed to final name only when finished.

I'm also going to relax anti-shrink protection - move it to qvm-volume tool (GUI settings already prevent it) and allow to do that at API level (and with some option to qvm-volume, probably --force). This way, it will be possible to reinstall template and really have it in initial state.

[tasket]

Thanks. This is pretty extensive.

The direction I was going was only to remove and re-create the volume in lvm.py. But I was having trouble tracking the call paths from the admin code to qubesd import_data() so that the size of the new data set could be passed to it.

BTW - The reflink.py handler appears to deal with import volume safety already, but file.py does not.

[marmarek]

Thanks. This is pretty extensive.

Yes. This ticket is not the only reason for it (actually, importing into temporary volume could be done without all the other changes). Other thing is commit operation required multiple steps, which, if something goes wrong, can lead to data loss, or at least need for manual recovery. Extreme result was #3164.
I think the scheme I've finally implemented was proposed by @jpouellet about that time, but I can't find the reference right now.

[rustybird]

I'm also going to relax anti-shrink protection - move it to qvm-volume tool (GUI settings already prevent it) and allow to do that at API level (and with some option to qvm-volume, probably --force).

AFAICT we could even make shrinking safe enough for the GUI, by doing the operations in reverse order. Growing a volume is currently

1. dom0 driver: resize backing storage
2. dom0 driver: resize view of backing storage (e.g. loop devices), if applicable
3. VM: resize partition/device-mapper table entry, if applicable
4. VM: resize filesystem

Shrinking would be 4, 3, 2, 1.

[marmarek]

Problem with point 4 is that for most filesystems it can't be done while the filesystem is mounted (in contrary to extending). And for some it isn't supported at all (xfs, but in practice we care about ext4 only). This is definitely doable (for example from initramfs), but not just "doing the operations in reverse order".

BTW currently we recommend universal method for shrinking, by creating new smaller VM and moving data: https://www.qubes-os.org/doc/resize-disk-image/#shrinking-a-disk-image

[rustybird]

Problem with point 4 is that for most filesystems it can't be done while the filesystem is mounted (in contrary to extending).

Damn, I never noticed that restriction in the resize2fs manpage. Too bad then.

[marmarek]

BTW - The reflink.py handler appears to deal with import volume safety already, but file.py does not.

Fixed here: marmarek/qubes-core-admin@f870272 (QubesOS/qubes-core-admin#206)

[tasket]

I think we still have a problem with resizing during import, if new image is smaller:

https://github.com/QubesOS/qubes-core-admin/blob/e2b70306e558face4003b3e42e3dcedd6f31949f/qubes/api/admin.py#L470-L472

 def import_root_img(vm, source_dir):
     '''Import root.img into VM object'''
 
 
     root_size = get_root_img_size(source_dir)
     vm.volumes['root'].resize(root_size)

Won't an error get raised in lvm.py resize() if size < self.size ?

        if size < self.size:
            raise qubes.storage.StoragePoolException(
                'For your own safety, shrinking of %s is'
                ' disabled (%d < %d). If you really know what you'
                ' are doing, use `lvresize` on %s manually.' %
                (self.name, size, self.size, self.vid))


        if size == self.size:
            return


        if self.is_dirty():
            cmd = ['extend', self._vid_snap, str(size)]
            qubes_lvm(cmd, self.log)
        elif hasattr(self, '_vid_import') and \

[marmarek]

You're right, working on it.

[tasket]

Also, is the new image size being propagated to (lvm|reflink|file).py import_data() when that function is called? I can't tell.

[marmarek]

That's why qvm-template-postprocess calls volume.resize first.

[tasket]

I see, the size attribute gets set without doing lvextend/truncate. Makes sense now.

[tasket]

PR QubesOS/qubes-core-admin-client#61

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.17-0.1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component core-admin-client (including package python2-qubesadmin-4.0.17-0.1.fc26) has been pushed to the r4.0 testing repository for the Fedora template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.17-1+deb9u1 has been pushed to the r4.0 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[awokd]

Verified to fix the missing directory after a reinstall. Thank you!

[qubesos-bot]

Automated announcement from builder-github

The package core-admin-client has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.17-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-admin-client has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component core-admin-client (including package python2-qubesadmin-4.0.17-0.1.fc26) has been pushed to the r4.0 stable repository for the Fedora template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.17-0.1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update