Incremental backups forever?

[dvitek]

As I understand the usage model of virtnbdbackup at present with incremental backups, you need to periodically perform a full backup in order expire sufficiently old backups. So for example a sequence might be something like:

• perform full backup to "january" (J0)
• perform incremental backup to "january" (J1)
• perform incremental backup to "january" (J2)
• perform incremental backup to "january" (J3)
• perform full backup to "februrary" (F0)
• delete "january"
• perform incremental backup to "february" (F1)
• ...

I am interested in avoiding the periodic cost of doing a linear pass over the entire disk of the backupee. However, I am OK with paying a linear cost on some other system with access to the backups. I am willing to do some coding to make this happen, provided there isn't some architectural reason it cannot be done.

It must be possible to synthesize a full backup of J3 by restoring it into a new domain and then doing a full backup of that restored domain. If you had this, then F0 could be performed as an incremental backup against J3, assuming the restore/backup procedure doesn't cause incidental modifications.

How possible would it be to produce a tool that takes J0...J3 as inputs and produces a non-incremental J3 as output, presumably in a more efficient way than by actually doing a restore/backup? Would you need support on the libvirt side to do something similar with the checkpoints in the domain, in order to make the J0..J2 checkpoints go away while keeping J3 around?

[abbbi]

See issue #149

#149 (comment)