Review batou ram usage

[zagy]

I had a batou run killed in configure phase on the controller:

Jan 08 07:33:50 fc61 kernel: Out of memory: Killed process 1877100 (batou) total-vm:5691140kB, anon-rss:3649708kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:10916kB oom_score_adj:0

I don't think batou should need >5G RAM 🤪

[zagy]

The reason for that is that there is in GitBundleRepository:

        rsync = execnet.RSync(bundle_file, verbose=False)
        rsync.add_target(
            host.gateway, host.remote_repository + "/batou-bundle.git"
        )
        rsync.send()

Execnet's rsync in turn loads the entire file in to memory. With large repositories this is a problem obviously.

[PhilTaken]

Do remotes generally need the entire git history?
I'd assume sending a shallow bundle would suffice in most cases, say 2 or 5 commits max.

[zagy]

This is already the branch-only and the difference between remote and local. The rsync needs to be fixed. Note that we have another rsync implementation already, which we may use.