Implement exponential backoff for HTTP PUT

[mariocynicys]

A feature request to track adding an exponential backoff strategy for HTTP PUTs.

shaka-packager/packager/file/http_file.cc

Lines 212 to 213 in 31129ee

	// TODO: Implement retrying with exponential backoff, see
	// "widevine_key_source.cc"

Not backing is sometimes problematic, as seen in here: shaka-project/shaka-streamer#103 (comment)

[joeyparrish]

Totally on board with this.

[mariocynicys]

I am trying to compile shaka-packager using these instructions, but getting the following error. I assume it has to do with the branch changes since the output logs mentions things are pulled from master and not main.

@meryacine ➜ /workspaces/pkger $ gclient config https://github.com/shaka-project/shaka-packager.git --name=src --unmanaged && gclient sync --no-history
Warning: Running gclient on Python 3. 
If you encounter any issues, please file a bug on crbug.com under the Infra>SDK component.
Warning: Running gclient on Python 3. 
If you encounter any issues, please file a bug on crbug.com under the Infra>SDK component.

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://github.com/shaka-project/shaka-packager.git --depth=1 /workspaces/pkger/_gclient_src_fg6pfb8i' in '/workspaces/pkger'
Cloning into '/workspaces/pkger/_gclient_src_fg6pfb8i'...
remote: Enumerating objects: 4095, done.        
remote: Counting objects: 100% (4095/4095), done.        
remote: Compressing objects: 100% (2698/2698), done.        
remote: Total 4095 (delta 1388), reused 2827 (delta 1210), pack-reused 0        
Receiving objects: 100% (4095/4095), 29.32 MiB | 2.67 MiB/s, done.
Resolving deltas: 100% (1388/1388), done.
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://github.com/shaka-project/shaka-packager.git --depth=1 /workspaces/pkger/_gclient_src_m4qopnbt' in '/workspaces/pkger'
Cloning into '/workspaces/pkger/_gclient_src_m4qopnbt'...
remote: Enumerating objects: 4095, done.        
remote: Counting objects: 100% (4095/4095), done.        
remote: Compressing objects: 100% (2698/2698), done.        
remote: Total 4095 (delta 1388), reused 2827 (delta 1210), pack-reused 0        
Receiving objects: 100% (4095/4095), 29.32 MiB | 3.61 MiB/s, done.
Resolving deltas: 100% (1388/1388), done.
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "refs/heads/master:refs/remotes/origin/master" "--no-tags"' in /workspaces/pkger/src failed; will retry after a short nap...
Syncing projects: 100% ( 1/ 1) src
src (ERROR)
----------------------------------------
[0:00:00] Started.
[0:00:00] 
[0:00:14] Finished running: git rev-parse --verify FETCH_HEAD
fatal: couldn't find remote ref refs/heads/master
[0:00:15] fatal: couldn't find remote ref refs/heads/master
[0:00:16] fatal: couldn't find remote ref refs/heads/master
[0:00:19] fatal: couldn't find remote ref refs/heads/master
[0:00:23] _____ Conflicting directory found in /workspaces/pkger/src. Moving to /workspaces/pkger/_bad_scm/srcrg6lpwam.
[0:00:23] 
[0:00:34] Finished running: git rev-parse --verify FETCH_HEAD
[0:00:34] fatal: couldn't find remote ref refs/heads/master
[0:00:35] fatal: couldn't find remote ref refs/heads/master
[0:00:36] fatal: couldn't find remote ref refs/heads/master
[0:00:39] fatal: couldn't find remote ref refs/heads/master
----------------------------------------
Error: Command 'git -c core.deltaBaseCacheLimit=2g fetch origin refs/heads/master:refs/remotes/origin/master --no-tags' returned non-zero exit status 128 in /workspaces/pkger/src


Warnings:
Conflicting directory /workspaces/pkger/src moved to /workspaces/pkger/_bad_scm/srcrg6lpwam.

@meryacine ➜ /workspaces/pkger $  ls
_bad_scm  src

Update: Solved this by editing .gclient file like so:

solutions = [
  { "name"        : 'src',
    # @main here to tell gclient to fetch main branch and not master
    "url"         : 'https://github.com/shaka-project/shaka-packager.git@main',
    "deps_file"   : 'DEPS',
    "managed"     : False,
    "custom_deps" : {
    },
    "custom_vars": {},
  },
]