Correctly handle `--all` without forcing a refresh

[rec]

If you do pullman.py --all after doing it without --all, it looks at the cache file, and doesn't recognize that this was only from a partial look at the pull requests.