New publish concurrent with yanks is missed

[Nemo157]

From investigation in rust-lang/docs.rs#1912 it looks like this is a crates-index-diff issue. The crate had a publish and two yanks in between two checks and we only see the yank event.

The relevant commit range contains just this publish and the two yanks:

> git log --oneline --reverse b49672ff6a2d40123a593cfbca9d05219346e398~1..92c18bdf30a4872d355e4a5b3a7f7c6c75323cf7
b49672ff6a2 Updating crate `aegis#0.2.4`
da97cd0243b Updating crate `ansi-color-codec#0.3.11`
1533f8e863a Yanking crate `ansi-color-codec#0.3.4`
92c18bdf30a Yanking crate `ansi-color-codec#0.3.5`

I wrote a little test program to verify this:

use crates_index_diff::{Change, CrateVersion};

fn main() -> anyhow::Result<()> {
    let index = crates_index_diff::Index::from_path_or_cloned("index")?;
    let mut args = std::env::args().skip(1);
    let start = git_hash::ObjectId::from_hex(args.next().unwrap().as_bytes())?;
    let end = git_hash::ObjectId::from_hex(args.next().unwrap().as_bytes())?;
    for change in index.changes_between_commits(start, end)? {
        match change {
            Change::Added(CrateVersion { name, version, .. }) => println!("added {name} {version}"),
            Change::Yanked(CrateVersion { name, version, .. }) => println!("yanked {name} {version}"),
            Change::Deleted { name } => println!("deleted {name}"),
        }
    }
    Ok(())
}

For the full range it shows the same behaviour:

> cargo run --quiet --release -- b49672ff6a2d40123a593cfbca9d05219346e398 92c18bdf30a4872d355e4a5b3a7f7c6c75323cf7
yanked ansi-color-codec 0.3.4
yanked ansi-color-codec 0.3.5

If I only include the first one or two commits it behaves correctly:

> cargo run --quiet --release -- b49672ff6a2d40123a593cfbca9d05219346e398 da97cd0243bd297e7b0e4b11040e531af2e256d1
added ansi-color-codec 0.3.11

> cargo run --quiet --release -- b49672ff6a2d40123a593cfbca9d05219346e398 1533f8e863aee3fd5340513acfeef9d42816cd08
yanked ansi-color-codec 0.3.4
added ansi-color-codec 0.3.11

[Byron]

Thanks a lot for the excellent analysis, which will help tremendously to produce a test for a fix. I am pretty sure it has something to do with these lines which exist to assure the normalization tests work (and don't register as change). Let's see.

[Byron]

A new bugfix release has been created.

The issue was I forgot to exhaust the whole diff and the algorithm would stop after the first hunk of modification. Here there were modifications and an addition.

That's probably one of the disadvantages of operating only on a sample and I wonder if the baseline could be made exhaustive despite a constantly changing index (in terms of git history). Maybe it's enough to get any clone of the crates index and assure that all changes obtained though this library, when aggregated, match all iterable crates.