Deduplicate code for tests of archive migration code

[sphuber]

Fixes #3923

Each archive migration was being tested on a archives included in this
repo in tools/fixtures/export/migrate and the test was always the
same. Simply take an archive, migrate it to the next version with the
appropriate method and check that both metadata and data dictionaries of
the migrated data match that of the reference archive. This same test
was implemented for each migration method, but has now been centralized
in a single test using pytest to parametrize the versions.

The testing on external archives from aiida-export-migration-testing
has also been streamlined a bit, by defining a new base class called
ArchiveMigrationTest that provides some utility methods.

[codecov]

Codecov Report

Merging #3924 into develop will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #3924      +/-   ##
===========================================
- Coverage    78.21%   78.21%   -0.01%     
===========================================
  Files          460      460              
  Lines        34035    34036       +1     
===========================================
  Hits         26620    26620              
- Misses        7415     7416       +1     

Flag	Coverage Δ
#django	70.24% <100.00%> (-0.01%)	⬇️
#sqlalchemy	71.10% <100.00%> (+<0.01%)	⬆️

Impacted Files	Coverage Δ
aiida/tools/importexport/migration/v03_to_v04.py	90.95% <100.00%> (+0.04%)	⬆️
aiida/transports/plugins/local.py	80.20% <0.00%> (-0.26%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39d8ea0...dd3d0e9. Read the comment docs.

[CasperWA]

Seems like an update that should've been done some time ago... :)

Concerning my comments, I am aware of the more weird things happening from v0.3 to v0.4, which is due to a migration (TrajectoryData) needing the folder and the node files, and not just the metadata and data JSON files. If it is not possible to use the new migrate method from ArchiveMigrationTest, then that's fine, but should maybe be commented upon, if it isn't already somewhere in there.
Also, the test function name can still be updated to match the rest :)

But all in all, no major changes to be done here that I can find - as long as all tests pass and the coverage does not change significantly.

[sphuber]

@CasperWA I addressed your comments

[CasperWA]

Thanks for the changes!

I found a single pylint disable comment that can be removed, and to round it off, I think there may be an a too much in your first commit message.
After these very minor updates I will approve this as is.

[CasperWA]

Suggested change

	def migrate_v3_to_v4(metadata, data, *args): # pylint: disable=unused-argument
	def migrate_v3_to_v4(metadata, data, *args):

[sphuber]

Thanks for the changes!

Thanks for the review 👍

I found a single pylint disable comment that can be removed,

Done

and to round it off I think there may be an a too much in your first commit message.

I might be blind to double a's as a result of my name, because I could not spot it.
If you could be more specific I can fix it when I merge

[CasperWA]

and to round it off I think there may be an a too much in your first commit message.

I might be blind to double a's as a result of my name, because I could not spot it.
If you could be more specific I can fix it when I merge

So very true. And so very tragic... 😆

Anyway, there is something funky with your first commit in the first sentence, around (...) tested on a archives included in this repo in (...) specifically the a archives-part.

[sphuber]

Thanks @CasperWA I will rebase PR #3912 now which should then also be ready