Mapping Academic- Reproducibility Indicators

[PetrosStav]

No description provided.

[PetrosStav]

Hi @PetrosStav , it would probably be easier to collect all your separate PRs (#57, #58, #59, #60, #61, #62) in a single new PR, because they form a logical whole. Otherwise there would need to be separate comments and feedback in every single PR, which would be rather difficult to track.

Could you please open up a separate PR, and then let's then discuss in more detail in there. Thanks!

Hi @vtraag, I think this should do it, I gathered everything in this PR. I will go ahead and close the other PRs.

[vtraag]

Thanks @PetrosStav ! Sorry for the late reply, I wanted to get back to this last week, but got sick unfortunately.

I think that it would be better to shorten some of the existing texts in the reproducibility section. Right now there's quite some overlap between the texts in reuse of code/data in reproducibility and the texts in use of code/data in academic impact. I think it would make more sense to refer to the academic impact for the actual indicators, and then explain how they could be interpreted to reflect something about reproducibility. All in all, I think this shouldn't take much more than one or two paragraphs. If there's anything you consider vital missing in the academic impact section, you might be able to add it there.

Similarly, for the Impact of code/data in the reproducibility section, the text on the NCI is not really needed, I think, so again can just refer to the citation impact in academic impact section. For the downloads/usage/stars you can refer to the use of data/code, I think. Again, you can probably use one or two paragraphs in total to argue how these indicators can also reflect elements of reproducibility.

If you prefer, I could prepare how I would like see it, and then you can provide feedback on it?

[vtraag]

Rebased and force pushed to accommodate the change of structure on main. Make sure to pull this branch locally for making changes.

@PetrosStav, would you have a chance to look at this today still?

[PetrosStav]

@vtraag I made the changes in the files, reducing the text where there was overlap and pointed to the academic indicators. Also, as suggested, I added 1-2 paragraphs in each of those metrics referencing how they reflect reproducibility.

In cases where data sources or methodologies were completely overlapping with the academic indicators, I removed them from the reproducibility indicators. However, some data sources remain within the reproducibility indicators as they are not currently included in the academic indicators. For now, I left those intact, but you can move them to the academic indicators if they seem relevant.

[vtraag]

Thanks @PetrosStav, this is indeed more in line with what I was thinking!

I think it makes sense to move your checks of whether a repo contains code to the academic impact section. I've done that already.

The other git repo hosts (gitlab and bitbucket) can I think better be removed, otherwise they're dangling a bit, I've already did this. We could also provide details for those in academic impact section, but since we already detail a number of different metrics there for one provider, I think that doing so would make it a bit much.

As for SciNoBo, perhaps it would also make more sense to include that in the academic impact section? I'll leave it up to you to do that, you can probably integrate it better. Could you also make sure that it contains a proper reference? Otherwise "SciNoBo" without any reference might not be directly clear to readers. I haven't yet removed mentions of SciNoBo, but at the moment SciNoBo is dangling a bit in these sections as just a sole methodology, I think, which doesn't make sense.

[vtraag]

I have now moved SciNoBo to the academic impact section myself.

I'll resolve the conflicts and merge.