DOIs and lesson retirement / updating policy

[mdlincoln]

Branching off of #1370

Committing to use DOIs has implications for how we update published lessons. In this ticket I want to propose a stricter policy around updating and retiring/replacing lessons in response.

From a technical standpoint, implementing DOIs is relatively straightforward once we have established a partnership with a DOI provider (I will open a separate ticket about the technical aspects). It means that we register a DOI through the provider's interface, submitting lesson metadata (title, authors, abstract, date published, etc.) and then provide a URL to the provider that we promise will stay online persistently. Anyone going to the DOI should then be redirected to our live URL.

Speaking with our scholarly communications librarian and our data management librarian, the general consensus is that the content of the page that a DOI points to should not change in major, substantive ways. Formatting and typo corrections are acceptable - but significant editorial changes to a journal-article-like document (which is how we advertise our lessons) should result in a new webpage with its own URL and a new DOI. For us, that would mean making a new markdown file with a new lesson title and slug (and then possibly retiring the "old" lesson, and adding a link to the new lesson)

We need to decide where we are going to draw the line between insubstantial edits, and substantial changes to content. Some cases are pretty clear, in my opinion:

• Replacing an old broken external link with a link to the Internet Archive version (again, in my opinion) is an insubstantial edit, that shouldn't mean needing to create a whole new page.
• However, under this new proposed policy, I would argue that changes like the shift from Python 2 to Python 3 would have resulted in brand new lesson pages with new URLs, and retiring the old lessons.
• What about Lavin tfidf update #1664, where the author is asking to do a post-publication update? It's a fairly minor addition, nothing like wholly updating code and lesson structure. But you'd never be able to do that in a regular journal, so where do we want to draw the line?

I think this is an outgrowth of the identity confusion that Programming Historian has had for a long time - are we a website of lessons? or a scholarly journal? And committing to DOIs for our journal-article-like lessons means committing more to the scholarly journal side of things. If we're doing that, then we should be much stricter about post-publication edits, and much more ready to retire & replace broken lessons.

n.b. replacement lessons may or may not have enough changes that it merits completely new peer review! The python 3 updates were substantial enough that I think they would have merited a new DOI, but not big enough to mean a whole new round of peer review! Whereas, were I to write a new SPARQL lesson, it would be so changed that it would probably merit a new round of peer review. So that's another question you need to decide on.

TL;DR: DOIs means being more strict about lesson updates

1. What updates (typos, broken links, other...?) are small enough to do without creating a new page+DOI?
2. What updates are big enough to merit a new page+DOI?
3. What updates are big enough to merit a whole new lesson proposal & peer-review cycle?

[acrymble]

I don't think "retirement" is conducive to being a scholarly journal. There is "published" and "retracted" but no one else that I know of uses this idea of retirement.

[svmelton]

To add to @mdlincoln's good questions above, I wonder if we should designate someone to be in charge of managing lesson updates. This has come up recently in PH EN conversations, and @acrymble and I just had an email exchange about potentially adding that as a role. With DOIs in the mix, it seems like we might need this role more acutely. (I know we've discussed it as a ME function—but to be honest, it's a lot of extra work!)

[mdlincoln]

Our "retirement" mechanism would be the way to manage creating versions @acrymble - rename it if the project team thinks it needs a re-name.

And @svmelton that may be a good idea - but I'd argue that the management should actually be made simpler by being much stricter in saying "No." to updates of a certain type.

[acrymble]

Whatever the criteria, it should be (in my view) on the side of preserving effort. If 200 hours has gone into producing a lesson and 1 hour and 7 lines can fix it, that's worth it, in my view. It's too easy to say "broken", without remembering how much energy went into creating something in the first place.

I think we can put forth an argument that the DOI is linked to a document with a purpose. As long as the document still serves that purpose, it can stay.

But I agree with @mdlincoln that we should say no more frequently to requests to update things for the sake of wanting to.

[drjwbaker]

A few brief thoughts:

• we need to involve our DOI provider a little in determining what changes create a new DOI because a) as librarians they are interested in the challenge, and b) they are paying per new DOI.
• agree that we need to say no more often.
• we need to change some of our ~"help us fix stuff!" language (eg https://programminghistorian.org/en/contribute#provide-feedback-or-report-problems) to be more realistic about what we'll do with that help (eg we might reject a PR that was a substantive change and would therefore mint a new DOI version because we didn't think the content of the PR warranted a new DOI version, even though we were grateful for the idea). I guess this is an expansion of the "say no more often" thought..

[drjwbaker]

Also, huge thanks to @mdlincoln for taking the time to kick this off.

[drjwbaker]

Final thing, can I propose we assemble some test cases from past edits and compile them as reference examples of New DOI and No New DOI, include our justifications against the DOI guidelines, and use that as a public/working document to guide (but not constrain, because there will always be weird edge cases) our future decisions. Again, I suspect our DOI partner would be interested in seeing this and helping us improve it.

[acrymble]

Re suggestion for updating text asking for updates, the following pages may be relevant:

https://programminghistorian.org/en/contribute#provide-feedback-or-report-problems
https://programminghistorian.org/en/feedback

(and obviously the other language equivalents).

Thanks @mdlincoln for initiating this discussion.

[mdlincoln]

Whatever the criteria, it should be (in my view) on the side of preserving effort. If 200 hours has gone into producing a lesson and 1 hour and 7 lines can fix it, that's worth it, in my view. It's too easy to say "broken", without remembering how much energy went into creating something in the first place.

Agreed @acrymble, that's why what I'd be proposing is like this:

1. A lesson is reviewed and published at /en/lessons/how-to-use-sparql, and a DOI is registered for it.
2. Some time later, substantive-enough changes are required in the code that it would cross the threshold of needing to be a new version. This could even just entail 7 lines.
3. We make a new copy of the lesson markdown to /en/lessons/how-to-use-sparql-2 and register a new DOI for it.
4. We move /en/lessons/how-to-use-sparql to /en/lessons/retired/how-to-use-sparql
5. We add a note to /en/lessons/retired/how-to-use-sparql that there's a more up-to-date version with corrected code available at /en/lessons/how-to-use-sparql-2.
6. We add a note to /en/lessons/how-to-use-sparql-2 that points to the earlier version.
7. This could be repeated again if we needed /en/lessons/how-to-use-sparql-3

This ensures that both the original as well as the new lesson keep their valid URLs and DOIs. It doesn't require a whole new peer-review process - we're just making a new copy of the lesson and updating small but significant portions of it. It ensures that all the versions remain available online, but only the most recent one is displayed in the lessons index. It ensures all the versions are linked to each other, so readers can see context.

I think this could require changing our default "lesson retired" text. Currently, it does make it sound like the only reason lessons are retired is due to massively-out-of-date software. That may still happen - in which case, we'd note that in the retirement notice at the top of the lesson. But we might rewrite it to say this lesson is no longer the most up-to-date version - click here to see the newer version.

Does that clarify things?

[spapastamkou]

Maybe it is already considered, but I thought of Zenodo's DOIs versioning with one global DOI for all versions of a digital object and specific DOIs for each version as described here.

[mdlincoln]

Zenodo doesn't actually provide DOIs for arbitrary URLs, we're going with @drjwbaker's supplier instead. Also, the DOIs themselves don't have versioning, other than using a string that looks like a version number. The actual content system (zenodo, or Programming Historian) is the one that is responsible for linking together versions of docs.