Treebank development in master branch

[msklvsk]

UD’s current model is to push new commits to dev, and merge to master twice a year. This often causes confusion: people expect new commits to be at master: they visit treebank’s main page (which is master) to see what’s happening. Instead of news they see a 3-month-old state (on average).
You tell a friend to go and check your updates, and he writes back there aren’t any because you forget to mention he should switch to dev.

What stops us from using a classical model, with stable releases branching out from master? You can even push bugfixes for old releases this way.

[jnivre]

The main motivation is to promote reproducibility of scientific results. Ideally, published experiments should always be performed on official releases, downloaded from the Lindat server. However, we know that many users prefer to download from GitHub instead, and we therefore want to ensure that the master branch is in sync with the latest official release.

[msklvsk]

Let me change this from a question to a suggestion.

The current workflow is anti-github.

• Just recently, I found people using our treebank from master, stumbling upon errors we’ve already fixed in dev.
• Constantly, somebody is pushing to master, like yesterday: Move README to README.md so it displays correctly UD_Japanese-GSD#5.
• People also visit repo’s index, which is master, and expect it to show what’s happening: latest commits, announcements in the readme etc.

You can’t blame people for using Github like they are supposed to. The policy to protect users who are doing it wrong at the expense of others is questionable. If you “prefer to download from GitHub instead”, you should know that master is the latest. And for the uninitiated, there’s Lindat. However, I would give our users more credit.

I believe there is more harm than benefit with the current solution and propose to develop in master and branch releases out.

[dan-zeman]

We did not come up with the current policy out of the blue. Unfortunately, there were already paper submissions presenting results obtained on a Github revision of a treebank, instead of a numbered release. The current policy is a reaction to that experience. UD treebanks are not used the same way as typical open software, so it should not be too surprising if the workflow differs from typical Github workflow.

It would have been possible to leave it to the responsibility of paper authors that they do not take their data from Github. The papers with unreproducible and unverifiable results should be rejected anyways. But the collective decision was to adopt the current policy. With 144 treebanks, reversing the policy would no longer be a simple step, and the benefits do not outweigh the drawbacks. (Personal note: I felt quite indifferently about the switch to the current policy when it was being adopted. But now I feel strongly against changing it unless there are extremely convincing reasons that the current policy is evil.)

[jnivre]

I agree with Dan. While it may be unfortunate if our policy clashes with the expectations of github users, our use case is in many ways different from the standard one. And changing it now could potentially create total chaos, so we would have to think this through very carefully before doing anything like that.