Suppress Unreleased Toolboxes in Develop Branch

[dankessler]

The current develop branch of UMPsychMethodsCore contains many folders that we do not intend to "release" in our initial methods core roll-out (hereafter called subtrees).

After much thinking about this, I've settled on what I think is the best way to handle this without rewriting tons of history.

This was presented at an impromptu meeting last week with @mangstad @sripada and @heffjos. I wanted to make sure @rcwelsh and @shijiaguo are on board, too.

We can think of many of these subtrees as projects or "toolboxes". For example, we have an svmbatch toolbox, a toolbox of ExtractionScripts, a toolbox for FirstLevel, etc.

Currently develop looks like this

ExtractionScripts
FirstLevel
*matlabScripts*
MotionRegressors
MotionSummary
PhysioSummary
QualityChecks
README
SecondLevel
som
*spm8*
*spm8Batch*
svmbatch

The plan for release 1.0 is that the released version should only include a subset of these trees, namely, those enclosed in asterisks.

So what do we do with all of these other trees? Ideally, they'd only be contained in dangling feature branches that haven't been merged into develop, but we didn't think that far ahead when we set things up. We could rewrite history using git-filter-branch and interactive rebasing and other sorts of git black magic, but that'd require a lot of thinking and coordination. We want a solution that enables us to write more history on top of our agreed upon history. Here's what I propose:

1. We check out a new branch called core/hide_unreleased from current version of develop
2. We introduce a new series of commits. Each will simply remove one of the toolboxes that are not ready for release.
3. We mark the beginning of these sets of commits with a tag, let's call it mc_CleanUp_Start
4. We mark the final of these commits with a tag, too, let's call it mc_CleanUp_End

The end result of this gets the branch name release and will serve as the first release candidate.

Here's a visual explanation of how merging will happen moving forward

         D1---D2---D3 feature D
       /
A1---A2---B1---B2---B3---B4 release & develop
 \
  C1---C2---C3 feature C

We have four sets of commits:

A is develop at you might find it now. Each contains all of the subtrees discussed above. A2 may introduce a few changes not contained in A1.

B are a series of commits that remove each of the subtrees that are meant to be hidden. The final result, B4 in this case, only contains subtrees intended for release.

C is a feature for toolbox C, whose development is based off of A1. Let's say that C is a feature for a toolbox that survived the pruning in B, so its subtree has persisted in B4.

D is a feature for toolbox D with development based off of A2. Let's say that D is a feature for a toolbox that did not survive the pruning in B, so its subtree no longer exists in B4.

We might also tag B4 as Release 1.0

Moving Forward: Merging in Features for Future Release

Merging a feature to an already released toolbox

Suppose it is time to merge in feature C to prepare for a future release. This will be relatively straightforward.

Recall that the current history looks like

         D1---D2---D3 feature D
       /
A1---A2---B1---B2---B3---B4 release & develop
 \
  C1---C2---C3 feature C

We can directly submit a pull request, namely requesting that the following occur

         D1---D2---D3 feature D
       /
A1---A2---B1---B2---B3---B4---E1 develop
 |            _______________/
 \           /
  C1---C2---C3 feature C

This merge should work fine, since git will perform a recursive merge. It will identify A1 as the common ancestor of B4 and C3, and see that C1...C3 only touches one subtree, namely /FeatureC/...

So, even though C3 contained a bunch of subtrees that B4 does not, they will not be reintroduced, nor will they lead to merge conflicts.

Preparing a Toolbox for Release And Merging a Feature Into It

Suppose the decision is made that toolbox D should be prepared for release, and that feature D needs to be merged into it. More specifically, let's suppose that commit B2 is the commit that removed toolbox D.

Recall that the current history looks like

         D1---D2---D3 feature D
       /
A1---A2---B1---B2---B3---B4 release & develop
 \
  C1---C2---C3 feature C

If we were to just try to checkout B4, then run git merge feature D things wouldn't work because D3 introduces changes relative to A1 (the common parent) that touch the path /ToolboxD, but the changes on B4 relative to A1 deleted this path.

Instead, we will do the following:

1. Checkout B4 as a new branch, call it ToolboxD_Activate_FeatureD_Merge
2. Run the following command git revert B2. This will create a new commit, let's call it B2*, as it will undo the changes introduced in B2. Effectively, this will reintroduce ToolboxD
3. Run this command git merge featureD. This will merge in the changes from feature D. This will now succeed because the delta from A1 to D3, which touches /ToolboxD, can now merge in with B2*. Let's call the result of this merge F1

Afterward, our repository will look like this

         D1---D2---D3--------------
       /                           \
A1---A2---B1---B2---B3---B4---B2*---F1
 \
  C1---C2---C3 feature C

Summary

I will take care of introducing the B range of commits. We will be left with a lot of features in progress. Some of these will apply to toolboxes that have been released. Moving forward, merging this into develop will be straightforward as it was before. For the features that apply to toolboxes not released, they will first have to be "unhidden" using a revert, and then merged into develop.

[dankessler]

This issue is now closed with the adoption of pull request #79