|
| 1 | +GitHub, Git, and related topics |
| 2 | +=============================== |
| 3 | + |
| 4 | +GitHub |
| 5 | +------ |
| 6 | + |
| 7 | +Open MPI's Git repositories are `hosted at GitHub |
| 8 | +<https://github.com/open-mpi/ompi>`_. |
| 9 | + |
| 10 | +#. First, you will need a Git client. We recommend getting the latest |
| 11 | + version available. If you do not have the command ``git`` in your |
| 12 | + path, you will likely need to download and install Git. |
| 13 | +#. `ompi <https://github.com/open-mpi/ompi/>`_ is the main Open MPI |
| 14 | + repository where most active development is done. Git clone this |
| 15 | + repository. Note that the use of the ``--recursive`` CLI option is |
| 16 | + necessary because Open MPI uses Git submodules:: |
| 17 | + |
| 18 | + shell$ git clone --recursive https://github.com/open-mpi/ompi.git |
| 19 | + |
| 20 | +Note that Git is natively capable of using many forms of web |
| 21 | +proxies. If your network setup requires the user of a web proxy, |
| 22 | +`consult the Git documentation for more details |
| 23 | +<https://git-scm.com/>`_. |
| 24 | + |
| 25 | +Git commits: open source / contributor's declaration |
| 26 | +---------------------------------------------------- |
| 27 | + |
| 28 | +In order to remain open source, all new commits to the Open MPI |
| 29 | +repository must include a ``Signed-off-by:`` line, indicating the |
| 30 | +submitter's agreement to the :ref:`Open MPI Contributor's Declaration |
| 31 | +<contributing-contributors-declaration-label>`. |
| 32 | + |
| 33 | +.. tip:: You can use the ``-s`` option to ``git commit`` to |
| 34 | + automatically add the ``Signed-off-by:`` line to your commit |
| 35 | + message. |
| 36 | + |
| 37 | +.. _git-github-branch-scheme-label: |
| 38 | + |
| 39 | +Git branch scheme |
| 40 | +----------------- |
| 41 | + |
| 42 | +Generally, Open MPI has two types of branches in its Git repository: |
| 43 | + |
| 44 | +#. ``main``: |
| 45 | + |
| 46 | + * All active development occurs on the ``main`` branch (new features, |
| 47 | + bug fixes, etc.). |
| 48 | + |
| 49 | +#. Release branches of the form ``vMAJOR.MINOR.x`` (e.g., ``v4.0.x``, |
| 50 | + ``v4.1.x``, ``v5.0.x``). |
| 51 | + |
| 52 | + * The ``.x`` suffix indicates that this branch is used to create |
| 53 | + all releases in the Open MPI vMAJOR.MINOR series. |
| 54 | + * Periodically, the Open MPI community will make a new release |
| 55 | + branch, typically from ``main``. |
| 56 | + * A Git tag of the form ``vMAJOR.MINOR.RELEASE`` is used to |
| 57 | + indicate the specific commit on a release branch from where |
| 58 | + official Open MPI release tarball was created (e.g., ``v4.1.0``, |
| 59 | + ``v4.1.1``, ``v4.1.2``, etc.). |
| 60 | + |
| 61 | +Once a bug is fixed or a new feature is implemented on ``main``, it is |
| 62 | +cherry-picked over to the relevant release branch(es). |
| 63 | + |
| 64 | +.. attention:: It may seem odd to some, but the Open MPI community |
| 65 | + development model does *not* PR bug fixes or new |
| 66 | + features directly to release branches. Instead, |
| 67 | + initial bug-fix / feature PRs are generally first made |
| 68 | + to ``main``. |
| 69 | + |
| 70 | + This helps us ensure that future releases (with |
| 71 | + ``main`` as a Git ancestor) will contain the bug fix / |
| 72 | + feature. |
| 73 | + |
| 74 | +For example: |
| 75 | + |
| 76 | +.. code:: sh |
| 77 | +
|
| 78 | + shell$ git checkout main |
| 79 | + shell$ git pull --rebase |
| 80 | + shell$ git checkout pr/bug-fix |
| 81 | +
|
| 82 | + # ... make changes / fix a bug / etc. ... |
| 83 | +
|
| 84 | + shell$ git add ... |
| 85 | + shell$ git commit -s ... |
| 86 | + shell$ git push myfork |
| 87 | +
|
| 88 | +At this point, you go create a PR from your fork's ``pr/bug-fix`` |
| 89 | +branch to the Open MPI community GitHub repo ``main`` branch. Work |
| 90 | +with the community to get the PR completed and merged. Then you can |
| 91 | +open a new PR to cherry pick the Git commits from that bug fix to each |
| 92 | +of the relevant release branches. |
| 93 | + |
| 94 | +Depending on how far the release branch has diverged from ``main``, |
| 95 | +there may be some porting effort involved in the cherry-pick. |
| 96 | + |
| 97 | +For example, if your bug fix on ``main`` is comprised of a single Git |
| 98 | +commit hash ``123abc``: |
| 99 | + |
| 100 | +.. code:: sh |
| 101 | +
|
| 102 | + # Fetch all upstream git activity, including the merge of the "main" PR. |
| 103 | + shell$ get fetch --all |
| 104 | +
|
| 105 | + # Check out the target release branch, and advance to the most recent commit. |
| 106 | + shell$ git checkout v5.0.x |
| 107 | + shell$ git pull --rebase |
| 108 | +
|
| 109 | + # Make a branch for your bug fix |
| 110 | + shell$ git checkout -b pr/v5.0.x/bug-fix |
| 111 | + # Cherry pick the commit from the "main" branch |
| 112 | + shell$ git cherry-pick -x 123abc |
| 113 | + # Push to your fork |
| 114 | + shell$ git push myfork |
| 115 | +
|
| 116 | +The Open MPI development community *requires* adding the following |
| 117 | +line to the commit message of cherry-picked commits on release |
| 118 | +branches: |
| 119 | + |
| 120 | +.. code:: text |
| 121 | +
|
| 122 | + (cherry picked from commit [git_hash_of_original_commit]) |
| 123 | +
|
| 124 | +.. note:: Note the use of the ``-x`` option to ``git cherry-pick``. |
| 125 | + This option automatically adds the ``(cherry picked from |
| 126 | + ...)`` line to your commit message. |
| 127 | + |
| 128 | +.. admonition:: Rationale |
| 129 | + :class: tip |
| 130 | + |
| 131 | + Git does not actually store any meta data about Git cherry-picks in |
| 132 | + the commit. Having a standardized text line containing the source |
| 133 | + Git commit hash in the commit messages helps the Open MPI |
| 134 | + development community track where commits came from on release |
| 135 | + branches, and therefore allows us to check whether all relevant |
| 136 | + commits have been ported to a given release branch. |
| 137 | + |
| 138 | +Once your commits are ready and pushed up to your fork, make a PR to |
| 139 | +the target release branch. |
| 140 | + |
| 141 | +.. warning:: A GitHub PR CI job checks all commits on release branches |
| 142 | + for the ``(cherry picked from...)`` line. It will also |
| 143 | + ensure that the Git hash cited in that line actually |
| 144 | + exists on the ``main`` branch. |
| 145 | + |
| 146 | + This check ensures that commits are not made to release |
| 147 | + branches before their corresponding ``main`` PR was |
| 148 | + merged. |
| 149 | + |
| 150 | +All this being said, sometimes there is a need for a non-cherry-picked |
| 151 | +commit on a release branch. E.g., sometimes a release branch has |
| 152 | +diverged so much that the bug no longer exists on ``main``. It would |
| 153 | +therefore not make sense |mdash| or even be impossible |mdash| to |
| 154 | +commit the bug fix in question to ``main``. |
| 155 | + |
| 156 | +In such cases, make a regular PR to the target branch (with commits |
| 157 | +that do *not* include ``(cherry picked from ...)`` lines). In the PR |
| 158 | +description, add a line with the following token: |
| 159 | + |
| 160 | +.. code:: text |
| 161 | +
|
| 162 | + bot:notacherrypick |
| 163 | +
|
| 164 | +This tells the GitHub CI job that this PR contains commits that are |
| 165 | +not cherry-picked from ``main``. |
| 166 | + |
| 167 | +.. warning:: ``bot:notacherrypick`` should only be used when |
| 168 | + absolutely necessary. It is not a license to avoid |
| 169 | + the process of PR'ing to ``main`` first. |
| 170 | + |
| 171 | +CI (testing) |
| 172 | +------------ |
| 173 | + |
| 174 | +The Open MPI community generally runs two flavors of testing: |
| 175 | + |
| 176 | +#. A bunch of tests on each PR (Continuous Integration / CI). These |
| 177 | + tests are a mixture of GitHub Actions and other CI systems (e.g., |
| 178 | + Jenkins). Examples include (but are not limited to): |
| 179 | + |
| 180 | + * Check each Git commit for bozo email addresses |
| 181 | + * Check that each Git commit contains a ``Signed-off-by`` line |
| 182 | + * Check that commits on release branches contain a cherry-pick |
| 183 | + notice |
| 184 | + * Build and publish the docs |
| 185 | + * Build Open MPI in a variety of environments and run sanity tests |
| 186 | + with that installation |
| 187 | + |
| 188 | +#. Daily testing via the MPI Testing Tool (MTT). |
| 189 | + |
| 190 | + * These are generally tests that take much longer to run than on a |
| 191 | + per-PR basis. `A "nightly snapshot" tarball |
| 192 | + <https://www.open-mpi.org/nightly/>`_ is created for ``main`` and |
| 193 | + each relevant release branch. |
| 194 | + * MTT tests are run with this snapshot tarball so that all |
| 195 | + organizations are testing with the same snapshots. |
| 196 | + * `Results are available here <https://mtt.open-mpi.org/>`_. |
0 commit comments