clone_extra_repos.py fails on Windows

[lefebvre]

clone_extra_repos.py fails attempting to import gitdist.

jap@warroom4 MINGW64 ~/projects/tutela
$ TriBITS/tribits/ci_support/clone_extra_repos.py
Traceback (most recent call last):
  File "TriBITS/tribits/ci_support/clone_extra_repos.py", line 45, in <module>
    import gitdist
  File "C:\Users\jap\projects\tutela\TriBITS\tribits\python_utils\gitdist.py", line 1, in <module>
    gitdist
NameError: name 'gitdist' is not defined

[bartlettroscoe]

I know what the problem is. The problem is that gitdist.py is a symlink to gitdist. To fix this on Windows, we just need to make gidist.py the real file and gitdist the symlink. Then, on windows, if you want to use gitdist, just use gitdsit.py (or copy and paste gitdist.py to gitdist where you want to install it).

I will go ahead and make this change.

@lefebvre, What repo are you pulling TriBITS from?

[lefebvre]

For this new project I am working on I have a fork of TriBITSPub/TriBITS.

[bartlettroscoe]

Okay, then I will push the the Github 'master' branch first. Then you can just merge from this main TriBITS 'master' branch to your fork.

If you are making changes to TriBITS, let's try to get then back into the main version so they can be maintained long term. For that, take note of Contributing to TriBITS.

[bartlettroscoe]

Jordan,

Please merge the updated 'master' branch (which I forced pushed to fix the GitHub Issue ID) into your fork and see if that does the trick.

Please give the results one way or the other so that I can confirm.

[lefebvre]

Ross,
It appears I'm running into an new issue.

$ TriBITS/tribits/ci_support/clone_extra_repos.py

**************************************************************************
Script: clone_extra_repos.py \
  --extra-repos='' \
  --not-extra-repos='' \
  --extra-repos-file='cmake/ExtraRepositoriesList.cmake' \
  --extra-repos-type='Nightly' \
  --gitolite-root='' \
  --with-cmake='cmake' \
  --verbosity='more' \
  --do-clone \
  --do-op \
  --create-gitdist-file='' \

Traceback (most recent call last):
  File "TriBITS/tribits/ci_support/clone_extra_repos.py", line 553, in <module>
    cloneExtraRepos(inOptions)
  File "TriBITS/tribits/ci_support/clone_extra_repos.py", line 457, in cloneExtraRepos
    verbose=isVerbosityLevel(inOptions, "most")
  File "TriBITS/tribits/ci_support/clone_extra_repos.py", line 297, in getExtraReposDictListFromCmakefile
    (headerOutput, extraReposPytonDictList) = getHeaderOutputAndExtraReposDictList(rawOutput)
  File "TriBITS/tribits/ci_support/clone_extra_repos.py", line 278, in getHeaderOutputAndExtraReposDictList
    pythonDictList = eval(pythonDictListStr)
  File "<string>", line 0

    ^
SyntaxError: unexpected EOF while parsing

[bartlettroscoe]

@lefebvre, can you provide more info on the context for this so I can try to reproduce?

[lefebvre]

Ross,
It looks like you are getting more then expected. Here is what Windows is getting on STDOUT.

$ TriBITS/tribits/ci_support/clone_extra_repos.py 2> error.txt

**************************************************************************
Script: clone_extra_repos.py \
  --extra-repos='' \
  --not-extra-repos='' \
  --extra-repos-file='cmake/ExtraRepositoriesList.cmake' \
  --extra-repos-type='Nightly' \
  --gitolite-root='' \
  --with-cmake='cmake' \
  --verbosity='more' \
  --do-clone \
  --do-op \
  --create-gitdist-file='' \

raw output:  *** Generate a Python datastructure containing TriBITS/git repos ...
PROJECT_SOURCE_DIR = C:\Users\jap\projects\tutela
TRIBITS_BASE_DIR = C:\Users\jap\projects\tutela\TriBITS\tribits
EXTRA_REPOS_FILE = cmake/ExtraRepositoriesList.cmake
EXTRA_REPOS =
EXTRA_REPOS_PYTHON_OUT_FILE =
ENABLE_KNOWN_EXTERNAL_REPOS_TYPE = Nightly
IGNORE_MISSING_EXTRA_REPOSITORIES =
CHECK_EXTRAREPOS_EXIST = FALSE
PROJECT_NAME = Tutela

Reading the list of extra repositories from cmake/ExtraRepositoriesList.cmake

NOTICE: the repo HYSPLIT is a Mercurial repo: these are tolerated, but not fully supported.
NOTICE: the repo APT is a Mercurial repo: these are tolerated, but not fully supported.

Selecting the set of 'Nightly' extra repos (asserting all selected repos exist) ...

-- Adding POST extra Continuous repository DELFIC ...
-- Adding POST extra Continuous repository HYSPLIT ...
-- Adding POST extra Continuous repository DELFIC-HYSPLIT ...
-- Adding POST extra Continuous repository APT ...

*** Extra Repositories Python Dictionary
[
{'NAME' : 'DELFIC', 'DIR' : 'DELFIC', 'REPOTYPE' : 'GIT', 'REPOURL' : 'http://ci.ornl.gov:8000/scm/git/delfic', 'HASPKGS' : 'HASPACKAGES', 'PREPOST' : 'POST', 'CATEGORY' : 'Continuous', },
{'NAME' : 'HYSPLIT', 'DIR' : 'HYSPLIT', 'REPOTYPE' : 'HG', 'REPOURL' : 'https://fogbugz.ornl.gov/kiln/code/Fallout/group/hysplit', 'HASPKGS' : 'HASPACKAGES', 'PREPOST' : 'POST', 'CATEGORY' : 'Continuous', },
{'NAME' : 'DELFIC-HYSPLIT', 'DIR' : 'DELFIC-HYSPLIT', 'REPOTYPE' : 'GIT', 'REPOURL' : 'https://code.ornl.gov/RNSD-NSM/DELFIC-HYSPLIT.git', 'HASPKGS' : 'HASPACKAGES', 'PREPOST' : 'POST', 'CATEGORY' : 'Continuous', },
{'NAME' : 'APT', 'DIR' : 'APT', 'REPOTYPE' : 'HG', 'REPOURL' : 'https://fogbugz.ornl.gov/kiln/code/Fallout/group/APT', 'HASPKGS' : 'HASPACKAGES', 'PREPOST' : 'POST', 'CATEGORY' : 'Continuous', },
]

My guess is that you are just expecting the array structure.

[bartlettroscoe]

The clone_extra_repos.py script has not been tested with Hg repos. It would need to be extended for Hg repos. You can look at the automated tests for clone_extra_repos.py in the file:

 TriBITS/test/ci_support/clone_extra_repos_UnitTests.py

It should be much easier to add Hg support to the clone_extra_repos.py script than say the checkin-test.py script.