Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package] [ci] Add Windows CI for R package (fixes #2335) #2936

Merged
merged 93 commits into from
Apr 26, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
19ed854
first test of appveyor
jameslamb Mar 21, 2020
16476da
strings are strings
jameslamb Mar 21, 2020
f7573f0
lil bit of build script
jameslamb Mar 21, 2020
311d3d8
fixing paths
jameslamb Mar 21, 2020
c422b15
removed unnecessary file
jameslamb Mar 21, 2020
b2fce60
updated CRAN URL
jameslamb Mar 21, 2020
b973b7d
added a lot more printing
jameslamb Mar 21, 2020
a8873f6
fixing paths
jameslamb Mar 21, 2020
952ca55
more stuff
jameslamb Mar 21, 2020
b4cf398
fixed paths
jameslamb Mar 21, 2020
cc6d02b
more stuff
jameslamb Mar 21, 2020
adfb832
more path guessing
jameslamb Mar 21, 2020
92b0032
even more paths
jameslamb Mar 22, 2020
8d77941
more stuff
jameslamb Mar 22, 2020
b7e2abe
moar logz
jameslamb Mar 22, 2020
12fb638
and now for something totally different
jameslamb Mar 22, 2020
1091504
please work
jameslamb Mar 22, 2020
96dcc18
ok could be a thing
jameslamb Mar 22, 2020
97d9913
changing directories
jameslamb Mar 22, 2020
6ed498d
we might be in business
jameslamb Mar 22, 2020
b275d22
fixed install syntax
jameslamb Mar 22, 2020
f701325
tryinv mingw
jameslamb Mar 22, 2020
618086a
more mingw
jameslamb Mar 22, 2020
90f91e5
ignore Suggests check
jameslamb Mar 22, 2020
2f4979a
trying Azure DevOps
jameslamb Mar 22, 2020
3b07c7b
just run bare minimum for Azure DevOps
jameslamb Mar 22, 2020
2d17cae
fixed build dir thing
jameslamb Mar 22, 2020
2e9fe09
trying to set libPaths
jameslamb Mar 22, 2020
855cdc9
more testing
jameslamb Mar 22, 2020
6d8878b
trying R 3.6.3
jameslamb Mar 22, 2020
c854fe3
R 3.6.3
jameslamb Mar 22, 2020
86f4eec
this feels right
jameslamb Mar 22, 2020
8b12858
still messing around with libraries
jameslamb Mar 22, 2020
dc50ad5
more paths
jameslamb Mar 22, 2020
065df95
removed duplication in Windows testing code
jameslamb Mar 22, 2020
432df25
simpler
jameslamb Mar 22, 2020
6e73047
fixed conda stuff
jameslamb Mar 22, 2020
fec3de7
more conda stuff
jameslamb Mar 22, 2020
9b90d84
more fixes
jameslamb Mar 22, 2020
c305e9d
fixed testing script
jameslamb Mar 22, 2020
c70e352
moved AppVeyor setup to the top
jameslamb Mar 22, 2020
517e519
commenting
jameslamb Mar 22, 2020
56bf28a
ch-ch-ch-ch-chaaaanges
jameslamb Mar 22, 2020
f439ba2
paths
jameslamb Mar 22, 2020
2923111
plz work
jameslamb Mar 22, 2020
65915a1
fixed conda stuff in Windows CI
jameslamb Mar 22, 2020
f18399d
uncommented stuff to test a full build
jameslamb Mar 23, 2020
1de8f6c
fixed quotes and removed some unnecessary stuff
jameslamb Mar 23, 2020
a912213
added install.libs.R change
jameslamb Mar 23, 2020
290e5b7
quotes are impoortant
jameslamb Mar 23, 2020
2565df7
added commented-out stuff back in
jameslamb Mar 23, 2020
cd89065
added Windows script, download retries, and MSVC linking
jameslamb Mar 24, 2020
fbf494c
minor fixes
jameslamb Mar 25, 2020
244a14a
cleaned up debugging code in FindLibR
jameslamb Mar 29, 2020
42609cd
cleaned up debugging code and moved R first in CI
jameslamb Mar 29, 2020
2804d45
fixed vsts-ci indentation
jameslamb Mar 31, 2020
af3c004
cut documentation stuff out of MSVC build
jameslamb Mar 31, 2020
1c498d0
fix R CMD check for Azure
jameslamb Mar 31, 2020
9e6aa49
misc whitespace changes
jameslamb Mar 31, 2020
76d264d
merge in #2962
jameslamb Apr 2, 2020
01359ab
Merge branch 'master' into ci/r-windows-ci
jameslamb Apr 4, 2020
d100027
Added echoing of build logs from R CMD check
jameslamb Apr 4, 2020
912a2fd
cut out more documentation tests
jameslamb Apr 4, 2020
d76b551
fixed NOTE about imports from Matrix
jameslamb Apr 4, 2020
7cdfc62
Merge branch 'master' into ci/r-windows-ci
jameslamb Apr 4, 2020
1a2053f
Merge branch 'master' into ci/r-windows-ci
jameslamb Apr 13, 2020
bdf10e0
Merge branch 'master' into ci/r-windows-ci
jameslamb Apr 15, 2020
88a7a09
moved some changes out of this PR and into #2963
jameslamb Apr 15, 2020
ae813f2
fixed whitespace stuff
jameslamb Apr 15, 2020
1b2363a
added check on number of NOTES
jameslamb Apr 15, 2020
dd52dbf
adding better checks
jameslamb Apr 15, 2020
7f9eb7c
fixing check on NOTEs
jameslamb Apr 15, 2020
374ba31
removing unnecessary variable
jameslamb Apr 15, 2020
30d8af1
Update .ci/test_r_package_windows.ps1
jameslamb Apr 15, 2020
e3b9ac8
some changes
jameslamb Apr 15, 2020
419e39b
fix quoting
jameslamb Apr 15, 2020
bfebf1d
trying MINGW on Azure DevOps
jameslamb Apr 16, 2020
3e2dcfd
fixing paths
jameslamb Apr 16, 2020
8ba021a
more paths
jameslamb Apr 16, 2020
51e4d2d
fixing paths
jameslamb Apr 16, 2020
5d74019
testing paths
jameslamb Apr 16, 2020
d7c160b
fixing slashes
jameslamb Apr 16, 2020
a8582db
pinned CTAN mirror
jameslamb Apr 17, 2020
8264f4a
get better logs
jameslamb Apr 17, 2020
d843795
made sure Azure finds MinGW, fixed search for LIBR_CORE_LIBRARY, stop…
jameslamb Apr 19, 2020
10bd18d
Apply suggestions from code review
jameslamb Apr 19, 2020
6ca2be8
added CXX, CC for Windows builds and changed back to building docs on…
jameslamb Apr 19, 2020
cdf1df8
stored LIBR_CORE_LIBRARY hints in one variable
jameslamb Apr 19, 2020
9c59f33
Apply suggestions from code review
jameslamb Apr 20, 2020
747378e
changes from code review
jameslamb Apr 20, 2020
c0b87d6
increased parallel builds for Azure CI
jameslamb Apr 20, 2020
6820fc6
Apply suggestions from code review
jameslamb Apr 21, 2020
7075b80
Merge branch 'master' into ci/r-windows-ci
jameslamb Apr 25, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
first test of appveyor
  • Loading branch information
jameslamb committed Apr 1, 2020
commit 19ed854700b35fafaa4d665ae28287be06ccc2af
33 changes: 19 additions & 14 deletions .appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@ configuration: # a trick to construct a build matrix with multiple Python versi
environment:
matrix:
- COMPILER: MSVC
TASK: r-package
R_WINDOWS_VERSION: 3.6.1
- COMPILER: MINGW
TASK: r-package
R_WINDOWS_VERSION: 3.6.1

clone_depth: 5

Expand Down Expand Up @@ -41,17 +45,18 @@ build_script:
python setup.py install)

test_script:
- pytest %APPVEYOR_BUILD_FOLDER%\tests\python_package_test
- cd %APPVEYOR_BUILD_FOLDER%\examples\python-guide
- ps: >-
@("import matplotlib", "matplotlib.use('Agg')") + (Get-Content "plot_example.py") | Set-Content "plot_example.py" # prevent interactive window mode
(Get-Content "plot_example.py").replace('graph.render(view=True)', 'graph.render(view=False)') | Set-Content "plot_example.py"
- ps: >-
foreach ($file in @(Get-ChildItem *.py)) {
@("import sys, warnings", "warnings.showwarning = lambda message, category, filename, lineno, file=None, line=None: sys.stdout.write(warnings.formatwarning(message, category, filename, lineno, line))") + (Get-Content $file) | Set-Content $file
python $file
if (!$?) { $host.SetShouldExit(-1) }
} # run all examples
- cd %APPVEYOR_BUILD_FOLDER%\examples\python-guide\notebooks
- conda install -q -y -n test-env ipywidgets notebook
- jupyter nbconvert --ExecutePreprocessor.timeout=180 --to notebook --execute --inplace *.ipynb # run all notebooks
# - pytest %APPVEYOR_BUILD_FOLDER%\tests\python_package_test
# - cd %APPVEYOR_BUILD_FOLDER%\examples\python-guide
# - ps: >-
# @("import matplotlib", "matplotlib.use('Agg')") + (Get-Content "plot_example.py") | Set-Content "plot_example.py" # prevent interactive window mode
# (Get-Content "plot_example.py").replace('graph.render(view=True)', 'graph.render(view=False)') | Set-Content "plot_example.py"
# - ps: >-
# foreach ($file in @(Get-ChildItem *.py)) {
# @("import sys, warnings", "warnings.showwarning = lambda message, category, filename, lineno, file=None, line=None: sys.stdout.write(warnings.formatwarning(message, category, filename, lineno, line))") + (Get-Content $file) | Set-Content $file
# python $file
# if (!$?) { $host.SetShouldExit(-1) }
# } # run all examples
# - cd %APPVEYOR_BUILD_FOLDER%\examples\python-guide\notebooks
# - conda install -q -y -n test-env ipywidgets notebook
# - jupyter nbconvert --ExecutePreprocessor.timeout=180 --to notebook --execute --inplace *.ipynb # run all notebooks
- cmd /c "powershell -ExecutionPolicy Bypass -File %APPVEYOR_BUILD_FOLDER%\.ci\test_windows.ps1"
60 changes: 58 additions & 2 deletions .ci/test_windows.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,10 @@ elseif ($env:TASK -eq "bdist") {
cp @(Get-ChildItem *.whl) $env:BUILD_ARTIFACTSTAGINGDIRECTORY
}

$tests = $env:BUILD_SOURCESDIRECTORY + $(If ($env:TASK -eq "sdist") {"/tests/python_package_test"} Else {"/tests"}) # cannot test C API with "sdist" task
pytest $tests ; Check-Output $?
if ($env:TASK -ne "r-package") {
$tests = $env:BUILD_SOURCESDIRECTORY + $(If ($env:TASK -eq "sdist") {"/tests/python_package_test"} Else {"/tests"}) # cannot test C API with "sdist" task
pytest $tests ; Check-Output $?
}

if ($env:TASK -eq "regular") {
cd $env:BUILD_SOURCESDIRECTORY/examples/python-guide
Expand All @@ -50,3 +52,57 @@ if ($env:TASK -eq "regular") {
conda install -q -y -n $env:CONDA_ENV ipywidgets notebook
jupyter nbconvert --ExecutePreprocessor.timeout=180 --to notebook --execute --inplace *.ipynb ; Check-Output $? # run all notebooks
}

# test R package
# based on https://github.com/RGF-team/rgf/blob/master/R-package/.R.appveyor.ps1
if ($env:TASK -eq "r-package"){

Import-CliXml .\env-vars.clixml | % { Set-Item "env:$($_.Name)" $_.Value }
tzutil /s "GMT Standard Time"
[Void][System.IO.Directory]::CreateDirectory($env:R_LIB_PATH)

$env:PATH = "$env:R_LIB_PATH\Rtools\bin;" + "$env:R_LIB_PATH\R\bin\x64;" + "$env:R_LIB_PATH\miktex\texmfs\install\miktex\bin\x64;" + $env:PATH
$env:BINPREF = "C:/mingw-w64/x86_64-8.1.0-posix-seh-rt_v6-rev0/mingw64/bin/"

# set up R if it doesn't exist yet
if (!(Get-Command R.exe -errorAction SilentlyContinue)) {

# download R and RTools
(New-Object System.Net.WebClient).DownloadFile(https://cloud.r-project.org/bin/windows/base/R-${env:R_WINDOWS_VERSION}-win.exe, ./R-win.exe)
(New-Object System.Net.WebClient).DownloadFile(https://cloud.r-project.org/bin/windows/Rtools/Rtools35.exe, ./Rtools.exe)

# Install R
Start-Process -FilePath .\R-win.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /DIR=$env:R_LIB_PATH\R /COMPONENTS=main,x64"
Start-Process -FilePath .\Rtools.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /DIR=$env:R_LIB_PATH\Rtools"

# download Miktex
(New-Object System.Net.WebClient).DownloadFile(https://miktex.org/download/win/miktexsetup-x64.zip, ./miktexsetup-x64.zip)
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory("miktexsetup-x64.zip", "miktex")
.\miktex\miktexsetup.exe --local-package-repository=.\miktex\download --package-set=essential --quiet download
.\miktex\download\miktexsetup.exe --portable="$env:R_LIB_PATH\miktex" --quiet install
}

initexmf --set-config-value [MPM]AutoInstall=1
conda install -y --no-deps pandoc

Add-Content .Renviron "R_LIBS=$env:R_LIB_PATH"
Add-Content .Rprofile "options(repos = 'https://cran.rstudio.com')"
Add-Content .Rprofile "options(pkgType = 'binary')"
Add-Content .Rprofile "options(install.packages.check.source = 'no')"

Rscript -e "install.packes(c('data.table', 'jsonlite', 'Matrix', 'R6', 'testthat'), dependencies = c('Imports', 'Depends', 'LinkingTo'))" ; Check-Output $?

Rscript build_r.R ; Check-Output $?

$PKG_FILE_NAME = Get-Item *.tar.gz
$PKG_NAME = $PKG_FILE_NAME.BaseName.split("_")[0]
$LOG_FILE_NAME = "$PKG_NAME.Rcheck/00check.log"

R.exe CMD check "${PKG_FILE_NAME}" --as-cran --no-multiarch; Check-Output $?

if (Get-Content "$LOG_FILE_NAME" | Select-String -Pattern "WARNING" -Quiet) {
echo "WARNINGS have been found by R CMD check!"
Check-Output $False
}
}
3 changes: 3 additions & 0 deletions .vsts-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,9 @@ jobs:
bdist:
TASK: bdist
PYTHON_VERSION: 3.5
rpkg:
TASK: r-package
R_WINDOWS_VERSION: 3.6.1
steps:
- powershell: Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
displayName: Enable conda
Expand Down