what if you need pandoc?

[jennybc]

Currently, all builds use the --no-multiarch switch for checking, and all vignettes (and the VignetteBuilder entry in DESCRIPTION) are removed prior to building (due to the absence of pandoc and LaTeX which are likely to be needed).

What if the package you are checking needs pandoc to run its tests? Is there a way to make it available?

All my tests fails with this error:

pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available).

I believe I requested a successful installation of pandoc via:

before_test:
  - cinst pandoc

but that does not seem to help.

[krlmlr]

Thanks. Could you check if you can actually access pandoc after installing it?

before_test:
- cinst pandoc
- pandoc -v

[jennybc]

No, apparently I cannot.

cinst pandoc
Installing the following packages:
pandoc
By installing you accept licenses for the packages.
mm-choco.extension v0.0.4 [Approved]
 Installed/updated mm-choco extensions.
 The install of mm-choco.extension was successful.
  Software installed to 'C:\ProgramData\chocolatey\extensions\mm-choco'
pandoc v1.18 [Approved]
Downloading pandoc 
  from 'https://github.com/jgm/pandoc/releases/download/1.18/pandoc-1.18-windows.msi'
Progress: 100% - Saving 20.96 MB of 21.02 MB (21974597/22040576)   
Download of pandoc-1.18-windows.msi (21.02 MB) completed.
Hashes match.
Installing pandoc...
pandoc has been installed.
pandoc installed to 'C:\Program Files (x86)\Pandoc'
Adding pandoc to the PATH if needed
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).
 The install of pandoc was successful.
  Software installed to 'C:\Program Files (x86)\Pandoc\'
Chocolatey installed 2/2 packages. 0 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
pandoc -v
'pandoc' is not recognized as an internal or external command,
operable program or batch file.
Command exited with code 1

[jennybc]

But I note this instruction, which I've clearly not addressed:

Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).

[jennybc]

I inserted refreshenv prior to pandoc -v and can report some progress wrt pandoc. But the bad news is the R package now fails to build entirely (I've already shut testing down on appveyor).

Here's proof that pandoc is installed and callable.

The install of pandoc was successful.
  Software installed to 'C:\Program Files (x86)\Pandoc\'
Chocolatey installed 2/2 packages. 0 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
refreshenv
Reading environment variables from registry. Please wait... Done.
pandoc -v
pandoc 1.18
Compiled with pandoc-types 1.17.0.4, texmath 0.8.6.6, highlighting-kate 0.6.3
Default user data directory: C:\Users\appveyor\AppData\Roaming\pandoc
Copyright (C) 2006-2016 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

And here's where the R package build goes off the rails:

[00:00:50] Building with: R CMD build --no-manual
[00:00:50] + [[ MSYS_ == \M\I\N\G\W ]]
[00:00:50] + R CMD build --no-manual .
[00:00:50] ../travis-tool.sh: line 274: R: command not found
[00:00:50] Command exited with code 127
[00:00:50] 7z a failure.zip *.Rcheck\*
[00:00:50] 
[00:00:50] 7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
[00:00:50] 
[00:00:50] Scanning the drive:
[00:00:50] 0 files, 0 bytes
[00:00:50] 
[00:00:50] Creating archive: failure.zip
[00:00:50] 
[00:00:50] Items to compress: 0
[00:00:50] 
[00:00:50] 
[00:00:50] Files read from disk: 0
[00:00:50] Archive size: 22 bytes (1 KiB)
[00:00:50] Everything is Ok
[00:00:50] appveyor PushArtifact failure.zip
[00:00:50] 'appveyor' is not recognized as an internal or external command,
[00:00:50] operable program or batch file.
[00:00:50] Command exited with code 1

In a successful appveyor build (where I don't refresh the environment vars and don't run the tests), the first bit of that looks like so:

[00:00:45] Building with: R CMD build --no-manual
[00:00:45] + [[ MINGW == \M\I\N\G\W ]]
[00:00:45] + [[ -d vignettes ]]
[00:00:45] + R CMD build --no-manual .
[00:00:46] * checking for file './DESCRIPTION' ... OK
[00:00:46] * preparing 'reprex':
[00:00:46] * checking DESCRIPTION meta-information ... OK
[00:00:46] * checking for LF line-endings in source and make files
[00:00:46] * checking for empty or unneeded directories
[00:00:46] * building 'reprex_0.0.0.9001.tar.gz'

[krlmlr]

@FeodorFitsner: Any idea?

[gaborcsardi]

FWIW R-hub has binaries for pandoc, you might want to try to download these and place them in your path: https://files.r-hub.io/pandoc/pandoc-1.17.2/windows/
The windows ones were built by RStudio.

[krlmlr]

Looks like refreshenv overwrites the PATH entries, which r-appveyor inserted for R, and also which AppVeyor itself inserted for the appveyor tool. Can you try echo $PATH and/or echo %PATH% after refreshenv, and then add the pandoc path manually instead of writing refreshenv?

[jennybc]

Here's the data on PATH before and after refreshenv.

@krlmlr What specifically is your suggestion for adding the pandoc path manually? Alternatively, dare I fiddle even more with environment variables and try to set the one rmarkdown consults for the pandoc path?

before_raw <- readLines("appveyor-path-before.txt")
after_raw <- readLines("appveyor-path-after.txt")

before_split <- strsplit(before_raw, split = ";")[[1]]
after_split <- strsplit(after_raw, split = ";")[[1]]

setdiff(before_split, after_split)
#> [1] "C:\\R\\bin\\i386"                         
#> [2] "C:\\MinGW\\msys\\1.0\\bin"                
#> [3] "C:\\Program Files (x86)\\Git\\bin"        
#> [4] "C:\\Program Files\\AppVeyor\\BuildAgent\\"
setdiff(after_split, before_split)
#> [1] "C:\\Program Files (x86)\\Pandoc\\"          
#> [2] "C:\\Users\\appveyor\\AppData\\Roaming\\npm "

[krlmlr]

One of the following two might do the job:

• PowerShell, via http://stackoverflow.com/a/1333717/946850

  before_test:
  - cinst pandoc
  - $env:Path += ";C:\Program Files (x86)\Pandoc\"

• cmd, folklore

  before_test:
  - cinst pandoc
  - set PATH="%PATH%;C:\Program Files (x86)\Pandoc\"

[jennybc]

It's working 🎉!

In the end, all I needed was this:

before_test:
  - cinst pandoc
  - ps: $env:Path += ";C:\Program Files (x86)\Pandoc\"

It would be nice if pandoc was cached. Right now I am installing it afresh upon every single build. I think I see how to do that? But making it available via the proper r-appveyor machinery would be even better.

Thanks for all the help!

[jennybc]

Caching pandoc ended up being simple. My cache directives look like this now:

cache:
  - C:\RLibrary
  - C:\Program Files (x86)\Pandoc\

[krlmlr]

Thanks a lot, this brings us a step closer to full pandoc support in r-appveyor. I'm downloading and installing full R and Rtools afresh every time, the download literally takes seconds from RStudio's cloud. What's the cached size of pandoc, can you see this in AppVeyor?

[jennybc]

I spoke too soon. It is being cached but I am not successfully preventing the re-installation yet 😔. I look around for the cache size while I try to figure this out.

[krlmlr]

Have you tried installr's install.pandoc()?