Unable to use prex_rtf() under R Studio/Windows

[friendly]

I'm trying to use reprex:::prex_rtf() on Windows under R Studio, but I get an error saying highlight is not installed. What is the problem?

> library(reprex)
> prex_rtf()
Error: `highlight` command line tool doesn't appear to be installed.
Therefore, `venue = "rtf"` is not supported.
> packageVersion("highlight")
[1] ‘0.5.0’
> packageVersion("reprex")
[1] ‘0.3.0.9001’

# insert reprex here

[batpigandme]

You need to have the highlight command line tool on your system (outside of R).
http://www.andre-simon.de/doku/highlight/en/highlight.php
The Windows installer is here.
On a Mac, you can install this with homebrew:

brew install highlight

but you can see the details of installation or download the source package via the highlight page.

[friendly]

Pls consider re-opening this.
I have installed highlight v. 3.57 from the link. Installed includes the command line tool, C:\Program Files\Highlight\highlight.exe.

C:\Users\friendly\Dropbox\Documents\SCS\VisMLM-course>"C:\Program Files\Highligh
t\highlight.exe" --version

 highlight version 3.57
 Copyright (C) 2002-2020 Andre Simon <a dot simon at mailbox.org>

 Argparser class
 Copyright (C) 2006-2008 Antonio Diaz Diaz <ant_diaz at teleline.es>

 Artistic Style Classes (3.1)
 Copyright (C) 2006-2018 by Jim Pattee <jimp03 at email.com>
 Copyright (C) 1998-2002 by Tal Davidson

 Diluculum Lua wrapper (1.0)
 Copyright (C) 2005-2013 by Leandro Motta Barros

 xterm 256 color matching functions
 Copyright (C) 2006 Wolfgang Frisch <wf at frexx.de>
....

I still get the same highlight not found error.

Q:

• How does reprex search for highlight? The installation did not put it on my PATH, but I modified the PATH to add it.

• Perhaps an environmental variable could be used to specify it?

• More generally, a useful fallback would be just to produce the code and output w/out highlighting, or an option, e.g., highlight=T/F in the arguments

[batpigandme]

Happy to reopen. You didn't mention in your initial comment that you actually had highlight installed.
Here is how highlight is found:

reprex/R/highlight.R

Line 27 in d3fc4b8

highlight_found <- function() Sys.which("highlight") != ""

What happens when you run Sys.which("highlight")?

Sys.which("highlight")
#                  highlight 
# "/usr/local/bin/highlight" 

When you run Sys.getenv("PATH") the modified path that includes highlight shows up, right?

[friendly]

Thanks so much for this. Perhaps there are other Windows users who have faced this problem.

For some reason, R still cannot find highlight via Sys.which("highlight") but it does now appear on my PATH

> Sys.which("highlight")
highlight 
       "" 
> Sys.getenv("PATH")
[1] "C:\\R\\R-3.6.3\\bin\\x64;C:\\Rtools\\bin;C:\\Program Files\\ImageMagick-7.0.9-Q16;c:\\Rtools\\mingw_32\\bin;c:\\Rtools\\gcc-4.6.3\\bin;C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Perl64\\site\\bin;C:\\Perl64\\bin;c:\\Rtools\\gcc-4.6.3\\bin;C:\\Program Files\\ImageMagick-6.8.1-Q16;C:\\Program Files (x86)\\Intel\\iCLS Client\\;C:\\Program Files\\Intel\\iCLS Client\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files (x86)\\SecureCRT\\;C:\\Program Files (x86)\\MiKTeX 2.9\\miktex\\bin\\;C:\\Program Files\\SlikSvn\\bin;C:\\Program Files\\Git\\cmd;C:\\Program Files (x86)\\IDM Computer Solutions\\UltraCompare\\;C:\\Program Files (x86)\\IDM Computer Solutions\\UltraEdit\\;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\friendly\\AppData\\Local\\Pandoc\\;C:\\batchfiles\\;C:\\Program Files\\gs\\gs9.52\\bin\\;C:\\Program Files\\Highlight\\highlight.exe"
> 

Perhaps this is a Windows issue with the highlight command line tool.

But it seems to me that regexp(..., venue="rtf") or prex_rtf() should not have such a hard depend on an external tool, without a fall-back that wouldn't trigger an error.

Given this, I'm surprised that {reprex} passes CRAN checks.

[jennybc]

FWIW the prex_*() functions are unexported and are strictly "use at your own risk".

But it seems to me that regexp(..., venue="rtf") or prex_rtf() should not have such a hard depend on an external tool, without a fall-back that wouldn't trigger an error.

I think this section of the help and the error is quite clear:

https://reprex.tidyverse.org/reference/reprex.html#syntax-highlighting

Error: `highlight` command line tool doesn't appear to be installed.
Therefore, `venue = "rtf"` is not supported.

I don't think anyone has ever used it before on Windows, so I'd be happy to get a PR that makes it actually work.

[friendly]

I don't blame {reprex} for this. Yes, the functions that use venue="rtf" are experimental and advertised to "use at own risk".
Is this good enough?

Just to be clear: this is really about reprex(..., venue="rtf") not just the unexported reprex:::prex_rtf()

But it doesn't work for Windows users. I'm surprised no one has tested this before, and that {reprex} passed CRAN checks with this flaw.

I'm not sufficiently skilled to make a PR, but surely it would entail something like.

highlight_found <- function() Sys.which("highlight") != "" 
if (highlight_found) {}  # use highlight
else {}                        # don't

[jennybc]

I think it's better to throw an error if we can't do something, as reprex(venue = "rtf") currently does for you.

These functions are meant to be run interactively.

[cderv]

@friendly One hint about your PATH but Sys.which not finding it, mentioned in #331 (comment).

Currently you have this in your PATH, from what you posted

C:\\Program Files\\Highlight\\highlight.exe"

I think you should not have the path to the exe file directly in the PATH but instead the path to folder where to look.
If you add only instead

C:\\Program Files\\Highlight

I guess Sys.which('highlight') will find something.

Hope it helps

[friendly]

For the record, @cderv found the PATH problem so far. Please note again that I'm not blaming the nice work on {reprex}, but trying to contribute to a solution that works for Windows users.

• Including only C:\\Program Files\\Highlight on the PATH results in

  friendly@UIT-HHP-31TRNW1 ....
  $ which highlight
  /c/Program Files/Highlight/highlight

In R, I now get:

> Sys.which("highlight")
                             highlight 
"C:\\PROGRA~1\\HIGHLI~1\\HIGHLI~2.EXE" 

Yet, even with this, I get a further and unfathomable error. Is there some way to test this further?

> prex_rtf()
Rendering reprex...
Writing file 1 of 3: C:/Users/friendly/AppData/Local/Temp/RtmpuIfbL7/reprex28cc19047a95/reprex_reprex_rendered.R.rtf
highlight: Lua error ( cannot open C:\PROGRA~1\HIGHLI~1\langDefs\>.lang: Invalid argument ) in >.lang
Error: `highlight` call unsuccessful.

[jennybc]

I would first try to have a successful call to highlight in a shell, because this suggests there are problems with Windows and its tendency to convert to '8.3' paths and perhaps R itself is also getting in the way? R on Windows has various gotchas like this. After success in a shell, I'd move on to calling highlight from R via reprex.

In the article on the RTF venue, I give an example of the type of highlight command we are executing:

highlight foo_reprex.R --out-format rtf --no-trailing-nl --encoding=UTF-8 --style darkbone --font 'Courier Regular' --font-size 50 > foo_reprex.rtf

Possibly relevant:
https://stackoverflow.com/questions/34030087/how-to-find-correct-executable-with-sys-which-on-windows

[cderv]

FWIW I can confirm I can reproduce this on windows 10. highlight is found correctly in PATH with Sys.which and the command used by reprex works in Terminal but not from R. there seems to be an issue when R is executing the command 🤔

# Highlight is correctly found
Sys.which("highlight")
#> highlight
#> "C:\\Users\\chris\\Logiciel\\HIGHLI~1.1-X\\HIGHLI~2.EXE"
system("highlight --version")
#> 
#> highlight version 3.57
#> Copyright (C) 2002-2020 Andre Simon <a dot simon at mailbox.org>
#> 
#> Argparser class
#> Copyright (C) 2006-2008 Antonio Diaz Diaz <ant_diaz at teleline.es>
#> 
#> Artistic Style Classes (3.1)
#> Copyright (C) 2006-2018 by Jim Pattee <jimp03 at email.com>
#> Copyright (C) 1998-2002 by Tal Davidson
#> 
#> Diluculum Lua wrapper (1.0)
#> Copyright (C) 2005-2013 by Leandro Motta Barros
#> 
#> xterm 256 color matching functions
#> Copyright (C) 2006 Wolfgang Frisch <wf at frexx.de>
#> 
#> This software is released under the terms of the GNU General Public License.
#> For more information about these matters, see the file named COPYING.
#> 
#> [1] 0
# I can reproduce the error
reprex:::prex_rtf(1+1)
#> Rendering reprex...
#> Writing file 1 of 3: C:/Users/chris/AppData/Local/Temp/Rtmp6nV497/reprex52cc159a148d/reprex_reprex_rendered.R.rtf
#> highlight: Lua error ( cannot open C:\Users\chris\Logiciel\HIGHLI~1.1-X\langDefs\>.lang: Invalid argument ) in >.lang
#> Erreur : `highlight` call unsuccessful. >
# simulate reprex
# to test the command
temp_R <- tempfile(fileext = ".R")
temp_rtf <- tempfile(fileext = ".rtf")
brio::write_lines("1 + 1\n#> [1] 2", temp_R)

bin <- "highlight"
args <- c(temp_R, "--out-format=rtf", "--no-trailing-nl", "--encoding=UTF-8",
          +           "--style=darkbone", "--font=\"Courier Regular\"",
          +           "--font-size=50", ">", temp_rtf)
cmd <- paste(c(bin, args), collapse = " ")
cmd
#> [1] "highlight C:\\Users\\chris\\AppData\\Local\\Temp\\Rtmp6nV497\\file52cc7fbe7f03.R --out-format=rtf --no-trailing-nl --encoding=UTF-8 --style=darkbone --font=\"Courier Regular\" --font-size=50 > C:\\Users\\chris\\AppData\\Local\\Temp\\Rtmp6nV497\\file52cc34cd1774.rtf"

# works in the terminal
# require RStudio
id <- rstudioapi::terminalExecute(cmd, show = FALSE)
rstudioapi::terminalExitCode(id)
#> NULL
file.exists(temp_rtf)
#> [1] FALSE
xfun::file_string(temp_rtf)
#> Error in file(con, "r") : impossible d'ouvrir la connexion
#> De plus : Warning message:
#> In file(con, "r") :
#> impossible d'ouvrir le fichier 'C:\Users\chris\AppData\Local\Temp\Rtmp6nV497\file52cc34cd1774.rtf' : No such file or directory
unlink(temp_rtf)

# do not work from R
system(cmd)
#> Writing file 1 of 3: file52cc7fbe7f03.R.rtf
#> highlight: Lua error ( cannot open C:\Users\chris\Logiciel\HIGHLI~1.1-X\langDefs\>.lang: Invalid argument ) in >.lang
#> [1] 1
system2(bin, args)
#> Writing file 1 of 3: file52cc7fbe7f03.R.rtf
#> highlight: Lua error ( cannot open C:\Users\chris\Logiciel\HIGHLI~1.1-X\langDefs\>.lang: Invalid argument ) in >.lang
processx::run(bin, args, echo_cmd = TRUE)
#> Running highlight "C:\Users\chris\AppData\Local\Temp\Rtmp6nV497\file52cc7fbe7f03.R" \
#> "--out-format=rtf" --no-trailing-nl "--encoding=UTF-8" "--style=darkbone" \
#> "--font=\"Courier Regular\"" "--font-size=50" ">" \
#> "C:\Users\chris\AppData\Local\Temp\Rtmp6nV497\file52cc34cd1774.rtf"
#> Error in processx::run(bin, args, echo_cmd = TRUE) :
#> System command 'highlight' failed, exit status: 1, stderr:
#> E> highlight: Lua error ( cannot open C:\Users\chris\Logiciel\highlight-3.57.1-x64\langDefs\>.lang: Invalid argument ) in >.lang
#> Type .Last.error.trace to see where the error occured
sys::exec_wait(bin, args)
#> Writing file 1 of 3: file52cc7fbe7f03.R.rtf
#> highlight: Lua error ( cannot open C:\Users\chris\Logiciel\highlight-3.57.1-x64\langDefs\>.lang: Invalid argument ) in >.lang
#> [1] 1

I don't know what this highlight error is and why it fails but hihglight and R does not seem to play well on window, and it is not a reprex issue per-se

[friendly]

Thx @cderv for tracking this further.

It does indicate an error when highlight is called from R, even if it is found by Sys.which(): highlight cannot find the langDefs file.

Thus, it is an issue for {reprex}, in the sense that venue="rtf" does not work as advertised under Windows. At the very least, this should be noted in the docs.

I return to my earlier questions:

• Would an environmental variable, HIGHLIGHT_CMD help?
• Is it not possible to have a fall-back to produce RTF w/o needing highlight?

[jennybc]

Thus, it is an issue for {reprex}, in the sense that venue="rtf" does not work as advertised under Windows. At the very least, this should be noted in the docs.

This has always been noted in the docs, as I have linked before:

https://reprex.tidyverse.org/reference/reprex.html#syntax-highlighting

I will paste that into this thread for efficiency:

A secondary use case for reprex is to produce syntax highlighted code snippets, with or without rendering, to paste into presentation software, such as Keynote or PowerPoint. Use venue = "rtf".

This feature is experimental and currently rather limited. It probably only works on macOS and requires the installation of the highlight command line tool, which can be installed via homebrew. This venue is discussed in an article.

There are many packages and indeed R itself that work differently (and often less well) on Windows. This is commonly noted in documentation. This is an example.

I am not willing to remove this feature, which is documented to be experimental, that works fine for people on *nix, because we don't currently have it working on Windows.

It would be great if we can figure out the incantation that makes this work on Windows.

Is it not possible to have a fall-back to produce RTF w/o needing highlight?

I'm open to suggestions of a package on CRAN that can generate the RTF we need.