use litedown for rendering vignettes #6583

ben-schwen · 2024-10-19T20:43:11Z

Closes #6394
Towards #6584

codecov · 2024-10-19T20:49:31Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.77%. Comparing base (84b0e32) to head (ea7eec4).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6583      +/-   ##
==========================================
- Coverage   98.77%   98.77%   -0.01%     
==========================================
  Files          81       81              
  Lines       15215    15213       -2     
==========================================
- Hits        15029    15027       -2     
  Misses        186      186

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ben-schwen · 2024-10-20T21:02:44Z

This PR is on hold until current GH version of xfun makes its way to CRAN.

MichaelChirico · 2024-12-03T07:47:00Z

Earmark: after #6589, if {knitr} is not in Suggests it should be in Enhances.

MichaelChirico · 2025-01-21T20:41:20Z

NB: After #6618, we'll also want to check this issue whether it should be included, or left as a TODO: yihui/litedown#67

yihui

NB: After #6618, we'll also want to check this issue whether it should be included, or left as a TODO: yihui/litedown#67

It's safe to use litedown:::.env$input with the current CRAN version of litedown. The next CRAN release should take place in a few weeks, and you'll be able to use litedown::get_context('input').

vignettes/datatable-benchmarking.Rmd

Co-authored-by: Yihui Xie <xie@yihui.name>

rikivillalba · 2025-03-09T20:15:57Z

Sorry I meant to push a sub-branch to merge but instead forget to switch and pushed the commit directly.
The commit updates the translation script with litedown and changes instances of `r ...` to `{r} ...` as is recognized by litedown

MichaelChirico · 2025-03-09T21:00:27Z

vignettes/datatable-faq.Rmd

@@ -26,9 +26,9 @@ h2 {
 }
 </style>

-```{r echo=FALSE, file='_translation_links.R'}
+```{r, echo=FALSE, file='_translation_links.R'}


i imagine this will later trigger a test for un-named chunks, are names supported in this context?

jangorecki · 2025-04-21T07:21:49Z

xfun is fixed on CRAN so we could proceed with this issue, isn't it? @ben-schwen

ben-schwen · 2025-04-21T18:01:14Z

xfun is fixed on CRAN so we could proceed with this issue, isn't it? @ben-schwen

Yes. Only blocker, I'm aware of now is that := is printing on litedown and not printing on knitr

Since we register a method for knitr::knit_print, keep it mentioned in the DESCRIPTION.

As with knit_print, avoid printing x when !shouldPrint(x). Use the overloaded method in the vignettes instead of relying on the auto-printing detection in print.data.table.

aitap · 2025-04-22T10:26:23Z

This can be solved similarly to #6589, except that methods for xfun::record_print must return the printed output, not print it:

library(data.table)
library(xfun)
x <- data.table(a = 1)
xfun::record('x[,a:=1]') |> print(browse = FALSE)
# x[,a:=1]
# 
# #>        a
# #>    <num>
# #> 1:     1
#
.S3method(
  'record_print', 'data.table',
  \(x, ...) if (!shouldPrint(x)) character() else NextMethod()
)
xfun::record('x[,a:=1]') |> print(browse = FALSE)
# x[,a:=1]
# 
xfun::record('x[,a:=1][]') |> print(browse = FALSE)
# x[,a:=1][]
# 
# #>        a
# #>    <num>
# #> 1:     1
#

Additionally, inline expressions of the form `r ... ` aren't parsed the same way. For example, data.table-intro.Rmd uses

In (1), for each group, a vector is returned, with length = 6,4,2 here. However, (2) returns a list of length 1 for each group, with its first element holding vectors of length 6,4,2. Therefore, (1) results in a length of ` 6+4+2 = `r 6+4+2``, whereas (2) returns `1+1+1=`r 1+1+1``.

...and with litedown, the inline expressions inside code blocks are not evaluated.

At some point we'll need to transition the translated vignettes too, right?

MichaelChirico · 2025-07-08T16:42:22Z

inst/tests/other.Rraw

@@ -182,6 +182,9 @@ if (loaded[["knitr"]]) {
  # Which is fine and works thanks to cedta().
  DT = data.table(x=1, y=2)
  test(11, kable(DT), output="x.*y.*1.*2")
+  invisible(knit(testDir("knitr.Rmd"), quiet=TRUE))


I think we can delete the Rout.mock, right? IIRC that only exists to mock out when {knitr} is not installed, but other.Rraw should always be run where all the required packages are available, in dev.

We may way to migrate this check to litedown

R/onLoad.R

MichaelChirico · 2025-07-08T16:44:19Z

tests/knitr.R

Should we instead replace this with tests/litedown.R?

At a minimum we should add a regression test of the new xfun method in other.Rraw.

vignettes/datatable-importing.Rmd

MichaelChirico

I think this is ready to ship after maybe another round of touch-up as commented.

Feel free to mark any of my comments for follow-up.

Great work!

jangorecki · 2025-07-08T18:49:32Z

Are there links for rendered versions?

aitap

In datatable-intro.Rmd, by-group j-expressions that call print() render like:

DT[, print(.SD), by = ID]

#        a     b     c
#    <int> <int> <int>
# 1:     1     7    13
# 2:     2     8    14
# 3:     3     9    15
#        a     b     c
#    <int> <int> <int>
# 1:     4    10    16
# 2:     5    11    17
#        a     b     c
#    <int> <int> <int>
# 1:     6    12    18

ID

I wonder where is the last ID coming from.

I couldn't make the `some text `r expression(...)`` syntax work in Litedown and wrote a replacement function for datatable-intro.Rmd.

In datatable-reference-semantics.Rmd, the CSS class syntax #### {.bs-callout .bs-callout-info} does not work; it produces a level-4 heading titled {.bs-callout .bs-callout-info} instead.

R/onLoad.R

vignettes/datatable-importing.Rmd

MichaelChirico · 2025-07-10T16:46:51Z

NB: please Update branch before merging & possibly include #7151.

aitap · 2025-07-10T19:05:03Z

Unfortunately, pkgdown::build_site does not understand the new `{r} expression...` syntax in the vignettes, so the translation marks are lost, and so is other dynamic text.

MichaelChirico · 2025-07-12T23:36:27Z

Per the {litedown} README, {litedown} itself is supposed to do the site rendering, right?

Otherwise, we should ask the {pkgdown} maintainers to support {litedown} rendering; IINM, the issue is that {pkgdown} relies on {rmarkdown} to render:

https://github.com/r-lib/pkgdown/blob/824ab4a4f20ecd895282b434d8652120e1f62ba4/R/build-article.R#L294
https://github.com/r-lib/pkgdown/blob/824ab4a4f20ecd895282b434d8652120e1f62ba4/R/build-article.R#L114

jangorecki · 2025-07-13T03:49:30Z

It does support basic website.
https://github.com/jangorecki/pkgup/blob/a1ec87d02209423a28c432c92b7269a086cd3e7b/.github/workflows/pkgup.yaml#L56
https://jangorecki.github.io/pkgup/

pkgdown could support it anyway, but not sure if and how fast that will happen.
pkgdown did not support well markdown vignettes in the past, only rmarkdown.

ChristianWia · 2025-07-24T15:44:21Z

comparing the results of html vignette generation with knitr and litedown on "data-table-joins.Rmd" :
litedownKnitrCompared.txt

knitr/litedown diff screenshot section 7.2:

litedown resulting file to follow the comments:
litedownJointuresavecdatatable.pdf

My Windows environment :

R version 4.0.2 (2020-06-22) -- "Taking Off Again"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

installation from CRAN source packages : ‘commonmark’, ‘xfun’, ‘litedown’

URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/commonmark_2.0.0.tar.gz'
URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/xfun_0.52.tar.gz'
URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/litedown_0.7.tar.gz'

jangorecki · 2025-07-24T19:01:22Z

comparing the litedownKnitrCompared.txt results of html vignette generation with knitr and litedown on "data-table-joins.Rmd"

This may be as well interesting for @yihui

ben-schwen added 2 commits October 19, 2024 22:13

add first steps

fedc6ef

add litedown R CMD check occasional

b1f1adb

ben-schwen added 4 commits October 19, 2024 23:26

remove knitr::opts_chunk

2fc3ddc

set comment char

ac1fb44

add error option

905586d

update comment char

1c14712

ben-schwen added 3 commits November 2, 2024 23:02

adjust print

aa7bdad

remove space

071078c

fix namespace

1f347bd

This was referenced Dec 3, 2024

Solve the knitr auto-printing problem by registering a method for knit_print #6589

Merged

Automate lang links #6618

Merged

yihui reviewed Jan 22, 2025

View reviewed changes

vignettes/datatable-benchmarking.Rmd Outdated Show resolved Hide resolved

ben-schwen and others added 4 commits March 8, 2025 21:33

Update vignettes/datatable-benchmarking.Rmd

e1d0c6f

Co-authored-by: Yihui Xie <xie@yihui.name>

use yihuis suggestions

4e7809f

Merge branch 'master' into litedown

c653cf6

upd translation script and r -> {r} code blocks

4bacd15

MichaelChirico reviewed Mar 9, 2025

View reviewed changes

ben-schwen added 2 commits March 24, 2025 22:49

Merge branch 'master' into litedown

a1819e1

move knitr tests

5ed155d

Merge branch 'master' into litedown

e1329dd

aitap added 2 commits April 22, 2025 12:50

Enhances: knitr

e4b7912

Since we register a method for knitr::knit_print, keep it mentioned in the DESCRIPTION.

Register xfun::record_print method for data.table

1e191d5

As with knit_print, avoid printing x when !shouldPrint(x). Use the overloaded method in the vignettes instead of relying on the auto-printing detection in print.data.table.

ben-schwen requested review from a team, jangorecki and tdhock as code owners July 7, 2025 21:17

jangorecki requested review from aitap, yihui and MichaelChirico July 8, 2025 09:21

nocov range

9f35906

MichaelChirico reviewed Jul 8, 2025

View reviewed changes

R/onLoad.R Show resolved Hide resolved

MichaelChirico reviewed Jul 8, 2025

View reviewed changes

vignettes/datatable-importing.Rmd Show resolved Hide resolved

MichaelChirico approved these changes Jul 8, 2025

View reviewed changes

aitap added 3 commits July 9, 2025 00:45

Add missing periods.

9a49b1d

Fix footnote

010c238

Fix remaining r ... expressions

d01e83c

aitap reviewed Jul 8, 2025

View reviewed changes

R/onLoad.R Show resolved Hide resolved

vignettes/datatable-importing.Rmd Show resolved Hide resolved

ChristianWia mentioned this pull request Jul 24, 2025

add french version of datatable-joins.Rmd #7151

Merged

ben-schwen added 5 commits July 28, 2025 11:44

Merge branch 'master' into litedown

d9300ad

update joins vignette

e32a5db

set/unset threads

a6afc0a

sys.date

bae1e3c

add xfun to enhances

ea7eec4

use litedown for rendering vignettes #6583

Are you sure you want to change the base?

use litedown for rendering vignettes #6583

Uh oh!

Conversation

ben-schwen commented Oct 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Oct 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ben-schwen commented Oct 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MichaelChirico commented Dec 3, 2024

Uh oh!

MichaelChirico commented Jan 21, 2025

Uh oh!

yihui left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rikivillalba commented Mar 9, 2025

Uh oh!

MichaelChirico Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

jangorecki commented Apr 21, 2025

Uh oh!

ben-schwen commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aitap commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MichaelChirico Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

jangorecki Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MichaelChirico Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

jangorecki Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MichaelChirico left a comment

Choose a reason for hiding this comment

Uh oh!

jangorecki commented Jul 8, 2025

Uh oh!

aitap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MichaelChirico commented Jul 10, 2025

Uh oh!

aitap commented Jul 10, 2025

Uh oh!

MichaelChirico commented Jul 12, 2025

Uh oh!

jangorecki commented Jul 13, 2025

Uh oh!

ChristianWia commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jangorecki commented Jul 24, 2025

Uh oh!

Uh oh!

ben-schwen commented Oct 19, 2024 •

edited

Loading

codecov bot commented Oct 19, 2024 •

edited

Loading

ben-schwen commented Oct 20, 2024 •

edited

Loading

yihui left a comment •

edited

Loading

ben-schwen commented Apr 21, 2025 •

edited

Loading

aitap commented Apr 22, 2025 •

edited

Loading

ChristianWia commented Jul 24, 2025 •

edited

Loading