Skip to content

Conversation

@gadenbuie
Copy link
Member

@gadenbuie gadenbuie commented Jan 29, 2021

Fixing envir_prep and envir_result

This PR fixes rstudio/gradethis#199, namely we use a two-step rendering process to ensure that we accurately return the prep environment in envir_prep.

Previously the envir_prep environment contained the results of evaluating the global setup chunk, but not the exercise-specific setup chunks. With the two step evaluation via rmarkdown::render(), we now guarantee that envir_prep contains the results of the global setup and the exercise setup.

We also ensure that envir_result is a cousin environment duplicated from envir_prep. If an error occurs in the setup code, envir_result and envir_prep are identical. If an error occurs in the user code, envir_result reflects evaluated code up to the error.

New Testing Infrastructure 🏗️

This PR also adds exercise mocking functions that let us create mock exercises without having to write and evaluate a tutorial Rmd. This simplifies testing and lets us directly test render_exercise() and evaluate_exercise(). I focused mostly on testing render_exercise(), so we'll likely want to increase testing on evaluate_exercise() in the near future.

Exercise Checker for Debugging 🚫 🐛

As part of this testing infrastructure, I added a new internal function debug_exercise_checker(). It serves as a counterpart to debug_event_recorder() and may be helpful to tutorial authors who are trying to write custom error checking functions.

Here's a small reprex. By setting the default exercise.checker to debug_exercise_checker, Submit Answer returns the values of the arguments expected by a custom exercise checking function.

---
title: "A tutorial"
output: learnr::tutorial
runtime: shiny_prerendered
---

```{r setup, include=FALSE}
library(learnr)

tutorial_options(exercise.checker = learnr:::debug_exercise_checker)
```

```{r example-setup}
x <- 1
```

```{r example, exercise=TRUE}
# user submits
x <- 4
x - 1
```

```{r example-solution}
x + 3
```

```{r example-check}
# check code
```

image

PR task list:

  • Update NEWS
  • Add tests (if possible)
  • Update documentation with devtools::document()

@gadenbuie gadenbuie requested a review from schloerke January 29, 2021 17:04
Copy link
Contributor

@nischalshrestha nischalshrestha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just had a comment + question.

Copy link
Collaborator

@schloerke schloerke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending the learnr::: in the test is removed

@gadenbuie gadenbuie merged commit 1ac35b1 into rstudio:master Feb 2, 2021
@gadenbuie gadenbuie deleted the fix-envir-prep branch February 2, 2021 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Environment for evaluation of .solution

3 participants