Automate randomization of answers, questions, and pages

[StefanMunnes]

Description

Hello again!
I've had this idea in mind for a while, but now it would be really handy for a new project. In social science surveys and experiments, we often want to randomize the order of questions and answers to avoid order effects.

Coming from LimeSurvey, there is a simple option in the UI to set a randomized group name for questions and question groups (pages) and set the randomization option for answer options. I know I could use a workaround that sets the content of the questions and answer options dynamically in the app's server, but I would also like to have the ability to set this as a default or with an option for the relevant questions.

This could look like the following:

sd_question(
  id    = "favorite_penguin",
  type  = "mc",
  label = "Which type of penguin do you like the best?",
  randomize_group = "penguin_fruit",
  option_randomize = TRUE,
  option = c(
    "Adélie"    = "adelie",
    "Chinstrap" = "chinstrap",
    "Gentoo"    = "gentoo"
  )
)

sq_question(
  type = "mc",
  id = "fruits",
  label = "What is your favorite fruit?",
  randomize_group = "penguin_fruit",
  option = c(
    "Apple" = "apple",
    "Banana" = "banana",
    "Dragon Fruit" = "dragon_fruit",
    "Blood Orange" = "blood_orange"
  )

The advantage of something like this is obvious: you have a clean and structured user input for the questions and answers. This would also works perfectly with the import/export of the _questions.yml file.

Ideally, there would also be a variable that contains the actual order of the questions and answers.

Having something similar for the pages (question groups) would be really nice, but I can image this is way harder to archive with the filter and redirect to specific pages.

[jhelvy]

I like this idea, not sure how to implement it yet though. If I'm understanding it correctly, the option_randomize argument is TRUE or FALSE to randomize the options for a single question. I suppose it'd be ignored for questions that don't have options, e.g. text type questions.

For randomize_group, my understanding is that all questions in the same group would be displayed in a random order, is that correct? We'd probably have to add some restrictions, like the questions would need to be on the same page, and they would need to be inserted without any text in between them. I'm afraid that if the questions spanned different pages that could break things, and likewise having text in between questions might create some issues. Is that what lime survey does?

If we go with these restrictions, then a different UI might be to handle this in the server side. We could have an option to randomize the order of all questions on a given page, in which case all you need to specify is the page_id. So something like:

sd_server(
  randomize_questions = c("page1", "demographics")
)

This would indicate that the order of all questions on pages with the IDs page1 and demographics would be displayed in a random order. This avoids the need to specify a randomize_group name, which if done inside sd_question() would require you to copy that argument name to each question on the page, which could get annoying. I also think this would make it easier to implement, because we already keep track of the list of question_ids on each page, so we could just insert a tiny bit of code to check if the page is randomized or not, and if so randomize the id order before inserting the content.

Now, for randomizing the order of the pages themselves, that could get a lot trickier. I would still think a similar approach should be taken, meaning we handle this in the server, something like:

sd_server(
  randomize_pages  = c("page1", "demographics")
)

In this case, we're saying the pages page1 and demographics should come in a randomized order. Again, I think this could be handled in a similar way where the randomization would happen earlier inside the sd_server() function. Once we have the content all inside a list, we would just need to swap around the order. But again, I think we should put in restrictions where the pages need to at least be in a consecutive order in the survey.qmd file. If there are any pages in between them, that could mess up navigation. But so long as they're specified all in sequential order, we should be able to more easily randomize the order before displaying the pages.

What do you think? The names randomize_questions and randomize_pages could maybe be improved, but that's just the first thing I thought of. Happy to review a PR if you want to try implementing this!