Prairielearn on Datahub?

[ericvd-ucb]

Summary

There seems to be interest in what it would take to use Datahub for a deployment of prairielearn - an open source online assessment tool. I was just talking to Andrew Bray and he was wanting to explore the possibilities for Summer 23 version of Stat 20.

This could have implications for classes across campus / across RTL world

The tool is already used by Dan Garcia and Armando Fox for a few CS classes, and I think they are currently running a server for it. Maybe one starting point would be to talk to them about their deployment

https://prairielearn.readthedocs.io/en/latest/running-in-production/setup/

Need to understand if this could work with our architecture

Instructors who want to do some online assessments, could be quizzes or other assessments

Acceptance criteria

Stat 20 could dogfood it out Summer 23 ?

Important information

• check in on our own limited capacity to start new projects!
• Talk to Finsen Chiu about how hard this would be
• figure out what are the elements that need to be engineered

[balajialg]

Seems like there is a docker image for JupyterLab-PrairieLearn integration which should make this a possibility

• https://hub.docker.com/r/prairielearn/workspace-jupyterlab/tags
• https://hub.docker.com/r/prairielearn/workspace-jupyterlab/dockerfile

Wondering if there is a proxy endpoint for Prairie Learn just like how there is one for RStudio, VSCode, etc.

Also, RTL already has a premium subscription for IClickers and PollEverywhere. What would the value add of a tool like PrairieLearn over these pre-existing tools? Also, Can PrairieLearn be integrated to bcourses instead of Jupyterhub - https://prairielearn.readthedocs.io/en/latest/courseInstance/

Also UBC uses PrairieLearn but not sure whether that is integrated into their campus specific Jupyterhub deployments - https://ctlt.ubc.ca/2022/07/28/edubytes-computational-tools/

[ryanlovett]

@balajialg PrarieLearn's workspaces seem to be for deploying jupyterlab, rstudio, and other apps from prarielearn rather than the other way around using jupyter-server-proxy.

https://github.com/PrairieLearn/PrairieLearn/tree/master/workspaces

If this is deployed somewhere locally, I'd be up for seeing a demo.

[balajialg]

Thanks @ryanlovett for the clarification. That makes this a complex integration. I am also curious about local PrairieLearn deployments.

fwiw, there seems to be a demo deployment - https://us.prairielearn.com/pl/course_instance/4970/assessments

Wondering the desirability of this use case given the alternatives that exist,

• https://caen.engin.umich.edu/online-assessment-tools/ (Canvas Quizzes)
• https://pkgs.rstudio.com/learnr/articles/questions.html (LearnR package for creating quizzes)
• https://github.com/jmshea/jupyterquiz (Jupyter Quiz Creator) and https://github.com/jmshea/jupytercards (Jupyter Flash cards)

[andrewpbray]

@balajialg To my assessment, PrairieLearn is considerably more sophisticated than the platforms you list. I understand it to be both a structure in which to build a question/quiz bank and also serve up a UI to instructors (to craft quizzes) and students (to take the quizzes). By sophisticated I mean:

• The quiz bank is built around "question generators" instead of static questions. These are questions written with many variables in them so that you use to make many variants of the same question.
• The question generators permit very rich meta data that allows these quiz banks to get really rich and flexible / easily queriable over time.
• The classes of questions / tasks can be quite rich: diagram manipulation, parsons problems, and setting that provide an interface to a live computing environment.

[balajialg]

Thanks @andrewpbray! That's super helpful. I did not factor in your 2nd and 3rd points as part of my filtering criteria for quiz tools. Have you tried their integration with bcourses? (Check LTI section in the docs for more info - https://prairielearn.readthedocs.io/en/latest/courseInstance/)

[balajialg]

Seems like two universities have SSO with PraireLearn. We can reach out to them to understand their deployments if the EECS outreach is not working out.

[andrewpbray]

No, I haven't tried the LTI integration (I don't use bCourses for my courses and don't yet have a PL server set up), but I imagine that could be helpful for some instructors.

[shaneknapp]

pinging @finsenchiu (COE CBT) for some PL insight.

@andrewpbray and @ericvd-ucb : i'm confused when i read "deploying prairelearn in datahub". does this just mean that it's a prairielearn instance that we're hosting, or is it somehow integrated in to datahub?

for the former, if we (UC) have a managed/hosted solution already in place, it seems to me that the hosting should happen there.

if the latter (integrating in to datahub), do you have any examples of what you're thinking?

[shaneknapp]

Seems like there is a docker image for JupyterLab-PrairieLearn integration which should make this a possibility

• https://hub.docker.com/r/prairielearn/workspace-jupyterlab/tags
• https://hub.docker.com/r/prairielearn/workspace-jupyterlab/dockerfile

Wondering if there is a proxy endpoint for Prairie Learn just like how there is one for RStudio, VSCode, etc.

missed this reply -- i assume this is what we're talking about?

;)

[ryanlovett]

@shaneknapp That appears to be a way to run jupyterlab from prairielearn.

I'd be interested in seeing a demo of PL just to get a sense of how it might overlap, integrate, or play nice with jupyterhub.

[andrewpbray]

For my use case I'd be happy with either the existing UC hosted PL server or one deployed via JupyterHub. One use case for the latter that @ericvd-ucb mentioned to me the other day is for other institutions to spin up a PL server with minimal overhead. If courses like CS 10 and Data 8 have curricula that gets bundled for community colleges, with computing provided via a datahub, then the question was whether we could also share the quiz bank / assessment part of a curriculum (PL) through similar infrastructure.

[ericvd-ucb]

It sounds like its a pretty tough problem but the reasons I thought that it might be worth it

• Jupyter server proxy could authentication , who is in what course, which permissions
• Maybe its a way to generate PL scaling ( Stat 20 has 800 students) of the server
• Brays analogy was something like the shiny deployment
• Could RTL take on prarielearn service for more classes beyond EECS

Reasons its not worth is

• prairielearn expects to have some sort of database behind it and that would be problematic
• its just too much work that we dont have cycles for
• less community supporting it
• engineerign problems Im not thinking about

[ericvd-ucb]

@ryanlovett
https://cbt-dev.berkeley.edu/pl

[balajialg]

and a report about this deployment - https://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-156.html

[ryanlovett]

• Jupyter server proxy could authentication , who is in what course, which permissions

JupyterHub does authentication, as does PL. jupyter-server-proxy sits behind the hub's authentication in the singleuser server, and proxies web apps that run within the user's server on JupyterHub. For example, RStudio, VS Code, and Shiny are web apps that j-s-p proxies. If PL is a web app that is meant to be, or merely can be, run as a single user, then we could install and proxy it. If PL is really meant to be a multi-user web service like GitHub or Canvas (bcourses), or if for practical purposes it needs to be run multi-user, then jupyterhub's single user environment is not the place to install it.

Let's say that PL can be run in a singleuser environment and proxied, and that each datahub user has access to their own instance, but the data isn't shared between datahub users. Is that desirable or useful?

I just took a look at the installation instructions and they're limited to using a docker container (with native install not recommended). This is often the case when installation is non-trivial and requires specific versioned components. Its developer guide mentions that Postgres is the database (it'd be easier if PL was db-agnostic via sqlalchemy or some such). This all suggests to me that it is intended to be a multiuser service and that installation is not as trivial as installing an app like RStudio or Shiny. So if we could run it per-user via its docker container, it would probably have to be run as a kind of sidecar. But again, is a non-shared per-user instance useful?

For multiuser apps, JupyterHub can accommodate services that can sit behind JH authentication, and z2jh could deploy such services on our cluster, but we haven't done that before. It would need a support person who understands how to deploy and maintain the app, and run it on kubernetes.

[ericvd-ucb]

@ryanlovett thanks so much for this careful and useful explanation!