Running "create assignment" as a standalone module

[psychemedia]

Is it possible to install various components of nbgrader separately?

For example, is there a way I can just install:

• the create_assignment extension so that I could mark up a notebook for use in an assignment (but not do anything else with it);
• the formgrader extension, so that I could point it to a directory containing student directories/scripts and either just manually, or automatically and manually, grade those scripts?

Our workflow is such that different people create and mark assignments, with submission and return handled manually / using another process. Ideally, we want only want to install the components that individuals have to use, rather than the whole nbgrader environment.

[jhamrick]

Yes and no. You can't install separate parts of nbgrader separately, but different components of nbgrader can be used in a somewhat modular way. For the extensions, you can disable whichever ones you don't want to use (see https://nbgrader.readthedocs.io/en/stable/user_guide/installation.html#disabling-extensions). For the create_assignment extension specifically, you can use it with any notebook (it doesn't have to be in an nbgrader course directory), though if you want to then use nbgrader to generate the student version (via "nbgrader assign") then the notebook would have to be in an nbgrader course directory. For the formgrader, you do need to follow the course directory layout and the database needs to know about the assignment (meaning you'll have to have run "nbgrader assign" before running the autograder). In general, nbgrader supports the use case where you don't distribute files with nbgrader---the main constraint is that you have to have the nbgrader course directory set up correctly: https://nbgrader.readthedocs.io/en/master/user_guide/philosophy.html I think what you're asking though is for one person to be able to create the assignment and release it to students, and then other people collect and grade assignments, without sharing the same nbgrader course directory---is that right? That's unfortunately not something that's supported by nbgrader at the moment, and I'm not sure how feasible it is. The thing that's mainly required to share between the person who creates the assignment and the people doing the grading is the database. But if you remove the database from the equation, then you lose the ability to do things like check whether a student has modified the tests in their notebook. The formgrader also strongly relies on having a database to do fast queries. You could conceivably run the autograder without relying on the assignments to be in the database ahead of time (thus allowing separate graders to just have different versions of the database) but then all sorts of consistency issues come up like---what if one student created a new cell with a new grade id, which isn't in any of the other students' submissions?

…

On Mon, Jun 10, 2019 at 7:54 AM Tony Hirst ***@***.***> wrote: Is it possible to install various components of nbgrader separately? For example, is there a way I can just install: - the create_assignment extension so that I could mark up a notebook for use in an assignment (but not do anything else with it); - the formgrader extension, so that I could point it to a directory containing student directories/scripts and either just manually, or automatically and manually, grade those scripts? Our workflow is such that different people create and mark assignments, with submission and return handled manually / using another process. Ideally, we want only want to install the components that individuals have to use, rather than the whole nbgrader environment. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1148?email_source=notifications&email_token=AAAUL5CCZCWEUQMQ2J52KTLPZZTJ5A5CNFSM4HWUH7R2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GYSXHCQ>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAUL5DTSN6I3VCRX4I3KOTPZZTJ5ANCNFSM4HWUH7RQ> .

[psychemedia]

@jhamrick Yes, we are running without a shared database (indeed without shared anything).

Person creates nbgrader notebook which acts as marking guide notebook -> another person creates assignment from that -> assignment posted on VLE -> students collect assignment -> students complete assignment via a personal notebook server running in an isolated VM on their own computer -> students submit assignment into a central (non-Jupyter) assignment handling system -> markers get assignments -> markers mark assignments via a personal notebook server running in an isolated VM on their own computer with comparison to marking guide notebook -> markers return assignments+marks through a central (non-Jupyter) assignment handling system -> students look at their marked notebook.

What we can control, to a certain extent, is what goes in to each VM. I can probably find a way through the nbgrader workflow to make this sort of work at an individual level, I was just wondering if I could split out components into rather more standalone, loosely coupled tools.

It is highly unlikely that we will get a central nbgrader server in the near future. What I want to see is if we can quietly introduce steps along the way, getting individuals to perform assignment setting, completion, and marking steps as standalone nbgrader activities.

At that point, I may be able to approach central services with an argument for a central nbgrader service along the lines of: we have all these people doing these separate tasks; if you provide a central service, we can quickly plug those tasks into it and things will be much easier.

One argument against giving setters / markers the full nbgrader environment and then using only parts of it is that getting folk to do new things (and getting permission to do new things) is HARD. (In my institution, we may have 30-50 markers on course, which is a lot of people to persuade at a single time...) Installing an extension into a notebook and marking up a notebook as an assignment is easier than presenting someone with the whole nbgrader environment. Installing a grading extension into notebook server that would let a marker open a student notebook (as a notebook) and grade it against a 'marking guide' notebook somewhere else on their machine is easier than getting them to look at the formgrader tab. After a couple of rounds of doing that, I might then be able to move to the step of: if you put all your student files into a directory, you can then use the nbgrader formgrader tab to mark them all.

Essentially, I need a stealth guerrilla warfare adoption path where I can really gently introduce things a step at a time on the user side so quietly folk barely even notice other than life suddenly seems to have got a bit easier, and slowly build towards an attack on the centre where we can say "we have all these pieces in place, now you just need to give us a central server..."

[jhamrick]

Yeah, some of those are unfortunately tricky to do with the way nbgrader currently works. But I'd like to think more about this so I am going to reopen the issue and will reply with some more thoughts later...

[psychemedia]

Okay, so I'll maybe chip in more ideas too...

eg for a standalone autograder extension, it might be handy if I could:

• open a student notebook to mark;
• specify a marking guide notebook;
• autograde;
• manually grade.

If I have to manually grade every notebook, then I'm going to have to open them one at a time anyway.

The next consideration is markers who like to mark a question at a time; so eg if I have a directory containing 20 student scripts, I may want to open a marking tool (perhaps as a Jupyter app) that would let me select a question, then click through and mark each student's answer to that question, before moving on to the next question and marking it again for each student in turn.

[kcranston]

Adding my +1 to this issue, particularly the request to run create_assignment independently. As someone relatively new to nbgrader, and trying to teach others, I would really love to be able to simply install nbgrader, create a notebook, and validate without needing any of the other pieces (student list, exchange dir, database, even nbgrader directory).

Thinking about the workflow of an instructor deciding if they want to use nbgrader, the first step would probably be to create some content (i.e. notebooks) and then create the infrastructure for assignment / student management.

[psychemedia]

@kcranston Yep, the adoption path is something I've been thinking about too... The design requirement that student's should not have to install anything to complete an assignment is eminently sensible, and I think there should be a similar path of baby steps for instructors too.

eg a minimal viable nbgrader set might be a set up path for someone who wants to set and manually grade (by themselves) a simple assignment that they can easily mark and return feedback on without needing a gradebook / class management tool etc).

I think we could probably work out a range of scenarios and a range of extensions that are required for them, (whether or not nbgrader actually supports the installation of such subsets of extensions). This might then provide a rationale for helping imaging nbgrader as a set of loosely coupled interoperable tools, and clearly identifying what those tools are, as well as how those tools might work together to provide powerful class management and automated test assignment handling, as well as act as a prop for an individual instructor who needs to manually interface to pretty much every other institutional system (or work in an informal setting where there are no other digital systems to interoperate with...).

[jhamrick]

I would really love to be able to simply install nbgrader, create a

notebook, and validate without needing any of the other pieces FWIW, as I mentioned above, you can do this already with nbgrader: you don't need to have the course directory set up just to create the notebooks. You can use the "Create Assignment" and "Validate" extensions on any notebook, regardless of whether they are in the nbgrader directory or not.

…

On Sat, Jun 29, 2019 at 1:44 AM Tony Hirst ***@***.***> wrote: @kcranston <https://github.com/kcranston> Yep, the adoption path is something I've been thinking about too... The design requirement that student's should not have to install anything to complete an assignment is eminently sensible, and I think there should be a similar path of baby steps for instructors too. eg a minimal viable nbgrader set might be a set up path for someone who wants to set and manually grade (by themselves) a simple assignment that they can easily mark and return feedback on without needing a gradebook / class management tool etc). I think we could probably work out a range of scenarios and a range of extensions that are required for them, (whether or not nbgrader actually supports the installation of such subsets of extensions). This might then provide a rationale for helping imaging nbgrader as a set of loosely coupled interoperable tools, and clearly identifying what those tools are, as well as how those tools might work together to provide powerful class management and automated test assignment handling, as well as act as a prop for an individual instructor who needs to manually interface to pretty much every other institutional system (or work in an informal setting where there are no other digital systems to interoperate with...). — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1148?email_source=notifications&email_token=AAAUL5DLXNRCYKL6O7YMBFLP42V6HA5CNFSM4HWUH7R2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY3OAOY#issuecomment-506912827>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAUL5GTXXQ26HW6JYHQRILP42V6HANCNFSM4HWUH7RQ> .

[psychemedia]

I note that Github now supports template repos so it should be possible to create various example template repos, capable of running on MyBinder, that folk could use to bootstrap different ways of setting up / using nbgrader?