PG problem rendering execution path

PG problem rendering execution path

/webwork2/lib/WeBWorK/PG/Local.pm

• The actual constructor for the PG object is called in WeBWorK::PG but it passes the construction onto WeBWorK::PG::Local. There is a WeBWorK::PG;;Remote hook for executing PG rendering on a remote machine, possibly via a round-robin procedure to speed up execution, but it has never been fully implemented.

• Create a WeBWorK::PG::Translator object. my $translator = WeBWorK::PG::Translator->new; This also creates a Safe compartment (WWSafe.pm) which will enclose the evaluation of the .pg problem. This prevents an instructor from issuing commands that might modify the disk or steal passwords, etc. etc. It is why "print" is disabled and "TEXT" which is a restricted form of print takes its place.

• Set the translator's directory hash (courseScripts, macros, templates, and temp directories) from the course environment.

• Using the module list from the course environment (pg->modules), perform a "use"-like operation to evaluate the PG modules at runtime.

• Use data from the user, set, and problem, as well as the course environment and translation options, to set the problem environment hash $envir. The default subroutine, WeBWorK::PG::defineProblemEnvir, is used. $translator->environment($envir)

• Call $translator->initialize().

• Load PG.pl which is the unpacking/accepting macro file on the PG side of the WeBWorK2/PG software firewall. PG.pl must be loaded without opcode masking, so they are loaded here.

• Set the opcode mask to limit available perl subroutines. $translator->set_mask()

• Load the problem source $translator->source_string( $source )

• Install a safety filter $translator->rf_safety_filter

  The safety filter is used to preprocess student input before evaluation. The default safety filter, &WeBWorK::PG::safetyFilter, is used. Since almost all student input is now handled by MathObjects this filter may be obsolete.

• Translate the problem source into the format given by the display mode. $translator->translate();

• Process student answers obtained from the form field inputs. $translator->process_answers($formFields);

• Load and record the problem state $translator->rh_problem_state({ recorded_score => $ Use values from the database to initialize the problem state, so that the grader will have a point of reference.

• Determine an entry order @answerOrder

  Use the ANSWER_ENTRY_ORDER flag to determine the order of answers in the problem. This is important for problems with dependancies among parts.

• Install a grader $grader = $translator->rf_avg_problem_grader

  Use the PROBLEM_GRADER_TO_USE flag, or a default from the course environment, to install a grader.

• Grade the problem using the selected grader. ($result, $state) = $translator->grade_problem

• Clean up, set mailer, etc.

pg/lib/WeBWorK/PG/Translator.pm

• Contains the translator methods used in Local.pm. The most important is the translate method.

WeBWorK::PG::Translator::translate()

• Set warning and die handlers. These may be different then the handlers for webwork2

• Preprocess text: The input text is subjected to two global replacements.

  These global substitutions are implemented via a call back -- $evalString = &{$self->{preprocess_code}}($evalString); and can be changed or augmented

The standard preprocessing is defined by default_preprocess_code()

First every incidence of

	BEGIN_TEXT
	problem text
	END_TEXT

is replaced by

   	TEXT( EV3( <<'END_TEXT' ) );
	problem text
	END_TEXT

The first construction is syntactic sugar for the second. The second is defined and explained PGbasicmacros.pl.

Similar changes are made for `BEGIN_SOLUTION/END_SOLUTION` and so forth.

Second every incidence of \ (backslash) is replaced by \ (double backslash).

Third each incidence of ~~ is replaced by a single backslash.

This is done to alleviate a basic
incompatibility between TeX and Perl. TeX uses backslashes constantly to denote
a command word (as opposed to text which is to be entered literally).  Perl
uses backslash to escape the following symbol.  This escape
mechanism takes place immediately when a Perl script is compiled and takes
place throughout the code and within every quoted string (both double and single
quoted strings) with the single exception of single quoted "here" documents.
That is backlashes which appear in

    TEXT(<<'EOF');
    ... text including \{   \} for example
    EOF

are the only ones not immediately evaluated.  This behavior makes it very difficult
to use TeX notation for defining mathematics within text.

The initial global
replacement, before compiling a PG problem, allows one to use backslashes within
text without doubling them. (The anomolous behavior inside single quoted "here"
documents is compensated for by the behavior of the evaluation macro EV3.) This
makes typing TeX easy, but introduces one difficulty in entering normal Perl code.

The second global replacement provides a work around for this -- use ~~ when you
would ordinarily use a backslash in Perl code.
In order to define a carriage return use ~~n rather than \n; in order to define
a reference to a variable you must use ~~@array rather than \@array. This is
annoying and a source of simple compiler errors, but must be lived with.

The problems are not evaluated in strict mode, so global variables can be used
without warnings.

• A post_processing call back is also available, but currently does nothing default_postprocess_code

• Evaluate the problem text my ($PG_PROBLEM_TEXT_REF, $PG_HEADER_TEXT_REF, $PG_POST_HEADER_TEXT_REF,$PG_ANSWER_HASH_REF, $PG_FLAGS_REF, $PGcore) =$safe_cmpt->reval(" $evalString");

• Collect the output from the valuation

• Process error messages

• Prepare the return values

  • Actually all of the return values are contained in the PG_core object $rh_pgcore but for legacy reasons we also separate them out. Returning the entire PG_core object may be over kill but at least in some instances it was convenient to have it available.

WeBWorK::PG::Translator::process_answers()

• This is a little hard to read because the method of matching answers to answer evaluators has changed.
  • The original method involved dead reconning. The answers, in order, were in control: Match the first answer to the first answer evaluator -- do some fancy adjustments to handle matrices and other answer evaluators
  • For the new method the answer evaluators are in control and they know the names (or the numbers) of the answer blanks they need to check.
  • There is still some code for the first method in this subroutine, and in fact for some time it's been checking to make sure that the second procedure works correctly. I think the legacy code from the first method can now be removed.
• Set up custom warning and error handlers for the answer evaluation phase. These error handlers take an errorTable emitted by the MathObject answer evaluators (This makes three sets of error handlers set up for webwork2 and pg. The exception handling for webwork2&pg should be considered for refactoring.)
• Loop through the union of the answer evaluator names and the answer blank names in order.
• The answer evaluator and its corresponding answer blanks form an answer group.
  • Find the response object for the answergroup ( this has the names of the relevant answer blanks and the student entries)
  • Make sure that there is an answer evaluator in the answergroup. (The instructor might have created an answer blank without a corresponding answer evaluator.) $new_rf_fun = $answergrp->ans_eval; This has type AnswerEvaluator
  • Extract (use filter_answer) the student answers from the response object. The result will be a reference to a list or a scalar value.
  • Apply the answer evaluator to the answer or list of answers from the response object.
  • The result is an answerHash object
  • $new_rh_ans_evaluation_result = $self->{safe} ->reval('$new_rf_fun->evaluate($new_temp_ans, ans_label => \''.$ans_name.'\')');
  • Collect errors if the response object didn't have enough answer blanks for the answer evaluator or if other errors occur.
• the answerHash objects are collected into the hash rh_evaluated_answers and returned.
• errors are reported (maybe) -- I think the results of the errorTable are grabbed by the exception handlers -- but I'm not clear on the details. Should there be a die command after the evaluation?

WeBWorK::PG::Translator::grade_problem()

• $obj->rh_problem_state(%problem_state); # sets the current problem state
• $obj->grade_problem(%form_options); and returns (rh_problem_result, rh_problem_state)
• This is just a framework for problem graders which can be defined by the instructor and chosen individually for each problem (although usually default graders are used). The action is "within the safe compartment apply the grader object which takes the hash of evaluated answers, the hash representing the current problem state and possible optional inputs and creates a hash representing the problem grade (reported back to the student) and a new problem state which is saved to the database".
• Look to the default graders for examples and to understand the input and output format requirements:
  • std_problem_grader -- must get all answer correct,
  • avg_problem_grader -- gives partial credit
    • problem_result outputs:
      • my %problem_result = ( score => 0, errors => '', type => 'avg_problem_grader', msg => '', );
      • score will become the total number of correct questions divided by the total number of questions.
    • problem_state outputs
      • recorded_score => become the maximum of the previous recorded score and the current recorded score
      • num_of_correct_ans (really number of correct attempts) increases by 1 if all the answers in this problem are correct
      • num_of_incorrect_ans (attempts) increases by 1 otherwise.
• More graders are in the macro file pg/macros/PGgraders.pl