Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 103 additions & 28 deletions lib/WeBWorK/PG/Translator.pm
Original file line number Diff line number Diff line change
Expand Up @@ -1020,13 +1020,19 @@ sub rf_std_problem_grader {
return \&std_problem_grader;
}

sub std_problem_grader {
my ($rh_evaluated_answers, $rh_problem_state, %form_options) = @_;
=head2 std_problem_grader

This is an all-or-nothing grader. A student must get all parts of the problem
right before receiving credit. You should make sure to use this grader on
multiple choice and true-false questions, otherwise students will be able to
deduce how many answers are correct by the grade reported by webwork.

install_problem_grader('std_problem_grader');

my %evaluated_answers = %{$rh_evaluated_answers};
=cut

# By default the old problem state is simply passed back out again.
my %problem_state = %$rh_problem_state;
sub std_problem_grader {
my ($answers, $problem_state, %form_options) = @_;

# Initial setup of the answer.
my %problem_result = (
Expand All @@ -1036,51 +1042,120 @@ sub std_problem_grader {
msg => '',
);

my $ansCount = keys %evaluated_answers;
my $ansCount = keys %$answers;
unless ($ansCount > 0) {
$problem_result{msg} = 'This problem did not ask any questions.';
return (\%problem_result, \%problem_state);
$problem_result{msg} = eval('main::maketext("This problem did not ask any questions.")');
return (\%problem_result, $problem_state);
}

$problem_result{msg} = 'In order to get credit for this problem all answers must be correct.'
$problem_result{msg} =
eval('main::maketext("In order to get credit for this problem all answers must be correct.")')
if $ansCount > 1;

return (\%problem_result, \%problem_state)
unless defined $form_options{answers_submitted} && $form_options{answers_submitted} == 1;
# Return unless answers have been submitted.
return (\%problem_result, $problem_state) unless $form_options{answers_submitted} == 1;

my $allAnswersCorrectQ = 1;
for my $ans_name (keys %evaluated_answers) {
if (ref $evaluated_answers{$ans_name} eq 'HASH' or ref $evaluated_answers{$ans_name} eq 'AnswerHash') {
$allAnswersCorrectQ = 0 unless $evaluated_answers{$ans_name}->{score} == 1;
for my $ans_name (keys %$answers) {
if (ref $answers->{$ans_name} eq 'HASH' || ref $answers->{$ans_name} eq 'AnswerHash') {
$allAnswersCorrectQ = 0 unless $answers->{$ans_name}{score} == 1;
} else {
warn "Error: Answer $ans_name is not a hash";
warn "$evaluated_answers{$ans_name}";
warn 'This probably means that the answer evaluator for this answer is not working correctly.';
$problem_result{error} = "Error: Answer $ans_name is not a hash: $evaluated_answers{$ans_name}";
die "Error: Answer |$ans_name| is not a hash reference\n"
. $answers->{$ans_name}
. "\nThis probably means that the answer evaluator for this answer is not working correctly.";
}
}

# Report the results.
$problem_result{score} = $allAnswersCorrectQ;
$problem_state{recorded_score} //= 0;

if ($allAnswersCorrectQ == 1 || $problem_state{recorded_score} == 1) {
$problem_state{recorded_score} = 1;
} else {
$problem_state{recorded_score} = 0;
}
++$problem_state->{num_of_correct_ans} if $allAnswersCorrectQ == 1;
++$problem_state->{num_of_incorrect_ans} if $allAnswersCorrectQ == 0;
$problem_state->{recorded_score} //= 0;

++$problem_state{num_of_correct_ans} if $allAnswersCorrectQ == 1;
++$problem_state{num_of_incorrect_ans} if $allAnswersCorrectQ == 0;
# Increase recorded score if the current score is greater.
$problem_state->{recorded_score} = $problem_result{score}
if $problem_result{score} > $problem_state->{recorded_score};

return (\%problem_result, \%problem_state);
return (\%problem_result, $problem_state);
}

sub rf_avg_problem_grader {
my $self = shift;
return \&avg_problem_grader;
}

=head2 avg_problem_grader

This grader gives a "weighted" average score to the problem and is the default
grader.

The grader can be selected by calling

install_problem_grader('avg_problem_grader');

However, since this is the default grader, that is not necessary to use this
grader.

Each answer is assigned a weight (the default is 1). The score is then the sum
of the product of the weights and scores for the correct answers divided by the
total of the weights for all answers. (To assign weights as percentages, use
integers that add up to 100. For example, use 40 and 60 for the weights for two
answers.) Assign weights to answers using the C<cmp> option C<< weight => n >>.
For example, in PGML create the answer rule with

[_]{$answer}{10}{ cmp_options => { weight => 40 } }

With the classic C<ANS> method call

ANS($answer->cmp(weight => 40);

This grader also allows for one "goal" answer that is answered correctly to
automatically give credit for one or more other "optional" answers. This way, if
there are several "optional" answers leading up to the "goal" answer, and the
student produces the "goal" answer by some other means and does not answer the
"optional" answers, the student can be given full credit for the problem anyway.
To use this feature use the C<credit> option of the C<cmp> method for the "goal"
answer. For example, C<< credit => $answer1Name >> or C<< credit => [
$answer1Name, $answer2Name, ... ] >>, where C<$answer1Name>, C<$answer2Name>,
etc., are the names of the "optional" answers that will be given credit if the
"goal" answer is correct. Note that the other answers must be assigned names
either by calling C<NAMED_ANS_RULE> and C<NAMED_ANS>, or by creating the answer
rule in PGML with C<[_]{$answer1}{15}{$answer1Name}>, for example. The answer
names should be generated by calling C<NEW_ANS_NAME> (for example,
C<$answer1Name = NEW_ANS_NAME()>) rather than being made up. Otherwise the
problem will fail to work in many situations (for example, in tests). For
example, to set this up in PGML use

BEGIN_PGML
Optional Answer 1: [_]{$answer1}{10}{$answer1Name = NEW_ANS_NAME()}

Optional Answer 2: [_]{$answer2}{10}{$answer2Name = NEW_ANS_NAME()}

Goal: [_]{$answer3}{10}{ cmp_options => { credit => [ $answer1Name, $answer2Name ] } }
END_PGML

Note that the C<credit> and C<weight> options can be used together. For example:

BEING_PGML
Optional Answer: [_]{$optional}{10}{$optionalName = NEW_ANS_NAME()}{{ weight => 20 }}

Goal: [_]{$goalAnswer}{10}{ cmp_options => { credit => $optionalName, weight => 80 } }
END_PGML

This way, if the "optional" answer is correct but the "goal" answer is not, the
problem score will be 20%, but if the "goal" answer is correct, the problem
score will be 100%.

One caveat to keep in mind is that credit is given to an "optional" answer ONLY
if the answer is left blank (or is actually correct). Credit will NOT be given
if an "optional" answer is incorrect, even if the "goal" answer IS correct.

When credit is given to an "optional" answer due to the "goal" answer being
correct, a message will be added to the "optional" answer stating, "This answer
was marked correct because the primary answer is correct."

=cut

sub avg_problem_grader {
my ($answers, $problem_state, %form_options) = @_;

Expand Down
73 changes: 15 additions & 58 deletions macros/PG.pl
Original file line number Diff line number Diff line change
Expand Up @@ -706,34 +706,12 @@ =head2 ENDDOCUMENT

=item *

C<PROBLEM_GRADER_TO_USE>: a reference to the chosen problem grader.
C<ENDDOCUMENT> chooses the problem grader as follows:

=over

=item *

If a problem grader has been chosen in the problem by calling
C<install_problem_grader(\&grader)>, it is used.

=item *

Otherwise, if the C<PROBLEM_GRADER_TO_USE> PG environment variable contains a
reference to a subroutine, it is used.

=item *

Otherwise, if the C<PROBLEM_GRADER_TO_USE> PG environment variable contains the
string C<std_problem_grader> or the string C<avg_problem_grader>,
C<&std_problem_grader> or C<&avg_problem_grader> are used. These graders are
defined in L<PGanswermacros.pl>.

=item *

Otherwise, the C<PROBLEM_GRADER_TO_USE> flag will contain an empty value and the
PG translator should select C<&std_problem_grader>.

=back
C<PROBLEM_GRADER_TO_USE>: Either the string C<avg_problem_grader>, the string
C<std_problem_grader>, or a reference to problem grader code. Note that the
C<PROBLEM_GRADER_TO_USE> PG environment variable setting is transferred to this
flag in C<ENDDOCUMENT> if this flag is not set prior to the C<ENDDOCUMENT> call.
L<WeBWorK::PG> handles conversion of the string values into the appropriate
L<WeBWorK::PG::Translator> methods as needed.

=back

Expand Down Expand Up @@ -1460,24 +1438,10 @@ sub ENDDOCUMENT {
});
}

# Install problem grader.
# WeBWorK::PG::Translator will install its default problem grader if none of the conditions below are true.
if (defined($PG->{flags}{PROBLEM_GRADER_TO_USE})) {
# Problem grader defined within problem. No further action needed.
} elsif (defined($rh_envir->{PROBLEM_GRADER_TO_USE})) {
if (ref($rh_envir->{PROBLEM_GRADER_TO_USE}) eq 'CODE') {
# User defined grader.
$PG->{flags}{PROBLEM_GRADER_TO_USE} = $rh_envir->{PROBLEM_GRADER_TO_USE};
} elsif ($rh_envir->{PROBLEM_GRADER_TO_USE} eq 'std_problem_grader') {
$PG->{flags}{PROBLEM_GRADER_TO_USE} = \&std_problem_grader if (defined(&std_problem_grader));
} elsif ($rh_envir->{PROBLEM_GRADER_TO_USE} eq 'avg_problem_grader') {
$PG->{flags}{PROBLEM_GRADER_TO_USE} = \&avg_problem_grader if (defined(&avg_problem_grader));
} else {
warn "Error: $PG->{flags}{PROBLEM_GRADER_TO_USE} is not a known problem grader.";
}
} elsif (defined(&std_problem_grader)) {
$PG->{flags}{PROBLEM_GRADER_TO_USE} = \&std_problem_grader;
}
# Transfer the environment problem grader setting to the flag. WeBWorK::PG will handle conversion of the
# avg_problem_grader or std_problem_grader string values into the appropriate WeBWorK::PG::Translator methods, leave
# this as a code reference if that is what it is, or die if it is anything else.
$PG->{flags}{PROBLEM_GRADER_TO_USE} //= $rh_envir->{PROBLEM_GRADER_TO_USE};

TEXT(MODES(%{ $rh_envir->{problemPostamble} }));

Expand Down Expand Up @@ -1569,23 +1533,16 @@ sub loadMacros {
# problems that do so actually work even though this method does nothing.
sub findAppletCodebase { return ''; }

## Problem Grader Subroutines
# Problem Grader Subroutines

#####################################
# This is a model for plug-in problem graders
#####################################
# ^function install_problem_grader
# ^uses PG_restricted_eval
# ^uses %PG_FLAGS{PROBLEM_GRADER_TO_USE}
sub install_problem_grader {
my $rf_problem_grader = shift;
my $rh_flags = $PG->{flags};
$rh_flags->{PROBLEM_GRADER_TO_USE} = $rf_problem_grader if not_null($rf_problem_grader);
$rh_flags->{PROBLEM_GRADER_TO_USE};
my $problem_grader = shift;
$PG->{flags}{PROBLEM_GRADER_TO_USE} = $problem_grader if not_null($problem_grader);
return $PG->{flags}{PROBLEM_GRADER_TO_USE};
}

sub current_problem_grader {
install_problem_grader(@_);
return install_problem_grader(@_);
}

# FIXME? The following functions were taken from the former
Expand Down
55 changes: 24 additions & 31 deletions macros/core/PGanswermacros.pl
Original file line number Diff line number Diff line change
Expand Up @@ -1417,7 +1417,7 @@ sub check_units {

=head3 C<std_problem_grader>

This is an all-or-nothing grader. A student must get all parts of the problem write
This is an all-or-nothing grader. A student must get all parts of the problem right
before receiving credit. You should make sure to use this grader on multiple choice
and true-false questions, otherwise students will be able to deduce how many
answers are correct by the grade reported by webwork.
Expand All @@ -1427,62 +1427,55 @@ =head3 C<std_problem_grader>
=cut

sub std_problem_grader {
my ($rh_evaluated_answers, $rh_problem_state, %form_options) = @_;

my %evaluated_answers = %{$rh_evaluated_answers};

# By default the old problem state is simply passed back out again.
my %problem_state = %$rh_problem_state;
my ($answers, $problem_state, %form_options) = @_;

# Initial setup of the answer
# Initial setup of the answer.
my %problem_result = (
score => 0,
errors => '',
type => 'std_problem_grader',
msg => '',
);

my $ansCount = keys %evaluated_answers;

my $ansCount = keys %$answers;
unless ($ansCount > 0) {
$problem_result{msg} = 'This problem did not ask any questions.';
return (\%problem_result, \%problem_state);
$problem_result{msg} = maketext('This problem did not ask any questions.');
return (\%problem_result, $problem_state);
}
$problem_result{msg} = maketext('In order to get credit for this problem all answers must be correct.')
if $ansCount > 1;

$problem_result{msg} = 'In order to get credit for this problem all answers must be correct.' if ($ansCount > 1);

return (\%problem_result, \%problem_state) unless $form_options{answers_submitted} == 1;
# Return unless answers have been submitted.
return (\%problem_result, $problem_state) unless $form_options{answers_submitted} == 1;

my $allAnswersCorrectQ = 1;
for my $ans_name (keys %evaluated_answers) {
if (ref $evaluated_answers{$ans_name} eq 'HASH' || ref $evaluated_answers{$ans_name} eq 'AnswerHash') {
$allAnswersCorrectQ = 0 unless $evaluated_answers{$ans_name}{score} == 1;
for my $ans_name (keys %$answers) {
if (ref $answers->{$ans_name} eq 'HASH' || ref $answers->{$ans_name} eq 'AnswerHash') {
$allAnswersCorrectQ = 0 unless $answers->{$ans_name}{score} == 1;
} else {
die 'Error at file ', __FILE__, 'line ', __LINE__,
": Answer |$ans_name| is not a hash reference\n"
. $evaluated_answers{$ans_name}
. 'This probably means that the answer evaluator for this answer is not working correctly.';
$problem_result{error} = "Error: Answer $ans_name is not a hash: $evaluated_answers{$ans_name}";
die "Error: Answer |$ans_name| is not a hash reference\n"
. $answers->{$ans_name}
. "\nThis probably means that the answer evaluator for this answer is not working correctly.";
}
}

# Report the results
# Report the results.
$problem_result{score} = $allAnswersCorrectQ;

++$problem_state{num_of_correct_ans} if $allAnswersCorrectQ == 1;
++$problem_state{num_of_incorrect_ans} if $allAnswersCorrectQ == 0;
$problem_state{recorded_score} //= 0;
++$problem_state->{num_of_correct_ans} if $allAnswersCorrectQ == 1;
++$problem_state->{num_of_incorrect_ans} if $allAnswersCorrectQ == 0;
$problem_state->{recorded_score} //= 0;

# Increase recorded score if the current score is greater.
$problem_state{recorded_score} = $problem_result{score}
if $problem_result{score} > $problem_state{recorded_score};
$problem_state->{recorded_score} = $problem_result{score}
if $problem_result{score} > $problem_state->{recorded_score};

(\%problem_result, \%problem_state);
return (\%problem_result, $problem_state);
}

=head3 C<std_problem_grader2>

This is an all-or-nothing grader. A student must get all parts of the problem write
This is an all-or-nothing grader. A student must get all parts of the problem right
before receiving credit. You should make sure to use this grader on multiple choice
and true-false questions, otherwise students will be able to deduce how many
answers are correct by the grade reported by webwork.
Expand Down
2 changes: 1 addition & 1 deletion tutorial/sample-problems/Misc/ManyMultipleChoice.pg
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ $ENV{grader_message} =
. '60% partial credit for 4 - 5 correct answers.';

# All or nothing grader
# install_problem_grader(~~&std_problem_grader);
#install_problem_grader('std_problem_grader');

# Questions and answers
@s_and_a = (
Expand Down
2 changes: 1 addition & 1 deletion tutorial/sample-problems/Misc/Matching.pg
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ $ENV{grader_message} =
. '60% partial credit for 4 - 5 correct answers.';

# All or nothing grader
# install_problem_grader(~~&std_problem_grader);
#install_problem_grader('std_problem_grader');

# Create a matching list and use pop ups
$ml = new_match_list();
Expand Down
2 changes: 1 addition & 1 deletion tutorial/sample-problems/Misc/MatchingAlt.pg
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ $ENV{grader_message} =
. '60% partial credit for 4 - 5 correct answers.';

# All or nothing grader
# install_problem_grader(~~&std_problem_grader);
#install_problem_grader('std_problem_grader');

# Create a list of questions and answers.
@q_and_a = (
Expand Down