Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/jplag implementation #447

Open
wants to merge 15 commits into
base: development
Choose a base branch
from
Open

Feature/jplag implementation #447

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
dd70500
chore: Add files that werent committed first time
JackSCarroll Aug 18, 2024
469b1ac
refactor: jplag report generation works but only in student-work/new
JackSCarroll Aug 20, 2024
28c50d9
rework: add code file extraction from done files zips and generate jp…
JackSCarroll Aug 21, 2024
611bbac
refactor: add jplag function to unit_similarity_module and remove tem…
JackSCarroll Aug 30, 2024
6fa26f0
fix: make report file generation make more sense and remove tmp files
JackSCarroll Sep 19, 2024
c83919c
fix: make report downloading use new path
JackSCarroll Sep 20, 2024
4d0abe3
chore: clean files
JackSCarroll Sep 20, 2024
eb8c0e8
chore: clean files
JackSCarroll Sep 20, 2024
bf1f7d6
chore: remove hard coded path and update file_lang
JackSCarroll Sep 21, 2024
1caabe4
refactor: rename moss language to jplag language
JackSCarroll Sep 25, 2024
b95fa31
refactor: adjust similarity match code to be suited for jplag
JackSCarroll Oct 2, 2024
2a23759
refactor: add plagiarism link from jplag reports
JackSCarroll Oct 3, 2024
f9b0ae0
refactor: add jplag plagiarism link support for group tasks
JackSCarroll Oct 11, 2024
dab43cb
refactor: add hasJjplagReport value to task defintion and use for rep…
JackSCarroll Oct 12, 2024
a45eb6b
fix: fix report download
JackSCarroll Oct 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: make report file generation make more sense and remove tmp files
  • Loading branch information
@JackSCarroll
JackSCarroll committed Sep 21, 2024
commit 6fa26f07ab5528b9610d185da90ed07f048e063b
28 changes: 26 additions & 2 deletions app/models/similarity/unit_similarity_module.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def check_similarity(force: false)
tasks = tasks_for_definition(td)
tasks_with_files = tasks.select(&:has_pdf)

#JPLAG
# JPLAG
run_jplag_on_done_files(td, tasks_dir, tasks_with_files, unit_code)

JackSCarroll marked this conversation as resolved.
Show resolved Hide resolved
# Skip if not due yet
Expand Down Expand Up @@ -252,9 +252,33 @@ def run_jplag_on_done_files(task_definition, tasks_dir, tasks_with_files, unit_c
end
puts "Starting JPLAG container to run on #{tasks_dir}"
JackSCarroll marked this conversation as resolved.
Show resolved Hide resolved
tasks_dir_split = tasks_dir.to_s.split('/workspace/doubtfire-api')[1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardcoded dev container path... won't work in production

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have changed to Rails.root

`sudo docker exec jplag java -jar /jplag/jplag-5.1.0-jar-with-dependencies.jar #{tasks_dir_split} -l #{type_data[1]} --similarity-threshold=0.8 -r /jplag/results/#{unit_code}_#{task_definition.id}-result`

# Check if the directory exists and create it if it doesn't
results_dir = "/jplag/results/#{unit_code}"
`sudo docker exec jplag sh -c 'if [ ! -d "#{results_dir}" ]; then mkdir -p "#{results_dir}"; fi'`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sudo?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be outside the upload requirements loop? What happens if you have multiple files being checked in the same task definition?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sudo is needed to make the docker exec commands work. Not entirely sure why 🙃.
Have moved the dir checks outside the loop and still works as expected.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker exec doesn't require sudo in production since it'll run as the root user, but the permission errors in the devcontainer are due to it using the vscode user

You can fix this by setting permissions on /var/run/docker.sock in https://github.com/doubtfire-lms/doubtfire-deploy


# Remove existing result file if it exists
result_file = "#{results_dir}/#{task_definition.id}-result.zip"
`sudo docker exec jplag sh -c 'if [ -f "#{result_file}" ]; then rm "#{result_file}"; fi'`

case type_data[1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this all the languages available?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some of the language names in MOSS and JPLAG differ, i've added a check for c++ and python.
Currently only supporting languages defined from the task definition ui task-definition-upload.component.html
Have added comments to both task-definition-upload.component.html and unit-similarity-module.rb as reminder to check MOSS and JPLAG language defintions

when 'csharp'
file_lang = 'csharp'
when 'cc'
file_lang = 'cpp'
end

# Run JPLAG
puts "THE FILE TYPE IN MOSS: #{type_data[1]}"
JackSCarroll marked this conversation as resolved.
Show resolved Hide resolved
`sudo docker exec jplag java -jar /jplag/myJplag.jar #{tasks_dir_split} -l #{file_lang} --similarity-threshold=0.8 -nobr -M RUN -r #{results_dir}/#{task_definition.id}-result`
end

# Delete the extracted code files from tmp
tmp_dir = Rails.root.join("tmp", "jplag")
puts "Deleting files in: #{tmp_dir}"
JackSCarroll marked this conversation as resolved.
Show resolved Hide resolved
puts "Files to delete: #{Dir.glob("#{tmp_dir}/*")}"
JackSCarroll marked this conversation as resolved.
Show resolved Hide resolved
FileUtils.rm_rf(Dir.glob("#{tmp_dir}/*"))

self
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are the similarity matches created?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to parse the report and link related tasks

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a function to create the similarity matches. Here's the commit.
It opens the overview.json file from the report and extracts the max similarity between 2 submissions.
It also finds the task id by looking in /files/student_name/task_id and grabbing the task id folder name; it matches the student names from the two that are being looked at currently in overview.json.

end
end