Problem: Students often submit the wrong assignment by mistake.

Solution: Give student immediate feedback on their assignment by automatically checking it against the requirements set by the professor.

---

Assignment Precheck API

This is an experimental API project designed to automatically pre-validate student assignments against professor-defined requirements before submission. It leverages AI/NLP for complex checks like content coverage and style.

---

Requirements & Notes

• Python Version: Python 3.11 is recommended for compatibility.

• Dependencies: Install all required libraries listed in pyproject.toml. All of them are necessary for the project to run correctly.

• Additional Package: The SpaCy English model must be installed separately. Run the following command:

  pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl

• OpenAI Key: An OpenAI key is required. After creating one, create a file called .env in the root directory, adding the following content:

  OPENAI_API_KEY=paste_your_key_here
  

• Running the server: You can run the project with run.py. Note! It may take a while for the server to initialize on the first start, so please be patient.

• I had problems running this project on macOS.

• Current Limitations: The service currently supports only simple cases. For example, only Example 1 is fully implemented.

---

How it Works

A request should be sent to this API whenever a student submits an assignment. The request should include the assignment details and the required checks defined by the teacher.

When a request is submitted, the request gets added into a queue to get handled by worker. This logic is handled using dramatiq package.

Here is an example of possible implementation of this system:

Assignment Details (Input)

At least one of the following data points is required to identify or access the assignment content:

Parameter	Type	Description
assignmentDescription	string	The full description of the assignment, visible to the student (e.g., "Write a 1500-word report...").
assignmentUrl	string (array)	URLs or paths to the submitted assignment file(s) (e.g., PDF, DOCX, ZIP).

Assignment Requirements (Configuration)

These properties define the specific checks the assignment must pass, set by the professor.

Parameter	Type	Description
requirementSeverity	enum	How critical the check is: "critical" (must fix), "minor" (recommended fix, points deduction likely), or "suggestion" (optional advice).
requirementDescription	string	A clear, human-readable description of the rule (e.g., "File must be PDF").

Submission Details (Input)

At least one of the following data points is required to identify or access the assignment content:

Parameter	Type	Description
submissionText	string	The full text of student's submission to be evaluated.

Precheck Result (Output)

The API returns a list of results, one for each requirement checked. This output is typically displayed directly to the student or instructor (e.g., Metropolia's Oma).

Parameter	Type	Description
requirementSeverity	enum	The severity level of the requirement checked ("critical", "minor", or "suggestion").
requirementDescription	string	The original rule description.
passed	boolean	true if the check was successful, false otherwise.
feedback	string	A detailed, descriptive result explaining why the check passed or failed, often including specific locations or counts.

---

Examples

To keep this document concise, the detailed JSON input and output for various checks are provided in separate files. Click the links below to view each scenario:

• (✅ Implemented) Example 1: Content and Keyword Coverage Check

  • Focus: Using NLP to validate required academic topics and tone.
  • View Example 1 Details

• (❌ Not implemented, just an example of what could be possible) Example 2: Critical File Format & Content Check

  • Focus: Ensuring basic submission requirements (file type, minimum embedded objects).
  • View Example 2 Details

• (❌ Not implemented, just an example of what could be possible) Example 3: Code Quality and Efficiency Check

  • Focus: Analyzing programming assignments for complexity and style rules.
  • View Example 3 Details