AI-Powered Code Evaluation

Relevant source files

The following files were used as context for generating this wiki page:

• Services/OpenAIService.cs
• Services/ProgrammingExamService.cs
• Models/ProgrammingExamModels.cs
• Migrations/20251225105704_AddAIEvaluationFields.Designer.cs
• PeP.ExamApp/Installer/README.md
• Controllers/ExamAppController.cs

The AI-Powered Code Evaluation feature automates the grading of student code submissions for programming exams. It leverages an external AI service (OpenAI) to analyze the code against task requirements, providing a detailed breakdown of the score, qualitative feedback, and specific code-level comments. This system is designed to give students immediate, constructive feedback while reducing the manual grading workload for instructors.

The process is initiated after a student completes a programming exam. The ProgrammingExamService orchestrates the evaluation by calling the OpenAIService, which formats the student's code and task instructions into a structured prompt for the AI. The AI's JSON response is then parsed and the results are persisted in the database, linked to the student's exam attempt.

System Architecture

The AI evaluation system is primarily composed of two services: ProgrammingExamService for orchestration and OpenAIService for communication with the AI model. The results are stored in the ProgrammingExamAttempt and TaskProgress tables.

Core Components

Component	Description	Source File
ProgrammingExamService	Manages the overall evaluation workflow. Its EvaluateAttemptWithAIAsync method fetches the student's attempt, calls the AI service, and saves the results.	Services/ProgrammingExamService.cs
OpenAIService	Handles all interactions with the OpenAI API. It builds the detailed prompt, sends the request, and parses the JSON response into a structured AICodeEvaluation object.	Services/OpenAIService.cs
ProgrammingExamAttempt	The database model representing a student's attempt. It contains fields to track the status of the AI evaluation, such as AIEvaluationCompleted and AIEvaluationCompletedAt.	Models/ProgrammingExamModels.cs
TaskProgress	A database model that stores the evaluation results for each individual task within an exam attempt, including the AI-generated score, feedback, and various metric scores.	Models/ProgrammingExamModels.cs

Sources: Services/ProgrammingExamService.cs:100, Services/OpenAIService.cs:17, Models/ProgrammingExamModels.cs:278,411

Evaluation Flow

The following sequence diagram illustrates the process of evaluating a student's exam attempt with AI.

This diagram shows the sequence from an instructor triggering the evaluation to the final results being stored in the database.
Sources: Services/ProgrammingExamService.cs:910-980, Services/OpenAIService.cs:703-722

AI Service Implementation

The OpenAIService is responsible for the direct communication with the AI model. It uses a carefully constructed prompt to ensure a consistent and structured JSON output.

Key Methods

• EvaluateCodeAsync(...): Evaluates a single task's code. It constructs a detailed system prompt, combines it with the task details and student code, and sends it to the OpenAI API.
• EvaluateAllTasksAsync(...): Iterates through all tasks in a ProgrammingExamAttempt and calls EvaluateCodeAsync for each one, collecting the results.

Sources: Services/OpenAIService.cs:567-575,703-706

AI Prompt Engineering

The quality of the AI evaluation heavily depends on the system prompt sent to the model. The prompt defines the AI's role, grading principles, scoring guidelines, and the exact JSON structure for the response.

Key instructions in the prompt include:

• Role Definition: "You are a fair but thorough {languageName} code reviewer and programming instructor."
• Grading Principles: Strict rules are set, such as giving 0 points for empty or placeholder code, while encouraging partial credit for genuine attempts.
• Scoring Guidelines: A percentage-based rubric is provided to guide the AI in assigning scores.
• Evaluation Criteria: The AI must assess the code based on Correctness (40%), Code Quality (25%), Efficiency (15%), and Completion (20%).
• JSON Schema: A rigid JSON structure is defined for the output, which includes scores, feedback text, and code snippets.

Sources: Services/OpenAIService.cs:607-662

Response Data Structure

The OpenAIService expects a JSON response from the AI, which is deserialized into an AIEvaluationResponse class. This is then mapped to the public AICodeEvaluation class.

The AICodeEvaluation class diagram shows the structure of the evaluation result for a single task.

Database Schema and Data Persistence

The results from the AI evaluation are stored in the TaskProgress and ProgrammingExamAttempt tables. This allows the application to display detailed feedback and scores to both students and instructors.

Entity Relationship Diagram

The diagram below shows the relevant entities and their relationships for storing AI evaluation data.

AI Evaluation Fields

The following tables detail the specific fields added to the database to support AI evaluation.

ProgrammingExamAttempt Table

Field Name	Data Type	Description
AIEvaluationCompleted	bool	Flag indicating if the AI evaluation has finished.
AIEvaluationStartedAt	datetime?	Timestamp when the evaluation process began.
AIEvaluationCompletedAt	datetime?	Timestamp when the evaluation process finished.
RequestTeacherReevaluation	bool	Flag set by a student to request a manual review.
TeacherReevaluationCompleted	bool	Flag set by a teacher after completing a manual review.

Sources: Models/ProgrammingExamModels.cs:298-316

TaskProgress Table

Field Name	Data Type	Description
AIFeedback	string?	Detailed textual feedback from the AI.
AIScore	decimal(18,2)?	The score assigned by the AI for the task.
AIEvaluatedAt	datetime?	Timestamp when the specific task was evaluated.
CodeQualityScore	decimal(18,2)?	Score (0-100) for code quality.
CorrectnessScore	decimal(18,2)?	Score (0-100) for correctness.
EfficiencyScore	decimal(18,2)?	Score (0-100) for efficiency.
CompletionPercentage	decimal(18,2)?	The AI's estimate of task completion (0-100).
AICodeSnippets	string?	JSON string of CodeSnippetFeedback objects.
AISolutionSuggestion	string?	A brief suggestion from the AI on a better approach.

Sources: Models/ProgrammingExamModels.cs:438-455

Configuration

The AI evaluation feature requires configuration settings, which are stored as PlatformSetting records in the database. The OpenAIService retrieves these at runtime.

Key	Description	Default Value	Source
OpenAIApiKey	The API key for the OpenAI service.	(empty)	Services/OpenAIService.cs:122
OpenAIModel	The specific AI model to use for evaluation (e.g., gpt-4o-mini).	gpt-4o-mini	Services/OpenAIService.cs:129

While appsettings.json can be used for initial setup, these values are managed in the PlatformSettings table in a production environment.
Sources: PeP.ExamApp/Installer/README.md:83-93, Services/OpenAIService.cs:122-130