Solution to AST Conversion

Relevant source files

The following files were used as context for generating this wiki page:

• backend/src/ast/ast-conversion.service.ts
• backend/src/ast/converters/solution-conversion-worker.piscina.ts
• backend/src/ast/converters/scratch/tests/scratch-ast-converter.service.spec.ts
• frontend/src/api/collimator/hooks/solutions/useSolution.ts
• frontend/src/pages/class/[classId]/session/[sessionId]/task/[taskId]/solve.tsx
• frontend/src/components/task/TaskForm.tsx
• frontend/src/components/dashboard/CodeView.tsx
• frontend/src/pages/task/[taskId]/edit.tsx
• frontend/src/pages/task/[taskId]/detail.tsx
• frontend/package.json

The Solution to Abstract Syntax Tree (AST) conversion system is a core backend feature responsible for transforming student-submitted solutions from their original format (e.g., Scratch project JSON) into a standardized, language-agnostic General AST. This unified representation allows for consistent analysis, comparison, and processing of solutions, regardless of the source programming environment.

The process is initiated when a solution is submitted. The backend's AstConversionService delegates the heavy lifting of parsing and conversion to a dedicated worker thread to ensure the main application remains responsive. The resulting General AST is a structured tree of nodes representing the program's logic, including actors, event listeners, control flow, and expressions. This AST serves as the input for further analysis modules within the Collimator platform.

Backend Architecture

The backend conversion process is managed by the AstConversionService and executed by a worker pool. This design offloads CPU-intensive parsing from the main event loop.

Sources: backend/src/ast/ast-conversion.service.ts, backend/src/ast/converters/solution-conversion-worker.piscina.ts

Components

Component	Description	Source File
AstConversionService	A NestJS injectable service that orchestrates the conversion. It receives a task and a solution, determines the correct conversion path, and invokes the worker.	backend/src/ast/ast-conversion.service.ts:13-50
Piscina Worker Pool	A worker thread pool used by AstConversionService to run the conversion logic in a separate process. This prevents blocking the main Node.js event loop.	backend/src/ast/ast-conversion.service.ts:14-25
SolutionConversionWorker	The script executed by the Piscina worker. It receives the solution data, decodes it, and calls the appropriate language-specific converter (e.g., for Scratch).	backend/src/ast/converters/solution-conversion-worker.piscina.ts:5-25
convertScratchToGeneralAst	A specific converter function that transforms the JSON structure of a Scratch project into the General AST format.	backend/src/ast/converters/scratch/tests/scratch-ast-converter.service.spec.ts:8 (inferred from test)

Data Flow

The following diagram illustrates the sequence of operations for converting a solution to a General AST on the backend.

AstConversionService

This service exposes a single public method, convertSolutionToAst.

// backend/src/ast/ast-conversion.service.ts

// ...
  async convertSolutionToAst(
    task: TaskWithoutData,
    solution: Solution,
  ): Promise<GeneralAst> {
    let ast: Promise<GeneralAst>;

    if (
      task.type === TaskType.SCRATCH &&
      solution.mimeType === "application/json"
    ) {
      ast = this.conversionWorker.run({
        solution,
        taskType: task.type,
      });
    } else {
      throw new Error(
        `Unsupported (task, solution mime type) tuple '(${task.type}, ${solution.mimeType})'`,
      );
    }

    return ast;
  }
// ...

This method currently only supports Scratch projects (TaskType.SCRATCH) submitted as JSON (application/json). It uses the conversionWorker to execute the task asynchronously.

Sources: backend/src/ast/ast-conversion.service.ts:38-50

Scratch to General AST Conversion

The conversion from a Scratch project's internal JSON representation to the General AST is the primary implementation of this system. The process translates Scratch-specific concepts like sprites, hat blocks, and block stacks into a standardized, hierarchical structure.

Sources: backend/src/ast/converters/scratch/tests/scratch-ast-converter.service.spec.ts

General AST Structure

The GeneralAst is an array of ActorNode objects. Each ActorNode represents a sprite or the stage and contains its scripts.

The diagram below shows the main interfaces that constitute the General AST.

Block Type Mapping

Scratch blocks are mapped to different nodes in the General AST based on their opcode and structure.

Scratch Block Category	Example Opcodes	General AST Node Type -
Hat Blocks	event_whenflagclicked, event_whenkeypressed	EventListenerNode -
Control Blocks	control_repeat, control_forever, control_if	LoopNode, ConditionNode -
Stack/Code Blocks	motion_movesteps, looks_say, data_setvariableto	StatementNode (with statementType: functionCall) -
Reporter/Expression Blocks	operator_add, sensing_mousex, data_itemoflist	ExpressionNode (with expressionType: operator, variable, or functionCall) -
Custom Blocks	procedures_definition, procedures_call	FunctionDeclarationNode, FunctionCallNode -

Sources: backend/src/ast/converters/scratch/tests/scratch-ast-converter.service.spec.ts

Frontend Integration

The frontend interacts with the solution data and its AST representation (indirectly) through various hooks and components. It is responsible for fetching solution files, submitting new solutions, and displaying code.

Key Frontend Components

Component/Hook	Description -
useSolutionFile	SWR hook to fetch a solution file (Blob) by its hash. Used for displaying existing solutions. -
useFetchLatestSolutionFile	Hook that returns a function to fetch the latest solution file for a student and task. Used to load a student's prior work. -
solve.tsx Page	The primary UI for a student to solve a task. It loads the task, fetches the latest solution, and provides functionality to submit the new solution. -
TaskForm.tsx Component	A form used for creating and editing tasks. It handles the task file, initial solution, and reference solutions, which are all stored as Blob objects in the form state. -
CodeView.tsx Component	Displays a read-only view of a solution file. It uses useSolutionFile and useTaskFile to load the necessary data into an embedded application (e.g., Scratch player). -

Sources: frontend/src/api/collimator/hooks/solutions/useSolution.ts, frontend/src/pages/class/[classId]/session/[sessionId]/task/[taskId]/solve.tsx, frontend/src/components/task/TaskForm.tsx, frontend/src/components/dashboard/CodeView.tsx

Solution Submission Flow

When a student works on a task, the frontend orchestrates loading their previous work and submitting their new solution.

This diagram shows the user interaction on the "Solve Task" page.

Conclusion

The Solution to AST Conversion system is a critical piece of infrastructure in the Collimator project. By abstracting different programming environments into a General AST, it enables a powerful and scalable analysis pipeline. The backend uses a robust, non-blocking architecture with worker threads to handle the computationally expensive conversion process. The frontend provides a seamless user experience for students and instructors to submit, manage, and view solutions, fully integrating with the backend services that power the AST conversion.