WebGL Filters & Effects
Relevant source files
The following files were used as context for generating this wiki page:
The WebGL Filters & Effects system provides high-performance image processing capabilities within the Rocreate application. This system is primarily architected around a library called fx-canvas (based on glfx.js), which abstracts WebGL complexities into a manageable pipeline of textures, shaders, and filters. It allows for real-time adjustments of visual properties like brightness, contrast, and hue, as well as complex distortions and blurs.
The system integrates with the broader application by providing a WebGL-backed <canvas> element that can ingest image data from standard 2D canvases, apply a series of hardware-accelerated effects, and output the result back for further manipulation or display.
Sources: src/app/script/fx-canvas/fx-canvas.ts:25-30, src/app/script/fx-canvas/fx-canvas.ts:205-230
Architecture Overview
The filter system is built on three core pillars:
- Textures: Raw sources of image data created from
<img>,<canvas>, or<video>elements. - Shaders: GLSL programs representing the mathematical logic of an image effect.
- Canvas: A WebGL image buffer that stores and manages the results of filter operations.
Component Relationship
The following diagram illustrates how the core WebGL components interact during a filter application process:
Core Components
FxCanvas Manager
The fxCanvas function initializes a specialized WebGL canvas. It handles the management of global WebGL states, texture swapping (using a spareTexture for ping-pong rendering), and the lifecycle of shaders.
| Method | Description |
|---|---|
initialize(width, height) | Sets up primary, spare, and extra textures for processing. |
draw(texture, width, height) | Prepares the canvas with a source texture. |
update() | Flips the vertical coordinate system to match standard canvas orientation. |
getPixelArray() | Extracts raw Uint8Array data from the GPU buffer. |
Sources: src/app/script/fx-canvas/fx-canvas.ts:68-100, src/app/script/fx-canvas/fx-canvas.ts:143-155
FxShader Class
The FxShader class encapsulates WebGL program creation, vertex/fragment shader compilation, and uniform management. It automatically detects and applies the highest available float precision (highp, mediump, or lowp).
- Vertex Handling: Uses a default vertex shader that maps vertices and texture coordinates to a normalized device coordinate (NDC) quad.
- Uniform Support: Supports various data types including floats, 2D/3D/4D vectors, and 3x3/4x4 matrices.
Sources: src/app/script/fx-canvas/core/fx-shader.ts:55-80, src/app/script/fx-canvas/core/fx-shader.ts:100-140
FxTexture Class
FxTexture manages GPU memory for image data. It supports different formats (RGBA) and types (UNSIGNED_BYTE or FLOAT). The system attempts to use OES_texture_float for improved quality in specific filters like bokeh/blur.
Sources: src/app/script/fx-canvas/core/fx-texture.ts:41-65, src/app/script/fx-canvas/fx-canvas.ts:46-60
Filter Pipeline & GPU Compositing
The application uses a hybrid approach to rendering. While standard tools might use Canvas 2D, the GPUProjectViewport can offload layer compositing to the GPU using CanvasKit.
Compositing Flow
When GPU compositing is active, layers are uploaded as textures and blended using hardware acceleration:
Implemented Filters
The project defines a comprehensive library of filters, some using the fx-canvas WebGL pipeline and others using CanvasKit SKSL shaders.
Filter Categories
| Filter ID | Tech Base | Description |
|---|---|---|
brightnessContrast | WebGL (fx-canvas) | Adjusts brightness and contrast levels. |
hueSaturation | WebGL (fx-canvas) | Modifies color hue and intensity. |
chromaticAberration | CanvasKit (SKSL) | Shifts RGB channels to simulate lens distortion. |
triangleBlur | WebGL (fx-canvas) | Performs high-speed box-like blur. |
tiltShift | WebGL (fx-canvas) | Simulates shallow depth of field. |
perspective | WebGL (fx-canvas) | Applies a 4-point perspective warp. |
Sources: src/app/script/klecks/filters/filters.ts:25-100, src/app/script/klecks/filters/filter-chromatic-aberration.ts:23-35
Chromatic Aberration Implementation
The Chromatic Aberration filter specifically utilizes an SKSL shader to eval the image at offset coordinates for the red and blue channels.
// SKSL Logic
half4 color = image.eval(coord);
half4 red = image.eval(coord + displacement);
half4 blue = image.eval(coord - displacement);
return half4(red.r, color.g, blue.b, color.a);
Sources: src/app/script/klecks/filters/filter-chromatic-aberration.ts:28-34
Summary
The WebGL Filters & Effects system provides Rocreate with a robust, hardware-accelerated pipeline for image manipulation. By leveraging fx-canvas for traditional GLSL effects and CanvasKit for modern SKSL-based compositing, the application achieves high-performance rendering that scales with complex layer stacks and high-resolution canvases.
Sources: src/app/script/fx-canvas/fx-canvas.ts:20-25, src/app/script/klecks/ui/project-viewport/gpu-project-viewport.ts:115-125