Skip to content

Shaders Module

src/shaders/ contains every WGSL program that powers Visionary’s pipeline. The TypeScript layer simply imports the raw shader strings (preprocess.wgsl, adix_sort.wgsl, gaussian.wgsl, plus utility kernels). This README explains what each shader does, how they interact, and where overrides/live constants come from.

Files at a glance

File Stage Description
preprocess.wgsl Compute Projects 3D Gaussians into screen-space splats, evaluates SH/RAW color, writes depth keys + payloads. Contains overrides for raw color (USE_RAW_COLOR), SH layout (SH_LAYOUT_CHANNEL_MAJOR), and compressed/int8/f16 data.
gaussian.wgsl Render (VS/FS) Consumes sorted 2D splats, builds a quad per instance, evaluates Gaussian falloff, outputs premultiplied color.
adix_sort.wgsl Compute 4-pass GPU radix sort (histogram + prefix + scatter) with ping-pong buffers and indirect dispatch support. Constants such as workgroup sizes are injected from TypeScript.
compress_gaussians.wgsl Compute Optional kernel used by tooling to convert FP32 splats into quantized formats (int8/f16). Shares struct layout with preprocess.
convert_precision.wgsl Compute Runtime precision swap (FP32 �?FP16/INT8) used by the ONNX conversion path.
debug-helpers.wgsl Compute Small utilities for GPU-side debugging (e.g., copying counters, dumping buffers).
index.ts TS glue Re-exports raw shader strings via Vite’s ?raw loader.

Pipeline overview

3D Gaussians �? ├─ preprocess.wgsl (camera �?splats + depth + sort counters) �? ├─ radix_sort.wgsl (depth sort + indirect dispatch) �? ├─ gaussian.wgsl (render sorted splats �?color buffer) �? └─ display / compositor (handled in renderer; env map shader lives outside core)

  • Preprocess reads packed Gaussians (FP32/FP16/INT8) via uModel.gaussDataType, applies camera matrices, transforms covariances to screen-space ellipses, evaluates spherical harmonics or raw RGB (USE_RAW_COLOR = true), and writes to a global splat buffer plus sorter key/payload buffers. It also increments atomic counters for indirect sort/draw.
  • Radix sort reads the depth keys/payloads, runs histogram/prefix/scatter passes (with ping-pong key/payload buffers), and leaves the sorted payload list that gaussian.wgsl uses as an index buffer. Workgroup sizes / radix config are injected strings (see renderer setup).
  • Gaussian is a minimalist instanced render pipeline: vertex shader fetches eigenvectors + center + color, generates four vertices per instance, and fragment shader clips samples where dot(screen_pos, screen_pos) > 2*CUTOFF before computing exp(-r²) and returning premultiplied RGBA.

Overrides & specialization constants

  • MAX_SH_DEG �?injected by the renderer before shader compilation to limit SH evaluation cost.
  • USE_RAW_COLOR �?set to rue for color-mode models (e.g., DynamicPointCloud with RGB). When rue, preprocess skips SH evaluation and directly interprets the color_buffer payload.
  • SH_LAYOUT_CHANNEL_MAJOR �?toggles between legacy “interleaved�?SH storage (RGB triplets per coefficient) and the newer channel-major layout (Rdc,Gdc,Bdc,R�?.R�?G�?.G�?B�?.Bₘ`). This matches the loader/ONNX paths.
  • DISCARD_BY_WORLD_TRACE/MAX_WORLD_TRACE �?optional guard that drops splats whose world-space covariance trace exceeds a threshold (useful for debugging bad assets).
  • Radix sort constants (histogram_wg_size, histogram_sg_size, etc.) are produced by GPURSSorter.processShaderTemplate so they match the WebGPU device.

Data contracts

Preprocess output (Splat)

wgsl struct Splat { v_0: u32; v_1: u32; // packed eigenvectors (4×f16) pos: u32; // packed NDC x/y (2×f16) posz: f32; // un-packed NDC z (f32 precision) color_0: u32; color_1: u32; // packed RGBA (4×f16) } Each splat occupies 32 bytes (matches BUFFER_CONFIG.SPLAT_STRIDE). gaussian.wgsl reads these structs verbatim.

Sort metadata

wgsl struct SortInfos { keys_size : atomic<u32>; // how many splats were written this frame padded_size : u32; // capacity (multiple of workgroup block) passes : u32; even_pass : u32; odd_pass : u32; } struct DrawIndirect { vertex_count : u32; // always 4 instance_count : atomic<u32>; base_vertex : u32; base_instance : u32; } struct DispatchIndirect { dispatch_x : atomic<u32>; dispatch_y : u32; dispatch_z : u32; } Preprocess writes to keys_size/instance_count/dispatch_x, radix sort consumes them via indirect dispatch, and the renderer copies keys_size into the draw indirect buffer before issuing drawIndirect.

  • Renderer (src/renderer/gaussian_renderer.ts) injects MAX_SH_DEG, chooses raw vs SH color mode, and binds the shader layouts.
  • Preprocessor TS (GaussianPreprocessor) replaces storage buffers at runtime (e.g., after precision conversion) and forwards uModel values for ModelParams uniform.
  • Sorter TS (GPURSSorter) injects radix constants and orchestrates the histogram/prefix/scatter passes.

For deeper binding diagrams, layout details, and per-entry-point documentation, see architecture and API reference.

  • Architecture – Shader entry-point diagrams, specialization constants, and buffer schemas.
  • API Reference – Raw WGSL exports, override tables, and TS glue helpers.
  • Preprocess Module – Shows how preprocess.wgsl is dispatched from TypeScript.
  • Sorting Module – Explains how radix_sort.wgsl ties into indirect dispatch.
  • Renderer Module – Documents how gaussian.wgsl is bound and fed with data.