Skip to content

IO Module Architecture

The IO module serves as the Unified Data Ingestion Layer for Visionary. It powers everything related to loading splat data and 3D mesh data. It sniffs file formats, parses raw bytes, and hands off GPU-ready buffers to the rest of the stack. The module abstracts the complexity of various file formats (Standard PLY, Compressed Gaussian formats, and Three.js Meshes) into a consistent memory layout (DataSource) that the application can consume without knowing the underlying file structure.

Layered design

┌─────────────────────────────────────┐
│ Application layer                   │  UI, progress callbacks
├─────────────────────────────────────┤
│ Universal loader facade             │  Format detection, delegation
├─────────────────────────────────────┤
│ Format-specific loaders             │  PLY, SPZ, KSplat, SPLAT, SOG, etc.
├─────────────────────────────────────┤
│ Data processing & validation        │  Parsing, transforms, checks
├─────────────────────────────────────┤
│ Buffer management                   │  Packing, memory tuning
└─────────────────────────────────────┘

Interface layer (index.ts)

Defines the TypeScript contracts that all loaders must honour. Every loader returns a DataSource (either GaussianDataSource or ThreeJSDataSource) so the renderer never cares about original file formats.

interface GaussianDataSource {
  gaussianBuffer(): ArrayBuffer;    // Packed splat parameters (half-floats)
  shCoefsBuffer(): ArrayBuffer;     // Packed SH coefficients
  numPoints(): number;              // Total splats
  shDegree(): number;               // SH degree (0..3 typical)
  bbox(): { min: [number, number, number]; max: [number, number, number] };
  center?: [number, number, number];
  up?: [number, number, number] | null;
  kernelSize?: number;
  backgroundColor?: [number, number, number];
}

interface ThreeJSDataSource {
  object3D(): THREE.Object3D;      // Three.js object
  modelType(): string;              // Format identifier
  bbox(): { min: [number, number, number]; max: [number, number, number] };
  // ... optional metadata
}

Design principles:

  • A uniform interface regardless of source format.
  • Lazy buffers that are produced when accessed, not eagerly stored.
  • Optional metadata for format-specific hints (camera defaults, kernel sizes).
  • Fully typed TypeScript ergonomics for downstream callers.

Universal Loader (universal_loader.ts)

A facade that knows how to pick the right loader, stream progress, and keep the API surface consistent. It manages two separate registries: one for Gaussian formats and one for Three.js mesh formats.

class UniversalLoader implements ILoader, LoaderRegistry {
  private gaussianLoaders = new Map<string, ILoader>();
  private threeLoaders = new Map<string, ILoader>();

  register<T extends DataSource>(loader: ILoader<T>, extensions: string[], type: LoaderType): void;
  getLoader(filename: string, mimeType?: string, options?: { isGaussian?: boolean }): ILoader | null;
  loadFile(file: File, options?: LoadingOptions): Promise<DataSource>;
  loadUrl(url: string, options?: LoadingOptions): Promise<DataSource>;
  loadBuffer(buffer: ArrayBuffer, options?: LoadingOptions): Promise<DataSource>;
  canHandle(filename: string, mimeType?: string, options?: { isGaussian?: boolean }): boolean;
  getAllSupportedExtensions(): string[];
}

LoaderType Enum: 

enum LoaderType {
  GAUSSIAN = 'gaussian',
  THREE = 'three'
}

Highlights:

  • Dual Registry System: Separate registries for Gaussian and Three.js loaders, allowing the same extension (e.g., .ply) to be handled by different loaders
  • Intelligent Routing: Auto-detects formats by extension, MIME type, and magic-byte sniffing
  • PLY Disambiguation: Automatically detects if a PLY file is 3DGS or mesh format by checking header properties (rot_0, scale_0, opacity, etc.)
  • Runtime Registration: Supports runtime registration of new loaders (plugins, enterprise formats, etc.)
  • Progress Reporting: Emits stage-based progress updates so UX feels alive
  • Unified Output: Normalises all results to DataSource (either GaussianDataSource or ThreeJSDataSource)

Routing Strategy:

  1. Explicit Configuration: If options.isGaussian === true, forces search in Gaussian loader registry
  2. Special Handling: .compressed.ply is immediately routed to CompressedPLYLoader
  3. Extension Matching: Files like .spz, .sog, .splat, .ksplat are routed to their respective loaders
  4. PLY Detection: For .ply files, reads header to detect 3DGS properties (rot_0, scale_0, opacity)
  5. Content Sniffing: Detects binary magic bytes (GZIP: 0x1f8b for SPZ, KSPL: "KSPL", ZIP: 0x504b0304 for SOG) to disambiguate formats
  6. Fallback: If extension doesn't match, iterates through loaders using canHandle() method

Format-Specific Loaders

The IO module includes specialized implementations for multiple Gaussian formats:

PLY Loader (ply_loader.ts): - Handles ASCII and binary PLY with Visionary extensions - Validates required 3DGS properties (rot_0, scale_0, opacity, etc.) - Supports both little-endian and big-endian binary formats - Performs normalization, quaternion conversion, and SH coefficient packing

SPZ Loader (spz_loader.ts): - GZIP-based container format - Handles quantized positions/rotations - Detected by .spz extension or GZIP magic bytes

KSplat Loader (ksplat_loader.ts): - Luma AI optimized block format - Detects "KSPL" magic bytes (first 4 bytes) - Optimized for fast loading

Splat Loader (splat_loader.ts): - Raw binary dump format (32-byte stride per point) - Extremely fast loading (no parsing overhead) - Format: [x, y, z, r, g, b, a, rot_0, rot_1, rot_2, rot_3, scale_0, scale_1, scale_2] (all float32)

SOG Loader (sog_loader.ts): - SuperOrdered Gaussians format - Supports both Raw and ZIP-compressed variants - Detects ZIP magic bytes (0x504b0304) - Handles hierarchical Gaussian organization

Compressed PLY Loader (compressed_ply_loader.ts): - Quantized PLY format using chunk-based data compression - Uses chunk metadata to store quantization bounds (min/max values) - Packs positions, rotations, scales, and colors with bit-level quantization - Each chunk contains 256 points with shared quantization parameters - Detected by .compressed.ply extension

Three.js Adapters (threejs_adapters.ts): - Wraps standard Three.js loaders to conform to ILoader interface - Supports FBX, GLTF, OBJ, STL, and Mesh PLY - Automatically applies shadow settings and fallback materials - Preserves original materials when available

All loaders implement the ILoader<T> interface:

interface ILoader<T extends DataSource = DataSource> {
  loadFile(file: File, options?: LoadingOptions): Promise<T>;
  loadUrl(url: string, options?: LoadingOptions): Promise<T>;
  loadBuffer(buffer: ArrayBuffer, options?: LoadingOptions): Promise<T>;
  canHandle(filename: string, mimeType?: string): boolean;
  getSupportedExtensions(): string[];
}

Key Processing Steps (PLY Example):

  1. Header parsing – detect ASCII vs binary, collect property schema, validate required fields
  2. Vertex extraction – decode rows (text or DataView) into typed arrays
  3. Gaussian processing – map positions, quaternions, scales, opacities, SH coefficients
  4. Buffer packing – compress to half-floats and interleaved SH words ready for GPU upload

Data processing pipeline

PLY file ──► header parse ──► vertex decode ──► gaussian transform ──► buffer packing
  • Positions maintain min/max to build bounding boxes.
  • Quaternions are normalised ((x, y, z, w) → (w, x, y, z)).
  • Log-space scales become linear with Math.exp.
  • Opacity runs through a sigmoid.
  • Covariance matrix is derived and stored in upper-triangular form.
  • SH coefficients pack two float16 values into each uint32.

Progress architecture

LoadingOptions supports cancellation and fine-grained progress reporting:

interface LoadingOptions {
  onProgress?: (progress: LoadingProgress) => void;
  signal?: AbortSignal;
  debug?: boolean;
  isGaussian?: boolean;  // Explicit format hint
}

interface LoadingProgress {
  stage: 'fetch' | 'parse' | 'pack' | string;
  progress: number;            // 0..1
  message?: string;
}

The loader updates progress at stage boundaries and every few thousand vertices so long-running imports stay responsive.

Memory strategy

  • Allocate output buffers up front (Uint16Array for gaussians, Uint32Array for SH).
  • Stream rows point-by-point to avoid spikes in memory usage.
  • Convert to half-floats directly while iterating—no temporary arrays.
  • Clean up intermediate views immediately after use.

Error handling

The IO module validates at multiple levels and surfaces descriptive errors:

Layer Example validation Typical error
Format Magic header, ASCII marker UnsupportedFormatError
Schema Required properties present MissingPropertyError
Data Finite numeric values MalformedRowError
Processing Covariance math succeeds ProcessingError

All errors extend a shared IOError that includes stage?: 'fetch' | 'parse' | 'pack' | ... so the UI can annotate failures.

Format Detection Utilities

The IO module provides several helper functions for format detection:

GaussianFormat Enum: 

enum GaussianFormat {
  PLY = 'ply',
  SPZ = 'spz',
  KSPLAT = 'ksplat',
  SPLAT = 'splat',
  SOG = 'sog',
  COMPRESSED_PLY = 'compressed.ply'
}

Detection Functions: - detectGaussianFormat(filename: string): GaussianFormat | null – Returns format enum from filename - isGaussianFormat(filename: string): boolean – Boolean check for UI filtering - getSupportedGaussianFormats(): string[] – Returns all supported Gaussian extensions

Type Guards: - isGaussianDataSource(data: DataSource): data is GaussianDataSource – Runtime type checking - isThreeJSDataSource(data: DataSource): data is ThreeJSDataSource – Runtime type checking

PLY Disambiguation Logic

The module implements sophisticated PLY file disambiguation to distinguish between 3D Gaussian Splatting files and traditional mesh files:

private is3dgsPlyFromHeader(header: string): boolean {
  const requiredKeywords = [
    'property float opacity',
    'property float scale_0',
    'property float scale_1',
    'property float scale_2',
    'property float rot_0',
    'property float rot_1',
    'property float rot_2',
    'property float rot_3'
  ];

  return requiredKeywords.every(keyword => 
    header.toLowerCase().includes(keyword)
  );
}

Process: 1. For .ply files, read first 4KB of header 2. Check for all required 3DGS properties 3. If all present → route to PLYLoader (Gaussian) 4. If missing → route to ThreeJSPLYLoaderAdapter (Mesh)

Scene Saving Architecture

The unified-scene-saver.ts module provides scene export functionality:

interface SaveUnifiedSceneParams {
  scenes: SaveUnifiedSceneSceneEntry[];
  folderHandle: FileSystemDirectoryHandle;
  meta?: any;
  cameraParams?: any;
  totalFrames?: number;
}

Features: - Saves scene structure as JSON (scene.json) - Copies model files to target directory - Handles FileSystem API permissions - Validates folder handle validity - Clears target folder before saving - Supports keyframes and model metadata

Process: 1. Verify folder handle permissions 2. Clear existing folder contents 3. Copy all model files (deduplicated by name) 4. Generate scene.json with scene structure 5. Write JSON to target folder

Extensibility Playbook

To add a new format:

  1. Implement ILoader for the format (either ILoader<GaussianDataSource> or ILoader<ThreeJSDataSource>)
  2. Wrap results in a class that satisfies the appropriate DataSource interface
  3. Register the loader with the universal facade: 
    loader.register(new FooLoader(), ['.foo'], LoaderType.GAUSSIAN);
// or
loader.register(new FooMeshLoader(), ['.foo'], LoaderType.THREE);
  4. Optionally share helper utilities (half-float packing, bounding boxes) for consistency

The architecture also anticipates compressed or streaming formats—just feed the universal loader a loader that understands the protocol.