CSLabs
Product image for DeltaX

DeltaX

Modular, HPC pipeline that ingests, processes, fuses, and analyzes large-scale sensor data with plug-and-play algorithms and AI.

DeltaX — High-Performance Data-Processing Framework

DeltaX is a modular framework that processes and manages terabytes of data. It is implemented using an extended parallel pipes-and-filters design pattern, where filters are algorithmic processing units and pipes are message dictionaries that define the interfaces between them. This design enables seamless interchange of filters, easy addition of parameters, and minimal disruption when swapping algorithms.

DeltaX supports the interchange of different computer vision and AI algorithms, maintains system-level tracking and archival of results, and provides flexibility by treating the GUI and hardware interfaces as filters. A global shared memory store minimizes data transport costs, while serialization and restart support simplify algorithm testing and design. Developed in Python with bindings to languages such as C++, DeltaX runs efficiently on platforms ranging from small NUCs to large HPC clusters.

Mission Impact

  • Enables rapid algorithm interchange (e.g., Mask R-CNN ↔ U-Net) without redesigning pipelines
  • Supports configuration and result archival for repeatable experimentation
  • Provides restart and serialization features to protect long-running jobs from data loss

Key Capabilities

  • Configurable Pipelines — Define workflows using message dictionaries for flexible interfaces
  • AI/ML Testing Flexibility — Swap and test multiple algorithms with minimal setup
  • Checkpoint & Restart — Serialization allows pipelines to resume after interruptions
  • Global Shared Memory Store — Reduces the cost of data transport across filters

Technical Stack

  • Architecture: Extended pipes-and-filters design pattern
  • Pipelines: Human-readable configs which can be exported/imported
  • Core Language: Python with C++ bindings for HPC
  • Execution: Supports processes, threads, and co-routines
  • Data Management: Global shared memory, serialization, and archival

See DeltaX in Action

This is the dialog window that allows the user to configure an algorithm to run. Each algorithm type has its own set of parameters and some have presets for faster setup. The user can save/load algorithm configurations.

1. Algorithm Type – Select the algorithm type to configure

2. Preset – Select a configuration preset for the selected algorithm type

3. Algorithm Parameters – Configurable parameters for the selected algorithm type

4. Parameter Description – Description for the parameter that the mouse is currently hovering over

5. Current Loaded Configuration – If the current configuration is saved to or loaded from a file, the path to that configuration file is displayed here

6. Reset to Default – Reset the parameters to default settings

7. Save/Load Controls – Save/load the current configuration of parameters

8. Start Algorithm – Start running the current algorithm type with the current configuration of parameters