Litdata

Litdata is a high-performance open-source data optimization library developed by Lightning AI, designed to stream, optimize, and pre-process massive datasets directly into deep learning models with zero bottleneck latency.