Nvidia announced the open‑source cuFile API, which enhances GPU read/write performance for high‑speed storage systems, improving data throughput for AI and HPC workloads.

Nvidia has released the cuFile API as an open‑source library, giving developers direct control over GPU‑accelerated file I/O and dramatically boosting read/write speeds on high‑performance storage arrays.

Why cuFile matters for AI and HPC

Modern AI models and high‑performance computing (HPC) workloads often move terabytes of data between storage and GPUs. Traditional pathways force data through the CPU, creating bottlenecks that limit overall throughput. cuFile eliminates this middle step by enabling GPUs to access NVMe‑based storage directly, reducing latency and increasing effective bandwidth.

The open‑source release means the API can be integrated into a broader ecosystem of tools, from data‑loading frameworks to custom storage solutions, fostering community‑driven optimizations and faster adoption across platforms.

Key features of the cuFile API

  • Direct GPU‑to‑NVMe data paths without CPU staging
  • Support for asynchronous I/O operations to overlap compute and transfer
  • Compatibility with existing CUDA streams and events for seamless integration
  • Extensible plug‑in model allowing storage vendors to add custom optimizations

Developers can call cuFile functions from within CUDA kernels or host code, using familiar patterns such as cudaMemcpyAsync but targeting storage devices instead of host memory. This approach aligns with Nvidia’s broader strategy of unifying compute and data movement under a single programming model.

Performance expectations

Early benchmarks shared by Nvidia show up to a 3‑5× increase in effective bandwidth for large, sequential reads and writes compared with traditional CPU‑mediated pathways. The gains are most pronounced on systems equipped with PCIe 4.0 or newer NVMe drives, where the full potential of the GPU’s DMA engines can be leveraged.

While real‑world results will vary based on storage hardware, workload characteristics, and software stack, the open‑source nature of cuFile encourages developers to fine‑tune performance for their specific environments.

Getting started with cuFile

The cuFile source code and documentation are hosted on Nvidia’s GitHub repository. Installation follows standard CMake procedures, and sample applications demonstrate basic file I/O, asynchronous transfers, and integration with popular deep‑learning frameworks.

Nvidia also provides a set of validation tests to verify that the storage subsystem meets the latency and bandwidth requirements needed to achieve the advertised speedups.

Open‑sourcing cuFile is a game‑changer for anyone building data‑intensive AI pipelines.

The move reflects Nvidia’s commitment to lowering the barrier for high‑performance data handling, a critical factor as AI models continue to scale in size and complexity.

For more details, see the SiliconANGLE coverage of HappyRobot funding.