Skip to main content

DataFusion Nexus: Specialized GPU Algorithms, Now in SQL

DataFusion Nexus is a GPU query engine built on Apache DataFusion. It evaluates whole relational physical-plan candidates and selects supported candidates for GPU execution with NVIDIA cuDF. Graph algorithms from cuGraph are explicit GPU islands callable directly from SQL, and data is read from Apache Iceberg tables and Parquet files — on S3-compatible object storage or on local disk. A relational candidate that is not selected keeps the executable DataFusion baseline. It embeds as a Rust library inside your own service, or serves remote clients over Flight SQL.

Before walking through what Nexus does, this first post explains why it exists: where today's GPU analytics stack falls short, and what we think a fix should look like.