Overview
DataFusion Nexus sits above a feature-selected GPU stack. Relational execution uses RAPIDS cuDF, and graph SQL adds cuGraph. The cuVS SQL surface can be discovered and planned, but cannot run under bounded admission yet. A full-feature source build still includes libcuvs because cuGraph links it, so the native libraries must be built in a fixed, bottom-up order.
This guide walks the whole path: system prerequisites, one recursive clone, checkout-local environment selection, building each layer, running the verified test surface, and optionally packaging the Flight SQL server into a Docker image. Building from Source owns that complete linear path; the Docker packaging path follows it in Building with Docker.
The stack
| Component | Role | Language | How it is consumed |
|---|---|---|---|
| Root adapter | DataFusion adapter, native engine (nexus-query-engine), workload contracts, benchmarks, Flight SQL server | Rust | src/, crates/ |
components/cudf | Safe Rust API over libcudf (cudf-nexus, cudf-nexus-sys, rapids-interop); pins the cudf fork | Rust + C++/CUDA | one component + submodule |
components/cuvs | Safe Rust API over libcuvs (cuvs-nexus, cuvs-nexus-sys); pins official cuVS | Rust + C++/CUDA | one component + submodule |
components/cugraph | Safe Rust API over libcugraph (cugraph-nexus, cugraph-nexus-sys); pins the cugraph fork | Rust + C++/CUDA | one component + submodule |
The root .gitmodules pins the three native sources and the root Cargo
workspace resolves all Rust components through components/*. Cargo fetches
Apache DataFusion and the remaining registry or Git dependencies automatically;
there is one product checkout and one recursive submodule update.
datafusion-nexus/
├── components/cudf/cudf/ # pinned cuDF fork submodule
├── components/cugraph/cugraph/ # pinned cuGraph fork submodule
├── components/cuvs/cuvs/ # pinned official cuVS submodule
├── crates/ and src/ # root Rust workspace
└── target/native/
├── cudf-build/ # canonical cuDF CMake cache
├── cudf-install/ # canonical cuDF/RMM install prefix
├── cuvs-install/ # canonical cuVS install prefix
└── cugraph-build/ # canonical cuGraph CMake build tree
Build order
Keep this dependency model in mind:
libcudf ── produces ──> librmm.so
├──> libcuvs
└──> libcugraph <── libcuvs
datafusion-nexus uses the enabled cuDF, cuVS, and cuGraph Rust bindings.
The full-feature build order follows those dependencies:
- libcudf and shared RMM —
bash scripts/build/build_libcudf.shdrivescomponents/cudf/cudf, installs intotarget/native/cudf-install, and prepares the one shared dependency prefix. - libcuvs —
bash scripts/build/build_libcuvs.shconsumes that RMM and installs intotarget/native/cuvs-install. - libcugraph —
bash scripts/build/build_libcugraph.shconsumes the same RMM and cuVS installs and builds intotarget/native/cugraph-build. - Unified Rust workspace —
cargo build --workspace --all-features, then barebash scripts/check_all.shfor the canonical full pre-merge surface. - Docker image — optional packaging for the standalone Flight SQL server.
docker/stage-docker-payload.shstages the release binary and the locally built forked RAPIDS libraries;docker/build-image.shthen harvests the CUDA runtime libraries into a runnable image (Building with Docker).
Requirements
The checked-out submodule gitlinks are the native-version source of truth. Install the compatible GPU and native-build environment using the RAPIDS install guide, then follow Prerequisites for this checkout's Rust, linker, and test-runner requirements. The source-build guide owns the checkout-local environment and cross-GPU build instructions.
Continue with Prerequisites.