Skip to main content

Integrating Nexus

DataFusion Nexus fits behind ordinary DataFusion APIs. Choose the integration surface that matches the service boundary you own:

SurfaceUse it whenStart here
DataFusion sessionYou already own a Rust backend, DataFusion session lifecycle, auth, tenancy, and result formatting.DataFusion Session
Flight SQL serverYou want a remote Arrow Flight SQL endpoint for non-Rust clients, notebooks, agents, or service-to-service calls.Flight SQL server
Embedded backend exampleYou want one Rust process with product-owned REST workflows and an ad-hoc Flight SQL side door sharing one GPU-native admission ledger.Embedded Backend Example

All three surfaces use the same cuDF relational execution path. With the cugraph feature, cugraph_* table functions are executable. Backend-installed sessions also expose discoverable and plannable cuvs_* functions, but their execution is not yet available; see cuVS execution availability. The iceberg feature adds Iceberg/Glue/REST/S3 source integration.

Common flow

  1. Build the stack with the features your deployment needs. The full source build is covered in Build & Test.
  2. Pick an integration surface: a DataFusion session, the Flight SQL server, or the embedded dual-frontend example combining both.
  3. Size GPU memory and configure table sources before running user queries. The environment-variable contract is in Configuration.
  4. If you use GPU SQL, follow Discover & Validate GPU Functions to list, describe, and validate available calls; filter the inventory by provider when needed. The GPU Function Catalog API defines the exact metadata and validation schemas.
  5. Before shipping a query, check whether its planned path stays on the GPU — and why not, if it doesn't — with GPU coverage validation (nexus_explain_coverage in embedded SQL, EXPLAIN GPU through Flight SQL, and validate_query in Rust).
  6. At your application boundary, enrich errors that carry a typed Nexus source, then keep your existing policy for ordinary DataFusion failures. See Handling Errors.

The cuGraph SQL API and cuVS SQL API sections are the function-level contracts. They share discovery and validation while documenting their distinct relation syntax, input semantics, lifecycle, and execution constraints.