sparrowflight · arrow flight

The flavours of Apache Arrow Flight

Short answer: two official protocols — Arrow Flight RPC and Arrow Flight SQL — sitting on top of the Arrow IPC serialization format, plus gRPC-Web as the common browser transport adaptation. Substrait is an optional query representation Flight SQL can carry. Here is what each layer actually is.

The layers

layerwhat it is
Arrow IPCthe serialization format — how a stream of Arrow record batches is laid out on the wire (or on disk). Not a protocol; the payload everything else moves.
Arrow Flight RPCthe transport protocol. Moves Arrow record batches over gRPC (HTTP/2) with methods like DoGet, DoPut, DoExchange, GetFlightInfo and ListFlights. Think "HTTP for Arrow."
Arrow Flight SQLa standard SQL protocol built on Flight RPC. Adds query execution, prepared statements, catalog/schema/table metadata, transactions, bulk ingestion, and — optionally — Substrait plan execution. What databases like Dremio, InfluxDB 3 and GizmoSQL expose.
gRPC-Web Flightnot a separate Apache standard, but the practical browser transport. Native Flight needs raw gRPC/HTTP 2 trailers that browsers can't send, so a proxy (typically Envoy) bridges gRPC-Web; the browser keeps Flight RPC semantics over that transport.

So if someone asks "how many versions of Arrow Flight are there?", the honest answer is two official protocols (Flight RPC and Flight SQL), with gRPC-Web being a transport adaptation rather than a third protocol, and Substrait an optional payload the SQL layer can carry.

What Flight SQL adds — and who implements what

Flight SQL standardizes the SQL surface: metadata discovery (GetTables, GetCatalogs, GetDBSchemas, GetTableTypes, GetSqlInfo), prepared statements, statement execution, and optional transactions, bulk ingestion and Substrait plans. In practice vendors diverge — the spec is wide and support is uneven. This matrix is from running one client's conformance probe against five live servers (2026; full method-by-method version in docs/conform.md):

Flight SQL conformance across five servers
capabilityEnergy­ScopeGizmo­SQLRoAPI
(DataFusion)
Dremio 26InfluxDB 3
metadata RPCs (tables/catalogs/schemas)
GetTables includes schemas
GetTableTypesunimpl
prepared statements
declares result schema in FlightInfo
empty results keep their column schema✗ field⁠-⁠less
string columns arrive asUtf8Utf8Utf8ViewUtf8Utf8
custom actions (ListActions)none1388none
Substrait plan execution

The takeaway most references miss: advertising Substrait is rare — a Flight SQL server can accept a serialized Substrait plan via CommandStatementSubstraitPlan instead of SQL text, but almost none do. The honest signal is GetSqlInfo code 5 (FLIGHT_SQL_SERVER_SUBSTRAIT); most servers report it False or omit it.

Two quirks in the matrix bite real clients (found July 2026, verified against all five servers). Empty results: RoAPI/DataFusion returns a field-less schema for provably-empty results — in both the DoGet stream and FlightInfo.schema, so the column names never cross the wire and no client can recover them; the other four servers keep the schema. Notably InfluxDB 3 is DataFusion-based and does not share the gap — it's a serving-layer choice, not a DataFusion inevitability. String types: DataFusion-family servers ship parquet-sourced strings as Utf8View, which no released Arrow JS can decode (support merged upstream Nov 2025; 21.2.0-rc1 was cut 2026-07-17 but hasn't reached npm) — sparrowJS transcodes Utf8View to classic Utf8 client-side at the IPC layer, so DataFusion servers work in the browser as-is.

Where Substrait fits

Flight SQL can carry a query two ways: as SQL text (CommandStatementQuery) or as a serialized Substrait plan (CommandStatementSubstraitPlan). Substrait is a language-neutral query representation — software talking to software, no SQL parsing, no dialect differences. Its natural home is a query builder or dataframe library serializing plans under a typed API. Adoption lags the spec: you need clients that produce plans, servers that consume them, and matching Substrait versions — a real interop hazard (a producer emitting a newer FetchRel.count_expr against a consumer that reads only the deprecated integer field silently returns zero rows).

Clients and implementations

A partial map of the ecosystem, by the slot each piece fills:

pieceslot
ADBC / arrow-flight-sqlthe reference Flight SQL drivers (C++, Java, Go, Python, Rust) — how most tools connect
Dremio · InfluxDB 3 · GizmoSQLdatabases that expose a Flight SQL endpoint
sparrowJSthe gRPC-Web browser transport for Flight — Flight semantics from a web page, via Envoy
sparrowCLIa terminal client for any Flight SQL server — catalog, SQL, Parquet/CSV/JSON export, a Flight SQL conformance card, and a security-surface audit
Sparrowan index-first serving layer that speaks Flight RPC, Flight SQL and Flight SQL Substrait
A plain-language reference maintained alongside Sparrow, an Arrow Flight serving engine. Corrections welcome via the sparrowCLI repo. The conformance figures are reproducible: sparrow doctor --server against each endpoint. · sparrowflight.io · 2026