sparrowflight · cli
sparrowCLI — Arrow Flight data. At the speed of the command line. Query, stream, analyze, export — cross-platform, pipeable, automatable.
Working — v0.16.0 · validated against five Flight SQL servers · open source on GitHub
The terminal is the oldest client.
Time it spoke Arrow.
Every Flight server demo today starts with "first, write a Python script." sparrowCLI is one binary: connect to any Arrow Flight SQL server, browse the catalog, run SQL, and pipe columnar data onward — to a file, a chart, or the next process.
a real session — this server, today $ sparrow connect grpc+tls://flight.sparrowflight.io:443 --basic demo:demo
✓ connected in 535 ms — EnergyScope 1.0
$ sparrow info series_data
table: series_data (TABLE)
columns:  series_id utf8  ·  period utf8  ·  value float64
rows: 136,052,269
$ sparrow sql "SELECT series_id, COUNT(*) AS n FROM series_data GROUP BY 1 ORDER BY n DESC LIMIT 3" -o md
| series_id | n | …
$ sparrow sql "SELECT * FROM series_data WHERE series_id='PET.RWTC.D'" -o wti.parquet · an ordinary query — 2 round trips
✓ 10217 rows → wti.parquet in 602 ms
$ sparrow pull '{"series":["PET.RWTC.D"]}' -o wti.parquet · the same series as a Direct Pull — 1 round trip
✓ 10217 rows → wti.parquet in 363 ms · plan (skipped: 1-RTT) · wire 172.5 KB (2.0×) · codec lz4_frame
$ sparrow sql "SELECT value FROM series_data WHERE series_id='PET.RWTC.D'" \
  | duckdb -c "LOAD arrow; SELECT MIN(value), MAX(value) FROM read_arrow('/dev/stdin')"
-36.98 · 145.31 — forty years of WTI; the pipe is raw Arrow IPC, never text
(one-time: duckdb -c "INSTALL arrow FROM community" — read_arrow needs the explicit LOAD, it never autoloads)
Query
Any Flight SQL server — catalog discovery via the GetTables RPC, the one path that works identically everywhere
Stream
Columnar on the way through — Arrow record batches end to end; no row-by-row straw
Inspect
sparrow info — schema, catalog and row count before you commit to pulling anything
Export
Parquet · CSV · JSON · JSONL · Arrow · Markdown-o data.parquet writes a file; in a pipe, raw Arrow IPC streams to the next tool by default
Security
mTLS and sealed exports — client certificates for private-CA deployments (--tls-ca/--tls-cert/--tls-key), and --encrypt-key writes in-spec Parquet Modular Encryption that DuckDB/Spark/pyarrow open with the key and refuse without
Diagnose
sparrow doctor — walks config → DNS → TCP → TLS → auth → Flight SQL → round trip and names the layer that breaks, showing what the wire actually presented (TLS version, ALPN, certificate issuer). --server swaps in a conformance card: ten probes of the Flight SQL surface, plus a GetSqlInfo capability decode that lints the block — its first catch was this site's own demo server
Measure
sparrow ping and sql --stats — latency percentiles that split network from server, and the stream's full anatomy: plan · first byte · batch signature · wire-vs-decoded bytes + the declared IPC codec (lz4/zstd shows instantly) · dictionary-batch counts · pacing gaps · a first-second-vs-overall throughput ramp · per-column types and encodings. Every benchmark on this site is reproducible with a flag
Check
sparrow check — a doctor for the data itself: nulls, duplicate keys, staleness, frozen series, σ-outliers — computed server-side as SQL aggregates, the table is never downloaded; exit 1 on findings gates a pipeline
Compare
sparrow diff — the drift gate: one table on two servers — schema, COUNT(*), time bounds, numeric fingerprint, all server-side. Identical → exit 0; drift → exit 1, so one cron line watches a replica. It proved this site's two public engines serve an identical 136M-row snapshot in four aggregates
Audit
sparrow audit — the security surface: a Flight SQL client sends arbitrary SQL, and a DuckDB-backed server runs it with powers that reach past querying — reading host files, listing directories, writing files, fetching URLs (SSRF), loading native extensions, changing server config. audit probes each with a benign check (reads /etc/hostname, writes /dev/null, dials a dead loopback port) and reports what's exposed; exit 1 gates a deploy. A defender's tool — it verifies hardening (per DuckDB's own securing manual), it doesn't exploit. This endpoint audits clean
Agents
How AI agents look at Flight datasparrow orient maps any Flight server in one command (vendor, tables, schemas, as markdown); sql -o md returns results an agent reads natively, capped so a careless SELECT * can't flood its context; query builds the one-liner SELECT (--where --order --limit); exit codes distinguish "server down" from "bad SQL", doctor -o json turns a dead connection into a diagnosis the agent can act on, and sparrow feedback lets any user — human or agent — reach the maintainers from anywhere: it posts to sparrowflight.io over HTTPS, independent of whichever server is configured
Humans
Terminal niceties — table output on a TTY draws a sparkline under each numeric column, sampled from the whole stream (the 40 rows you see, the shape of all 500 you asked for); sparrow completion bash|zsh|fish gives tab-completion for every command and flag
one command maps a server — sparrow orient, real output, abridged $ sparrow orient
# EnergyScope 1.0
endpoint: `grpc+tls://flight.sparrowflight.io:443` (profile: sparrow)
## tables
| energyscope | main | series_data | TABLE |
| energyscope | main | series_meta | TABLE |
## series_data
| series_id utf8 | period utf8 | value float64 |
## series_meta
| series_id | name | units | description | source | start | end | last_updated | …
next: `sparrow info <table>` · `sparrow sql "SELECT … LIMIT 20" -o md`
That's the whole catalog — vendor, tables, every schema — as markdown, in one round trip. A human reads it; an AI agent reads it natively. It's why Claude Code can orient itself on any Flight server with a single command.
the demo server ships full-text search as a macro — ls shows it, sql calls it $ sparrow ls "%meta"
energyscope · main · series_meta · TABLE
energyscope · main · search_meta · MACRO
$ sparrow sql "SELECT * FROM search_meta('jet fuel europe', lim := 3)" -o md
| series_id | name | description | score | total_matches |
| INTL.63-2-EURO-MT.A | Jet fuel consumption, Europe, Annual | … | 7.43 | 685603 |
Anything a server advertises in GetTables, the CLI can reach — this endpoint advertises search_meta, a server-side BM25 full-text index over 2.8 million series descriptions: search_meta('<query>' [, lim := N] [, dedup := true]). Ranked matches (default cap 50), dedup := true collapses per-frequency duplicates, and total_matches carries the full hit count so the cap is never silent. A macro, not a subcommand — ls discovers it, plain SQL composes with it (WHERE, JOIN, anything).
The session above is real — the endpoint is this site's public Flight server, the same 136-million-row lake behind the browser demo, and demo:demo is yours to use. The client is validated against GizmoSQL, ROAPI/DataFusion, Dremio, InfluxDB 3, and Sparrow — both auth families (basic and bearer + headers), one binary, pure Go, no runtime. The source is public — Apache-2.0, on GitHub — and release binaries ship for Linux, macOS and Windows (amd64 + arm64).
when it doesn't connect — sparrow doctor names the layer that broke $ sparrow doctor -s grpc+tls://fixture:31337 --basic user:pass
 ✓ config    profile "(ad-hoc)" · auth basic · TLS system roots
 ✓ dns       fixture → 192.168.132.91 (18 ms)
 ✓ tcp       connected to 192.168.132.91:31337 (2 ms)
 ✗ tls       x509: certificate signed by unknown authority
            wire presented: issuer "Norton Web/Mail Shield Untrusted Root"
             hint: something between you and the server is intercepting TLS
 · auth      not reached
3 ok · 0 warn · 1 fail — first failure at tls
Every connection failure looks the same from a client — one opaque error. sparrow doctor walks the stack one layer at a time, and on a TLS failure it shows the certificate the wire actually presented. The capture above is real: a fixture that "wouldn't verify" turned out to be antivirus re-signing the connection. -o json hands agents the same diagnosis as structured data; any hard failure exits 2.
which parts of Flight SQL does a server actually implement? — doctor --server, real capture $ sparrow doctor --server
 ⚠ GetSqlInfo         5 entries — minimal (154 ms)
                       ⚠ no capability flags advertised (SQL / Substrait / transactions / cancel)
                       ⚠ code 100 (STATEMENT_TIMEOUT) expects int32, got string_value "EnergyScope Flight SQL"
 ✓ GetTables          2 tables listed (145 ms)
 ✓ GetTables+schema   schemas populated (2/2 tables) (154 ms)
 ✓ Prepare            prepare → execute → close round trip (291 ms)
 ✓ SELECT version()   v1.5.4 (150 ms)
 ⚠ ListActions        unsupported (Unimplemented)
8 supported · 2 unsupported · 0 errored — informational, exit 0
Ten probes of the Flight SQL surface — catalog RPCs, prepared statements, declared schemas, actions — and a GetSqlInfo decode that lints the capability block against the spec. The capture above is real and the server it flags is this site's own demo: an external tester proposed the linter, and its first catch was our mistyped SqlInfo codes — fixed the same day (the card now reads ✓ 9 entries · SQL ✓ · Substrait ✓ · read-only ✓). gizmosql decodes clean: 85 entries · SQL ✓ · Substrait ✗ · txns ✓ · cancel ✓. Same card, five vendors — the quirk matrix is in docs/conform.md.
the same series, two ways — an ordinary query vs a Direct Pull, real capture $ sparrow sql "SELECT series_id, period, value FROM series_data WHERE series_id='PET.RWTC.D' ORDER BY period" -o table --max-rows 5 · 2 round trips
$ sparrow pull '{"series":["PET.RWTC.D"]}' -o table --max-rows 5 · 1 round trip
series_id   period    value
PET.RWTC.D  19860102  25.56
PET.RWTC.D  19860103  26
PET.RWTC.D  19860106  26.53 …
both print identical rows · the pull: plan (skipped: 1-RTT) · wire 172.5 KB · 2.0× · codec lz4_frame
pull — a Direct Pull — sends a client-constructed ticket straight to the server (the Flight DoGet RPC) — no GetFlightInfo, no SQL text — so a known read is one round trip, not two. Sparrow serving nodes take JSON in two dialects: {"series":[…]} by key, or {"sql":"…"} for an arbitrary read-only query. The wire comes back lz4-compressed because the client advertised it (negotiated, never forced). doget still works as an alias. Opaque-handle vendors don't accept client tickets — doctor --server probes which kind a server is.
no SQL on the wire at all — a serialized Substrait plan, real capture $ sparrow sql --substrait wti_plan.pb -o csv --stats
(producing wti_plan.pb — and the optimizer footgun — is the recipe in docs/substrait.md, linked below)
series_id,period,value
PET.RWTC.D,20260706,69.6
PET.RWTC.D,20260702,69.73
PET.RWTC.D,20260701,69.74 …
plan (GetFlightInfo)     164 ms · total 245 ms
Flight SQL can carry a query as SQL text or as a serialized Substrait plan — sparrow implements both sides of the spec: sql --substrait plan.pb sends CommandStatementSubstraitPlan (with a capability pre-check, so a server that doesn't accept plans yields a clear message, not a raw gRPC error), and this site's own serving node consumes plans — the 277-byte plan above ran against the 136M-row public table with no SQL string anywhere on the wire. Substrait's audience is software talking to software: query builders and dataframe libraries serializing plans under a typed API — no SQL generation, no dialect differences. Producer recipe and the optimizer footgun we hit: docs/substrait.md.
and a doctor for the data itself — sparrow check, real capture $ sparrow check trades --key k --max-age 7d
 ✓ rows      60
 ✗ keys      1 duplicated (k, t) groups — e.g. DUP
 ⚠ time      newest point 13.7 days old — older than --max-age 7d: is the feed still running?
 ⚠ frozen    1 entity holds a constant value across ≥10 observations — e.g. FLAT
 ✓ numeric   v: min 1 · max 900000 · avg 16432.25
5 ok · 2 warn · 1 fail — checked in 0.0 s (10 queries, server-side)
sparrow check screens a dataset without downloading it — every check is one conservative SQL aggregate, pushed to the server: duplicate keys, NULL census, staleness against --max-age, frozen series (a stuck feed's signature), σ-outliers. Findings exit 1, so sparrow check t --key id && deploy is a data gate in CI.
has the replica drifted? — sparrow diff, real capture: two engines, one snapshot $ sparrow diff series_data --against gizmo2 --time period
A: demo       grpc+tls://flight.sparrowflight.io:443
B: gizmo2     grpc+tls://flight2.sparrowflight.io:443
 ✓ schema       3 columns, identical
 ✓ rows         136,052,269
 ✓ time         1859 → 2027Q4
 ✓ Σ value     count 136,052,269 · avg 1.113048882512338e+07
4 checks — identical
sparrow diff compares one table on two servers — schema, row count, time bounds, a numeric fingerprint — as conservative aggregates on both sides; nothing is downloaded. The capture proved this site's two public engines (a Python server and C++ GizmoSQL) serve an identical 136-million-row snapshot. Identical exits 0, drift exits 1 — one cron line becomes a replication monitor.
is it the network or the server? — sparrow ping · sql / pull --stats, real captures $ sparrow ping -n 5
          min    p50    p95    max
tcp     61.3   63.4   75.9   75.9  ms
rpc     75.4   81.4   86.1   86.1  ms   (5/5 ok)
≈ network 63.4 ms + server 18.0 ms (medians)
$ sparrow sql "SELECT * FROM series_data LIMIT 500000" --stats
plan 79 ms · first byte 404 ms · stream 2304 ms · rows/batch p50 2,048 · 78 Mbit/s
wire 22.5 MB · decodes to 22.4 MB (1.0×) · no body compression declared
pacing: 82% of the stream is waiting between batches (paced upstream, not the wire)
$ sparrow pull '{"series":["PET.RWTC.D"]}' --stats · a negotiated-lz4 Direct Pull
wire 172.5 KB · decodes to 347.4 KB (2.0×) · codec lz4_frame
ping runs a bare TCP connect next to a lightweight RPC on the warm channel — the gap between the two medians is the server, not the network. --stats breaks a query into plan / first byte / stream, with the bytes counted at the gRPC layer as they crossed the wire. Both captures above are real, against this site's public endpoint. Every benchmark number on this site is reproducible with a flag.
the landscape, verified july 2026 — other CLIs reach Flight SQL; none preserve Arrow end-to-end
toolscopeArrow stays Arrow?
flight_sql_client (arrow-rs) any Flight SQL server ✗ — self-described "basic" example binary; one-shot RPC commands, text out
arrow-flight-sql-client any Flight SQL server ✗ — small RPC-mirror CLI, text out
usql 40+ databases; Flight SQL as one driver ✗ — excellent universal shell, but results flatten through database/sql to rows and text
bendsql (né "Arrow CLI") Databend only — what happens when an Arrow CLI grows up inside one vendor
sparrow any Flight SQL server ✓ — raw Arrow IPC in a pipe, parquet sinks, typed formats
The gap sparrow fills isn't "a CLI exists" — it's Arrow-native ergonomics: catalog discovery over the Flight RPCs, profiles, orient, output that follows the consumer, and conventions agents can script against.
the full reference — every command, every flag (v0.16.0)
commanddoesown flags
connect <uri>verify a server + save it as a profile (the first becomes default)--name profile name
orientone-shot markdown map: vendor, every table, every schema
ls [pattern]list tables via the GetTables RPC; the pattern is a server-side SQL LIKE (%, _, case-sensitive)-o format
info <table>schema, catalog, row count--no-count skip the COUNT(*)
sql "…" · - (stdin)run a statement-o format/file
-f file.sql
--schema columns + types only, no rows
--bigint-as-string int64 as quoted (JS precision)
--substrait plan.pb — execute a serialized Substrait plan (pre-checks the server's capability)
--max-rows
--stats query anatomy
--ipc IPC manifest
--encrypt-key hex|env:VAR|file:path seals parquet
query <table>build the one-liner SELECT for you; output flags work like sql--cols a,b (expressions pass through)
--where clause
--order col · --desc
--limit N
+ all of sql's output flags
head <table> [n]preview the first n rows (default 10) — the SELECT * … LIMIT n you keep typing-o format
pull '<ticket>'Direct Pull (1-RTT): send a client-constructed ticket straight to the server (DoGet), skipping GetFlightInfo. Sparrow serving nodes accept JSON — {"series":[…]} or {"sql":"…"}; opaque-handle vendors don't (use sql). Requests lz4 on the wire by default (negotiated — a server that offers it compresses, arrow decodes it transparently). --stats shows plan (skipped: 1-RTT) and the codec lz4_frame it received. doget is a hidden alias--accept-compression lz4 (default; "" to opt out)
@file or - stdin
+ all of sql's output flags
profile <table>per-column nulls % · approx distinct · min · max, in one server-side pass-o json
doctorlayered connection diagnosis: config → dns → tcp → tls → auth → flight sql → round trip--server conformance card: 10 Flight SQL surface probes (incl. an IPC-compression probe — reports offered — codec lz4_frame) + GetSqlInfo capability lint
-o json
check <table>server-side data health screen — the table is never downloaded--key entity cols
--time col
--value col
--max-age 7d|48h
--strict warnings fail too
--show-violations emit offending keys + conflicting values
--approx memory-safe (HyperLogLog) uniqueness
--explain echo each stage's SQL
--baseline prior.json — gate on regressions
-o json
diff <table>drift gate: schema · COUNT(*) · time bounds · numeric fingerprint vs a second server; identical exit 0, drift exit 1--against profile or anonymous URI (required)
--time col bounds
-o json
auditsecurity surface: what client SQL can reach beyond queries — file read/list/write, SSRF, extension load, config change. Benign probes; exit 1 if exposed. Run against a server you operate-o json
pinglatency percentiles: network vs server-n rounds (10)
--interval pause (200ms)
-o json
feedback "msg"message the sparrow maintainers — goes to sparrowflight.io, independent of the connected server--category bug|idea|general
--from name ($SPARROW_USER)
profileslist saved connections · use <name> sets the default · rm <name> removes
completion bash|zsh|fishshell tab-completion script — source <(sparrow completion bash)
version · help [cmd]version string · per-command help (also <cmd> -h)
connection flags — accepted by every data command
-sprofile name or an ad-hoc grpc:// / grpc+tls:// URI (default: the default profile)
--basic user:passbasic auth — an API key as the user works; a Bearer handoff is adopted automatically
--bearer TOKENbearer token auth (InfluxDB 3 style)
--header k=vextra per-call gRPC metadata, repeatable (e.g. --header database=mydb)
--tls-skip-verifyaccept self-signed TLS certificates
--tls-ca · --tls-cert · --tls-keycustom CA bundle and client certificate/key for mTLS (PEM paths)
-h · --helpper-command help with usage, examples and every flag (same as sparrow help <cmd>)
Output (-o): table · csv · json · jsonl · md · arrow — or a file path (.parquet .csv .json .jsonl .arrow .md). Defaults: a TTY gets a table (numeric columns add a whole-stream sparkline), a pipe streams raw Arrow IPC; -o md to stdout caps at 1,000 rows (file sinks always get everything). Exit codes: 0 ok · 1 query error or a gate hit (check findings, diff drift, audit exposure — the command ran fine, it found something; that's what makes them CI gates) · 2 connection/auth · 3 usage — scripts and agents branch on them. Profiles live in ~/.sparrow/config.json.
sparrowflight.io · 2026