Live — 136,052,269 rows · verified with sparrowCLI over the public internet
flight.sparrowflight.io
One endpoint. Every Arrow client.
A public
Apache Arrow Flight SQL endpoint served by the
Sparrow engine — the same server behind
the
live demo. Behind it:
136 million rows across
2.8 million series of public energy and macro data (EIA, Eurostat, FRED, ECB,
JODI and more). Columnar from disk to your process — no REST, no JSON, no gateway.
It exists so you don't have to take the demo's word for it: point
your own tools at it.
The endpoint
endpointflight.sparrowflight.io — the Sparrow engine (Python/Arrow C++)
…orflight2.sparrowflight.io — GizmoSQL (C++/DuckDB), same lake
…orflight3.sparrowflight.io — ROAPI (Rust/DataFusion), same lake, no auth
port443 (TLS)
protocolApache Arrow Flight SQL over gRPC
usernamedemo
passworddemo
accessread-only · public demo data · no SLA
tablesseries_data (series_id, period, value) · series_meta
Three engines, three languages, one lake. The same
136-million-row columnar store sits behind all three hosts:
flight is the Sparrow engine in serving-only mode,
flight2 is GizmoSQL (C++, DuckDB), and
flight3 is ROAPI (Rust, DataFusion). Same SQL, same wire,
same clients — swap the hostname and nothing else changes. That's what
standardizing on Arrow Flight buys.
Python — ADBC
# pip install adbc-driver-flightsql pyarrow
from adbc_driver_flightsql.dbapi import connect
con = connect("grpc+tls://flight.sparrowflight.io:443",
db_kwargs={"username": "demo", "password": "demo"})
cur = con.cursor()
cur.execute("""
SELECT period, value FROM series_data
WHERE series_id = 'PET.RWTC.D' ORDER BY period
""")
table = cur.fetch_arrow_table() # 10,217 rows of WTI, columnar
The terminal — sparrowCLI
$ sparrow connect grpc+tls://flight.sparrowflight.io:443 --basic demo:demo
$ sparrow ls # GetTables discovery
$ sparrow info series_data # schema + row count (136,052,269)
$ sparrow sql "SELECT source, COUNT(*) FROM series_meta GROUP BY source" -o md
$ sparrow sql "SELECT * FROM series_data WHERE series_id='PET.RWTC.D'" -o wti.parquet
$ sparrow sql "SELECT value FROM series_data WHERE series_id='PET.RWTC.D'" \
| duckdb -c "SELECT MIN(value), MAX(value) FROM read_arrow('/dev/stdin')"
# -36.98 · 145.31 — forty years of WTI; the pipe is raw Arrow IPC, never text
Finding series — full-text search, server-side
The catalog is searchable where it lives:
search_meta() is a BM25
full-text index over all 2.8 million series (names +
descriptions), exposed as a plain table macro — SQL in, Arrow out. It works
from every client on this page, composes with JOINs, and the index never
leaves the server.
$ sparrow sql "SELECT series_id, name, round(score,2) AS score, total_matches FROM search_meta('jet fuel europe', lim := 3)" -o md
| series_id | name | score | total_matches |
| INTL.63-2-EURO-MT.A | Jet fuel consumption, Europe, Annual | 7.43 | 685603 |
| INTL.63-2-EURO-MTOE.A | Jet fuel consumption, Europe, Annual | 7.43 | 685603 |
| INTL.63-2-EURO-QBTU.A | Jet fuel consumption, Europe, Annual | 7.43 | 685603 |
# search_meta('<query>' [, lim := N] [, dedup := true]) — stemmed, ranked;
# total_matches = the pre-LIMIT count, so truncation is always explicit
Direct tickets — the 1-RTT pull
Flight SQL reads are two round trips by design (submit the query, get a
ticket, stream it). This server also accepts tickets you
construct yourself — a known pull goes straight to
DoGet, one round trip, no SQL parse:
measured 143 ms vs 224 ms for the same 10,217-row
series. The ticket is a stable, documented contract:
// the ticket — UTF-8 JSON bytes, sent as the DoGet ticket
{ "series": ["PET.RWTC.D", …], // required · list of series ids
"start": "2020-01-01", // optional · inclusive (dashes ignored)
"end": "2024-12-31" } // optional · inclusive
# result: (series_id, period, value) ordered by (series_id, period);
# unknown ids are omitted — none matching yields an empty result with schema.
# Unknown JSON fields are ignored (additive evolution; shape is stable).
# A bare series-id string is also accepted as the whole ticket.
$ sparrow pull '{"series": ["PET.RWTC.D"], "start": "2020-01-01"}' -o md
# sparrowJS: await client.pull(["PET.RWTC.D"], { start: "2020-01-01" })
# pyarrow: client.do_get(flight.Ticket(json.dumps({"series": ["PET.RWTC.D"]}).encode()))
# or the whole query rides the ticket — arbitrary read-only SQL, 1 RTT
$ sparrow pull '{"sql": "SELECT series_id, avg(value) FROM series_data GROUP BY 1"}'
The server advertises this contract. A client
shouldn't hard-code the shape from a docs page — so the server publishes it
in-band through a GetSqlInfo vendor code
(10100), and because
GetSqlInfo is already the auth-bootstrap
call, discovery costs no extra round trip. A client reads the ticket
templates at connect and routes 1-RTT wherever one matches —
capabilities().directTickets in sparrowJS,
the direct-tickets line in
sparrow doctor --server. To our knowledge
no other Flight SQL server ships a machine-readable fast-path contract.
Two templates are advertised: series-pull
(the known shape) and sql — the whole
statement rides the ticket, so anything computed is 1-RTT
too (measured: 137 ms vs 231 ms planned for the same series).
sparrowJS ≥ 0.4.0 routes query() through
it automatically wherever it's advertised. Vendors that mint opaque
statement handles (GizmoSQL, DataFusion) don't accept raw tickets at all —
cross-vendor portability lives on the standard SQL surface.
BI tools — ODBC / JDBC
Any Arrow Flight SQL ODBC or JDBC driver (Dremio ships a free
one) pointed at flight.sparrowflight.io:443 with
TLS on and the credentials above — Tableau, Power BI, Excel Power Query, DBeaver.
The browser — sparrowJS
Browsers can't speak native gRPC; they get the same data over
gRPC-web through this site's edge —
that's exactly what
the live demo does,
with
sparrowJS decoding Arrow record batches
straight into charts.
REST + JSON — the control group
$ curl --compressed "https://api.sparrowflight.io/api/series?id=PET.RWTC.D"
[{"period":"19860102","value":25.56}, …] # 10,217 rows of gzip'd JSON
$ curl --compressed "https://api.sparrowflight.io/api/series?ids=PET.RWTC.D,PET.RBRTE.D"
The same snapshot behind a deliberately typical JSON backend, so you can race the
wire formats yourself —
the demo has
buttons for it. Measured honestly: on a 10,000-row pull, gzip'd JSON keeps up
(one HTTP round trip beats Flight's two). On a ten-series pull, Arrow is ~1.4×
ahead — 1.7 MB of record batches against
4.8 MB of JSON
the backend had to manufacture — and the gap grows with size. The structural
difference never shows in milliseconds: this endpoint serves exactly two hardcoded
query shapes; the Flight endpoints take any SQL, with no backend at all.
What's behind it
ADBC · CLI · ODBC · JDBC · gRPC-web
│
Apache Arrow Flight SQL
│
Sparrow serving node — a third of a gigabyte of RAM
│
columnar lake · 136M rows · calved nightly from the cook
The serving node runs the Sparrow engine in
serving-only
mode: the whole 136-million-row lake served lazily from columnar files —
single-series pulls in tens of milliseconds server-side, ~250 MB resident.
The same engine, pointed at your own data, is
what EnergyScope runs in
production.
Fair use: it's a demo endpoint — read-only, unmetered but unwarranted. If you want
this serving
your desk's data — on your metal, behind your
firewall — that's the product.
Start with the
Excel story.