A Dataset With No Database
Every developer who has shipped data to someone else has hit the same wall: you don't know what the other side can open — not their database, not their language, not their libraries. So you fall back to the format that assumes nothing, and for structured data that means a spreadsheet. CSV, XLSX: the universal medium, and also where structured data goes to rot — untyped, schema-less, forked into a dozen copies named final_v3_REAL.xlsx.
That safe choice has a cost. A spreadsheet carries no schema and no source of truth, so each recipient opens their own copy, wrangles it by hand, and the versions fork. But who is on the receiving end is changing. Increasingly the recipient is an AI agent with a sandbox and a runtime, and for that reader you can assume far more than "opens a spreadsheet." So the recommendation shifts, and it is boring: bundle your data into a skill and tell the agent to load it into SQLite. That is the whole trick — you ship a CSV and a one-line instruction, and the agent stands up a real database at query time and answers exactly.
It works because SQLite ships inside Python's standard library — and in a sandbox that runs Python, which most agent runtimes do, there is nothing to install and nothing to download. The agent loads your CSV, runs SQL against it — filters, joins, aggregates, even full-text search — and hands back an exact answer computed by an actual query engine. The data ships as text you can read in a diff, and the file never enters the model's context: only the query and its result do, so a million-row table costs the same handful of tokens as a hundred-row one.
The one thing to avoid is skipping the engine — pasting the table into the skill's instructions and letting the model read it directly. I handed a current Sonnet-class model a 300-row table and asked it to count the rows in each category and name the five largest; it got the count off by one and the sort wrong. That is a single run, but it illustrates a well-worn failure: a language model is not a query engine, and asking one to do arithmetic over a few hundred rows in its head buys you a confidently wrong answer. Let SQLite do it.
That is the floor. The open question is whether a fancier format beats plain CSV-into-SQLite by enough to bother — so I ran the tour so you don't have to. The verdict, in one table:
| Option | Runs on | Install | Diffable | What the tests showed |
|---|---|---|---|---|
Table pasted into SKILL.md | the model itself | — | yes | Miscounted rows, botched the sort — a model is not a query engine |
Bundled SQLite .db | SQLite (stdlib) | none | no | Works, but an opaque binary in the repo |
| CSV / JSON → SQLite | SQLite (stdlib) | none | yes | Exact; same token cost as a .db, roughly half the size, loads in milliseconds |
| Parquet → DuckDB | DuckDB | pip | no | ~30 ms on 500k rows — but neither is preinstalled |
numpy .npy + memmap | numpy | usually present | no | Fastest (~4 ms at 100k rows) and smallest on disk, but no SQL |
On tokens they all tie — only the query and its result ever reach the model — so the choice is really the three middle columns: what you have to install, whether a human can read the diff, and whether you get SQL.
Which is where KISS and YAGNI come in. A reference dataset you bundle into a skill is small — kilobytes to a few megabytes, hundreds to low-thousands of rows. Past a few hundred thousand rows the specialist formats do start to win: Parquet with a query engine like DuckDB is genuinely faster and smaller, and memory-mapped numpy arrays are faster still for raw number-crunching. But you are not at that scale, and getting there means trading a text file anyone can read for a binary the sandbox may not open without a network install. You aren't gonna need it. Ship the spreadsheet.
Which is the joke, and it is the same joke as dropping the embedding model: after all the format-shopping, the answer is the humble thing we already hand to humans. The difference isn't the format, and it isn't that an agent can query where a person can't — plenty of people open a CSV in a real tool. It's that a skill ships the querying with the data. A bare spreadsheet arrives with no instructions, so each recipient forks a private copy that drifts; a skill arrives with the data and the one line that says load this into SQLite and query it, so the agent works the file you actually shipped. Hand someone a spreadsheet and you hope. Hand an agent a skill and you know.