One of these cities publishes nothing. The other one, we didn't read.
Two permit datasets that look identical in our database, and the check that tells them apart.
Every figure here was measured on and is not updated. This report is a claim about that date.
Two cities, one number
Philadelphia publishes 928,304 building permits. Every one of them carries a description field in our database, and across the whole corpus that field holds 30 distinct values.
Cincinnati publishes 161,940 permits. Every one carries a description. Across the whole corpus that field holds 16 distinct values.
From our own tables these are the same finding: a city that technically publishes a description and substantively publishes nothing. Both score 100% on any completeness measure — the field is present, populated, and empty of information.
One of those readings is right and the other is our fault.
Cincinnati publishes nothing, and we checked
Cincinnati’s source publishes 30 columns. We read all of them.
description holds 16 values: “Plumbing Permits”, “HVAC”, “Alteration”, “Fire Protection Systems”. workclass holds 52 short codes. workclassmapped holds two: “Existing” and “New”. permittype holds 19. permitclass holds three.
The richest remaining candidate is proposeduse, with 75 distinct values and a maximum length of 22 characters. That ceiling is what rules out free text: no field in Cincinnati’s feed is long enough to hold a sentence.
The absence is real, and it is the city’s. If you want to know what a Cincinnati permit is actually for, the published data cannot tell you, and no amount of better ingestion would change that.
Philadelphia publishes a great deal, and we read the wrong column
Philadelphia’s source publishes three description-ish fields, not one.
We ingest permitdescription. It holds 30 values — “PLUMBING PERMIT”, “ELECTRICAL PERMIT”, “ALTERATION PERMIT” — and several of those are case-variants of each other. It is not a description at all. It is a permit-type label wearing a description’s name, and it is the column we mapped.
Beside it sits approvedscopeofwork, populated on 99.0% of records, holding 662,784 distinct values:
REPLACE WATER HEATER
Installation of solar array in accordance with signed standard.
INSTALL (1) HVAC SYSTEM WITH DUCTWORK. (SFD)
614,562 rows — 66.9% of the populated ones — carry a description that appears exactly once in the entire corpus. That is what a real free-text field looks like.
Philadelphia is one of the better permit publishers in the country. Our database says it is one of the worst. The gap is entirely ours.
Why the two were indistinguishable
Everything we could compute without leaving our own database said the same thing about both cities:
| Philadelphia | Cincinnati | |
|---|---|---|
| description populated | 100.0% | 100.0% |
| distinct values | 30 | 16 |
| what our data implies | publishes almost nothing | publishes almost nothing |
| what the source says | rich free text, unread | genuinely nothing |
There is no query against our corpus that separates these. Null rates, distinct counts, entropy measures, completeness scores — all of them describe what we hold, and what we hold is a function of two things that look identical from inside: what the city published, and which column we chose.
A null rate is an upper bound on “the city does not publish it.” The gap between that bound and the truth is our own ingestion, and it stays unmeasured until somebody fetches the source.
How wrong this can get
We published the Philadelphia claim before we ran the check. It appeared as a fact about the city — a dataset with a description field holding thirty values across nearly a million records — in a piece whose entire argument is that permit data hides its own gaps.
It was a fact about us. The check that would have caught it is the one this report is about, and we ran it afterwards.
The failure mode is not that a field is empty. It is that an empty field and an unread field are the same shape in your database, and only one of them is the publisher’s fault.
What to do about it
If you are computing anything across permit datasets — coverage, completeness, “which cities publish X” — the number you get is a joint fact about the publishers and about your own pipeline, and it will read as though it is only about the publishers.
The check is cheap and it is not optional. For any field you are about to make a claim about: fetch the source, list its columns, and confirm the field you are reading is the field you think it is. We do this now for every city we name. It has reversed two claims: this one, and a separate case where a city was listed as publishing no issue date when it publishes one on 84.7% of its records and our pipeline was discarding it.
Two reversals from the same check, in opposite directions, in one week.
Method
Source figures are the data as fetched, before any normalisation; database figures are what we ingested. Distinct counts use trimmed, non-empty values, and the normalisation is stated because a distinct count is one quantity per normalisation and the choice is usually unstated.
Cincinnati’s full key set was enumerated from a complete scan of its stored payloads, not a sample, so “no free-text field exists” is a statement about all 30 published columns rather than the ones we happened to look at.