District knowledge layer
Some district profiles include a "District knowledge" section with attributed statements and primary-source excerpts. Full budget reports can also surface budget-relevant source notes as local public-record context. This page explains what appears there, how each entry is sourced, and the rights and confidence conventions we follow.
What it is
The district knowledge layer is a curated set of public-record statements — board resolutions, budget presentations, meeting minutes, and linked press quotes — displayed alongside the statewide NJDOE data on district profile pages. Budget-relevant notes can also appear in full district budget reports as a bridge from generated UFB signals to local records and meeting statements. Each entry cites one primary source and one named speaker (when the statement is attributed) or one official document (when it is not).
This layer is separate from and complementary to the NJDOE User-Friendly Budget module. The UFB module shows statewide-comparable numbers from NJDOE's own CSV files. The knowledge layer adds local context available only in primary documents that the state does not centralize.
Rights
Every entry carries a rights tag that controls whether it can be published:
| Tag | Meaning |
|---|---|
| public_record | Official government document, board resolution, budget filing, or public meeting minutes. No permission required to cite the record. |
| link_quote | Third-party article or publication. A named speaker is attributed; the source URL links directly to the original article. |
Entries with any other rights tag are blocked from publication by a pre-publish gate that runs automatically before any profile rebuild is pushed live.
Rows marked for internal reporter use stay in the reporter table but are filtered before public rendering. Source paths are only linked when they are public http:// or https:// URLs; local warehouse paths are not rendered as links.
Confidence
| Level | Meaning |
|---|---|
| high | Public record or directly attributed source note that can be checked against the cited record. |
| medium | Useful attributed reporting or meeting-transcript fragment, but still needs closer review before it carries a hard budget claim. |
| low | Held for reporter context only unless upgraded with stronger sourcing. |
Currently covered districts
The first district covered is South Orange-Maplewood (SOMSD), with 18 rights-checked rows on budget structure, board-meeting claims, arts-funding cuts, and selected state-data context. Coverage will expand as additional districts are researched.
Budget-report use: The "Local public-record context" panel appears only when budget-relevant notes exist. It selects one strongest note per record thread and explains how that note should be checked against UFB evidence. It is not a full transcript, complete cut list, audited forecast, or independent verification beyond the cited source.
Source attribution format
Each entry on a district profile page shows:
- a metadata line (date, source type, editorial thread, confidence level)
- the attributed text or summarised claim
- the speaker's name and role (if the statement is attributed to an individual)
- a direct link to the primary source document or article when the URL is public
The issue registry
Every dataset behind this site keeps a structured registry of its known issues — the format breaks, suppression rules, entry errors, and definitional traps we hit while building on it — so that the next person (or agent) who works with this data doesn't rediscover them the hard way. Each issue has a stable id, a machine-readable scope (which years, columns, and tables it touches), and an effect: breaks stops a pipeline, corrupts silently wrongs the numbers, misleads invites a wrong reading of right numbers, context is background you must hold to use the data responsibly. Issues marked ★ are core: read them before any use of this data. The registry is maintained in the ergo format and served in machine-readable form alongside this page — links at the end of this section.
njschooldata (curated from public records) · 2026 (dates of the underlying statements/documents) · source confidence B · updated 2026-07-10 · 6 known issues (1 core)
The pitfall: Only rows with rights = public_record or link_quote may publish — anything else is rejected at build time, by the table's CHECK constraint, and by the check_notes_rights.py gate before any profile ships.
complementary-not-ufb-revision ★ [context · open] Notes render beside UFB budget figures on district pages but are not a revision to them
Type: definitional — applies to the whole dataset
The misread: Treating a quoted board statement or linked news claim as verified, audited budget data, or reading this layer's presence as a correction to NJDOE User-Friendly Budget figures — it is a separate, complementary evidentiary layer, a record-to-UFB bridge rather than an independent verification of UFB numbers.
Full entry, with the story and the numbers, in the served ergo doc.
invalid-rights-blocks-publish [breaks · mitigated] A rights value outside public_record/link_quote is rejected before it can publish
Type: coding — applies to tables: district_source_notes · columns: rights
How to spot it: A SEED row with rights not in {public_record, link_quote} raises ValueError in validate_seed() before any DB write; if one somehow reached the table anyway, the CHECK(rights IN ('public_record','link_quote')) constraint rejects the INSERT, and check_notes_rights.py exits non-zero at publish time, printing the offending row ids.
The misread: Assuming any rights value is fine as long as a source_url is present — rights is the publish gate itself, checked three separate times (SEED validation, table constraint, publish-time query), and only public_record and link_quote pass.
Full entry, with the story and the numbers, in the served ergo doc.
confidence-values-are-not-letters [corrupts · open] Loaded rows use confidence values 'high'/'medium', not the documented A/B/C letters
Type: coding — applies to tables: district_source_notes · columns: confidence
How to spot it: SELECT DISTINCT confidence FROM district_source_notes (or grep the SEED list) returns only 'high' and 'medium' for all 18 loaded SOMSD rows — never 'A', 'B', or 'C'.
The misread: Filtering or comparing district_source_notes.confidence against the page's documented A/B/C scale, e.g. a query for confidence = 'A' — it silently returns zero rows instead of erroring, because no loaded row uses a letter grade. the profile generator's confidence-rank lookups work around this by mapping both {high,medium,low} and {a,b,c} to the same rank; new code should not assume the documented scale is what's actually stored.
Full entry, with the story and the numbers, in the served ergo doc.
db-rebuilt-from-seed-each-run [context · mitigated] district_source_notes is dropped and rebuilt from the SEED list on every run
Type: revision — applies to the whole dataset
How to spot it: id (AUTOINCREMENT) resets and any hand-edited row disappears after the notes builder runs again — the build always does DROP TABLE IF EXISTS, then CREATE, then INSERT from the seed list.
The misread: Editing a row directly in the SQLite district_source_notes table (or in a copy of the warehouse file) instead of the curated seed list the builder loads from — the edit is silently discarded the next time the builder runs.
Full entry, with the story and the numbers, in the served ergo doc.
internal-only-marker-is-freetext [context · open] The internal_only flag is a substring inside quote_or_claim, not a column
Type: coverage — applies to tables: district_source_notes · rows: quote_or_claim rows containing the substring 'internal_only'
How to spot it: grep district_source_notes.quote_or_claim (or the SEED list) for the literal substring 'internal_only'. 1 of 18 loaded SOMSD rows carries it, in a parenthetical flagging parent reactions as internal_only.
The misread: Querying district_source_notes directly and treating every row as public-safe. The public profile and budget-report renderers (the district-notes module, the budget local-record panel, and the note-candidate filter in the profile generator) skip rows whose quote_or_claim contains 'internal_only', but the reporter source table keeps those rows, and nothing in the schema marks them structurally — rename or rephrase that parenthetical and the row silently becomes publishable.
Full entry, with the story and the numbers, in the served ergo doc.
source-url-non-http-not-linked [context · open] Local warehouse paths in source_url render without a link
Type: format — applies to tables: district_source_notes · columns: source_url
How to spot it: source_url does not start with http:// or https://. 3 of 18 loaded SOMSD rows (all source_type = state_data) carry the local path ed/data/kg/ed_kg.db; the other 15 are http(s) URLs (YouTube, Village Green).
The misread: Assuming every loaded note has a clickable public source. The public template’s public-URL test only linkifies values starting http:// or https://, so state_data rows citing the internal warehouse path render as an unlinked source line.
Full entry, with the story and the numbers, in the served ergo doc.
Changelog
- — Converted to ergo format: 6 issues registered — including the discovery that loaded confidence values are high/medium, not the documented A/B/C letters.
Machine-readable: this page as an ergo doc · all datasets (index.json) · format: ergo · implementation: source repo