# Data library - NJDOE User-Friendly Budgets

Reporter notebook for the district budget layer. This entry documents the source
shape and the assumptions behind the district profile budget summary and the
generated district budget report pages.

```toml ergo
[dataset]
ergo = "0.4"
slug = "ufb"
title = "NJDOE User-Friendly Budgets"
publisher = "New Jersey Department of Education, User Friendly Budgets"
subject = "https://www.nj.gov/education/budget/ufb/index.shtml"
source_url = "https://www.nj.gov/education/budget/ufb/index.shtml"
pitfall = "The per-pupil figures are NJDOE's own published comparative-cost calculation, not something to recompute from spending and enrollment — and every dollar column silently mixes actual, revised, original, and proposed budget bases."
status = "live"
confidence = "A"
updated = "2026-07-12"
implementation = "https://github.com/lavallee/njschooldata"

[dataset.coverage]
years = "2019-20 → 2025-26"
grain = "budget year × district (school code blank)"
entities = "NJ operating school districts that file a User-Friendly Budget"

[dataset.access]
keys = ["budget_year", "county_code", "district_code"]
```

## Source

- Publisher: New Jersey Department of Education, User Friendly Budgets
- Landing page: https://www.nj.gov/education/budget/ufb/index.shtml
- Current available years loaded: 2019-20 through 2025-26
- Warehouse tables (the names issue scopes refer to): `budget_revenue`,
  `budget_appropriation`, `budget_pupil_cost`, `budget_enrollment_ufb`,
  `budget_recap`, `budget_unusual`, `budget_shared_services`,
  `budget_tax_rate`, `budget_school_resource`, `budget_admin_salary`,
  `budget_summary`
- Public explainer: <https://njschooldata.fyi/data/user-friendly-budgets/>
- District-first budget entry: <https://njschooldata.fyi/budgets/>
- Full statewide budget workbench: <https://njschooldata.fyi/budgets/workbench/>
- District budget reports: `njschooldata.fyi/district/<slug>/budget/`
  wherever a generated district profile has a matching UFB row
- Inflation source for profile charts: BLS CPI-U, U.S. city average, not
  seasonally adjusted, annual averages through 2025

The NJDOE landing page describes the user-friendly budget as a summary that must
be posted by the district and by NJDOE after the public budget hearing and final
submission. The source includes revenues, appropriations, estimated tax-rate
detail, surplus, unusual revenues and appropriations, shared services, salary and
benefit data, and comparative per-pupil costs.

## Joins

Rows join to profile districts on the NJDOE district code. School code is blank
for district-level UFB files, so this source is district-first. Profile pages
compare each district with other districts in the same displayed county and, for
the comparison matrix, with all statewide UFB district rows that have a usable
value for the same latest-year measure.

## Issues

Every entry below would still be true if we rebuilt all of our tooling from
scratch: these are facts about NJDOE's files, not our rendering choices. The
methodology bullets that used to sit in the "Gotchas" section — how each panel,
layer, and chart reads the data — are preserved below under
[How our surfaces read this data](#how-our-surfaces-read-this-data), and they
link these issue ids inline where a choice rests on a dataset truth.

### CSV downloads are not all UTF-8

```toml ergo
[issue]
id = "cp1252-encoding-drift"
title = "Some UFB CSV downloads are not UTF-8 and break a naive decode"
effect = "breaks"
type = "format"
status = "mitigated"
detection = "raw.decode('utf-8') raises UnicodeDecodeError; the 2025-26 appropriation file decoded as Windows-1252"

[issue.scope]
all = true
```

The 2025-26 appropriation file decoded as Windows-1252 on the first pass.
`decode()` walks `utf-8-sig` then `cp1252` then `latin-1`, and only falls back
to a lossy replacement decode if all three raise.

### NJDOE renames the yearly CSV files

```toml ergo
[issue]
id = "filename-suffix-drift"
title = "CSV filenames use different year suffixes between years (rev26.csv vs rev25_minus_1.csv)"
effect = "breaks"
type = "availability"
status = "mitigated"
detection = "a hardcoded filename 404s; the same file family is rev26.csv one year and rev25_minus_1.csv another"

[issue.scope]
all = true
```

Recent years use slightly different suffixes for the same file family. The
loader discovers the year pages and download links from the NJDOE index and
matches each CSV by its dataset prefix (`rev`, `approp`, `pupcst`, `enroll`,
`recap`, `unusual`, `shared`, `esttax`, `schapp`, `employees`) instead of by a
year-suffixed name.

### Every dollar column mixes budget bases

```toml ergo
[issue]
id = "mixed-budget-bases"
title = "Actual-cost, original-budget, revised/current-budget, and proposed-budget columns are mixed within one file"
effect = "misleads"
core = true
type = "measurement"
status = "open"
detection = "revenue/appropriation amount_1..amount_3 and pupil_cost year_1..year_5 carry different budget bases across the same row"
misuse = "Reading a raw first-to-latest per-pupil or per-line rise as an audited year-over-year trend, or comparing a revised-budget figure against a proposed-budget figure as if they shared a basis."

[issue.scope]
tables = ["budget_revenue", "budget_appropriation", "budget_pupil_cost"]
years = "all"
```

The UFB time columns are not one consistent basis: the earlier columns are
actual or original/revised budget and the latest is the proposed budget. This is
why the loaded-year trend surfaces and the latest-file proposal surfaces are kept
separate rather than collapsed into a single bridge.

### Comparative cost per pupil is a published NJDOE calculation

```toml ergo
[issue]
id = "do-not-recompute-comparative-cost"
title = "Comparative cost per pupil is NJDOE's own calculation; recomputing it gives a different number"
effect = "misleads"
core = true
type = "measurement"
status = "open"
detection = "a total-spending / enrollment quotient disagrees with the published comparative-cost line"
misuse = "Dividing total spending by enrollment to get 'cost per pupil' — the result is not NJDOE's comparative cost and will not match the reader-facing published value."

[issue.scope]
tables = ["budget_pupil_cost", "budget_summary"]
columns = ["comparative_cost*", "Total Budgetary Comparative Per Pupil Cost"]
years = "all"
```

NJDOE publishes a specific comparative-cost-per-pupil calculation and the source
PDF confirms those are the reader-facing values. The summary builder carries the
published `budget_pupil_cost` row `Total Budgetary Comparative Per Pupil Cost`
as-is and converts only to 2025 dollars for charts; it never recomputes from
spending and enrollment.

### Rows with $0 comparative cost and 0 enrollment

```toml ergo
[issue]
id = "zero-comparative-cost-zero-enrollment"
title = "A few rows publish $0 comparative cost per pupil with 0 estimated on-roll"
effect = "corrupts"
type = "entry"
status = "mitigated"
detection = "comparative_cost_per_pupil <= 0 alongside estimated_onroll = 0"
misuse = "Treating $0 as a real value produces false 'lowest cost' ranks and '$0 per pupil' headlines."

[issue.scope]
tables = ["budget_summary"]
columns = ["comparative_cost*"]
```

A small number of UFB rows publish `0` comparative cost per pupil with `0`
estimated on-roll enrollment. `positive()` maps all five comparative-cost timing
columns to missing when nonpositive, so those rows never enter a ranking or a
headline.

### Revenue and appropriation categories are fund-specific

```toml ergo
[issue]
id = "fund-specific-categories"
title = "Revenue and appropriation rows belong to specific funds, not one all-funds ledger"
effect = "misleads"
type = "definitional"
status = "open"
misuse = "Summing operating, special-revenue, and debt-service rows into one mix, or labeling an operating revenue mix as an all-funds mix."

[issue.scope]
tables = ["budget_revenue", "budget_appropriation"]
```

Each revenue or appropriation row belongs to a specific fund (operating, special
revenue, debt service). Rows across funds are not one additive statement; the
all-funds/outside-operating rows share the operating denominator only as a scale
reference.

### Subtotal and detail rows overlap

```toml ergo
[issue]
id = "overlapping-subtotal-detail-rows"
title = "Subtotal and detail rows overlap and do not add up to a clean total"
effect = "misleads"
type = "definitional"
status = "open"
detection = "detail lines exceed their parent subtotal because they are nested inside it"
misuse = "Stacking lower-level lines such as benefits, transportation, or plant operations as a full spending mix, or summing named source rows into one total."

[issue.scope]
tables = ["budget_revenue", "budget_appropriation", "budget_pupil_cost"]
```

General current expense (`72260`) already contains lower-level lines such as
benefits, transportation, and plant operations; `budget_pupil_cost` rows likewise
include subcomponents of broader totals. Our tables drop outer totals and known
duplicate subrows and present the rest as reader context, never as one additive
schedule.

### UFB planning enrollment is not fall enrollment

```toml ergo
[issue]
id = "ufb-planning-vs-fall-enrollment"
title = "UFB planning enrollment is the budget denominator and differs from fall enrollment"
effect = "misleads"
core = true
type = "definitional"
status = "open"
detection = "budget_enrollment_ufb on-roll (lines 11 + 21) differs from the same-year district_enrollment fall count"
misuse = "Using fall enrollment to recompute UFB per-pupil values, or reading UFB on-roll as the district's fall headcount."

[issue.scope]
tables = ["budget_enrollment_ufb", "budget_summary"]
```

The UFB files carry a planning enrollment (regular on-roll plus special-education
on-roll) used as the budget denominator. It is not the fall enrollment reported
elsewhere and is not the denominator behind NJDOE's published comparative cost;
fall-enrollment joins are kept for coverage and student-need context only.

### Sent and received enrollment rows overlap

```toml ergo
[issue]
id = "sent-received-overlap"
title = "Enrollment file's sent and received rows are not mutually exclusive population categories"
effect = "misleads"
type = "definitional"
status = "open"
misuse = "Adding sent, received, private-placement, contract-preschool, and state-facility counts into one population total — they overlap and double-count."

[issue.scope]
tables = ["budget_enrollment_ufb"]
```

Sent and received placement rows can overlap each other and with tuition,
transportation, special-services, preschool, and receiving-district revenue
questions. Treat each as a separate service-load signal, not a slice of one
population.

### Estimated tax rates are per $100, sliced, and sometimes implausible

```toml ergo
[issue]
id = "tax-rate-slices"
title = "Estimated tax rates are per $100 of valuation, come in regional/municipal slices, and include implausible rows"
effect = "misleads"
type = "definitional"
status = "mitigated"
detection = "budget_tax_rate rows over 20 per $100; regional districts carry multiple municipality rows (muniflag)"
misuse = "Reading a rate as a tax bill, summing slice rows into a district levy, or averaging an implausible source row into a district-wide rate."

[issue.scope]
tables = ["budget_tax_rate"]
```

Rates are per $100 of valuation, and regional districts publish one row per
municipality; the surfaces keep those slices rather than flattening them into a
single district-wide rate. `tax_effort_context.sane_rate` drops rates over 20 per
$100 as implausible before statewide comparison, and levy per UFB on-roll comes
from the summary tax levy, not from summing slice rows.

### The operating tax-levy row is renamed across years

```toml ergo
[issue]
id = "tax-levy-label-drift"
title = "The operating tax levy appears under different labels across years (Local Tax Levy / Local Tax Levy-Base Budget / Total Tax Levy)"
effect = "corrupts"
type = "coding"
status = "mitigated"
detection = "the operating levy on revenue lines 100/114 changes label between years"
misuse = "A label-keyed trend renders the levy as disappearing one year and reappearing under a new name."

[issue.scope]
tables = ["budget_revenue"]
rows = "operating tax-levy rows on revenue lines 100 and 114"
```

`canonical_line` collapses revenue lines `100`/`114` to a single `Total Tax Levy`
key (taking the max across variants) so the operating levy stays continuous across
the label drift; the profile generator aliases `Local Tax Levy`, `Local Tax
Levy-Base Budget`, and `Total Tax Levy` the same way.

### Enrollment line 31 has blank labels

```toml ergo
[issue]
id = "line31-blank-labels"
title = "Enrollment line 31 (post-secondary, other) is sparse and carries blank labels in loaded CSVs"
effect = "corrupts"
type = "format"
status = "mitigated"
detection = "budget_enrollment_ufb rows with line_no = 31 have an empty line_desc"
misuse = "Keying enrollment rows by label drops line 31, so post-secondary counts only line 30 and undercounts."

[issue.scope]
tables = ["budget_enrollment_ufb"]
rows = "line_no = 31"
```

The loaders key enrollment rows by line number, so line 31 survives despite blank
labels; the explorer combines post-secondary line 30 with the sparse line 31
rather than relying on the missing label.

### Recap amount_4 is a projected balance, not spending

```toml ergo
[issue]
id = "recap-amount4-close-balance"
title = "Recap amount_4 is the projected close-of-year balance, not an appropriation"
effect = "misleads"
type = "definitional"
status = "open"
misuse = "Stacking recap amount_4 with current spending, or reading it as spendable cash — it is a projected balance, not an operating use."

[issue.scope]
tables = ["budget_recap"]
columns = ["amount_4"]
```

The balance/reserve surfaces read only `amount_4`, the estimated balance at the
close of the budget year, and group recognizable reserve labels. It is
fiscal-cushion context, not an operating-use chart or an audit of spendable cash.

### The side files are context, not replacement totals

```toml ergo
[issue]
id = "side-files-not-totals"
title = "The recap/esttax/unusual/shared/schapp/employees files are reader context, not replacement totals"
effect = "context"
type = "definitional"
status = "open"
misuse = "Adding side-file rows to the operating budget, or double-counting an unusual-item row that also appears as a named revenue or appropriation line in the main files."

[issue.scope]
tables = ["budget_recap", "budget_unusual", "budget_shared_services", "budget_tax_rate", "budget_school_resource", "budget_admin_salary"]
```

Each side file explains one thing — `recap` projected balances, `esttax`
tax-rate estimates, `unusual` one-off rows, `shared` shared-service arrangements,
`schapp` school-based budgeting statements, `employees` selected administrative
salaries. Unusual rows in particular can duplicate named state-aid, loan, or
appropriation lines already counted in the main revenue and appropriation files.

### Shared-service savings are self-reported

```toml ergo
[issue]
id = "shared-service-savings-self-reported"
title = "Shared-service amount_saved is district-reported, not an audited saving"
effect = "context"
type = "definitional"
status = "open"
misuse = "Treating reported shared-service savings as audited offsets to spending."

[issue.scope]
tables = ["budget_shared_services"]
columns = ["amount_saved"]
```

Districts optionally enter a positive `amount_saved` for a shared-service
arrangement. It is reported context, not an audited reduction, and is never netted
against operating totals.

### Administrative salaries are self-reported and mix units

```toml ergo
[issue]
id = "salary-self-reported"
title = "Administrative salary/benefit rows are self-reported and mix incompatible units"
effect = "context"
type = "definitional"
status = "open"
misuse = "Summing the employees file into a payroll or total-compensation figure — it mixes dollars, days, FTE, shared-role flags, and free text — or treating the numbers as NJDOE-verified."

[issue.scope]
tables = ["budget_admin_salary"]
```

NJDOE's source page cautions that the department does not approve or take
responsibility for the accuracy of the salary figures. The `employees` file also
mixes dollars, work/leave days, FTE, shared-role flags, and unnormalized text, so
the surfaces show selected listed base-salary rows as context and deliberately
avoid a grand compensation total.

### The salary file's district-code header changed across years

```toml ergo
[issue]
id = "admin-salary-header-drift"
title = "The administrative salary files changed the district-code header (district_id vs EMP_DISTRICT_ID)"
effect = "corrupts"
type = "format"
status = "mitigated"
detection = "older employees files carry EMP_DISTRICT_ID where newer files carry district_id"
misuse = "Keying on one header name silently drops the years that use the other, so those salary rows never join."

[issue.scope]
tables = ["budget_admin_salary"]
years = "all"
```

`district_fields` reads both `district_id` and the older `emp_district_id` into
the same district-code field (and `field()` normalizes header case and spacing)
before the join, so a header rename does not orphan a year's salary rows.

### Not every district-profile entity has a UFB row

```toml ergo
[issue]
id = "ufb-coverage-gaps"
title = "Some district-profile entities have no matching UFB row at all"
effect = "context"
type = "coverage"
status = "open"
misuse = "Reading a district's absence from the UFB files as $0 or lowest cost rather than as unreported — charters, special-services districts, and education services commissions may not file a district UFB."

[issue.scope]
entities = "charter, special-services, and education-services-commission profiles without a matching UFB district row"
```

UFB files are district-first (school code blank) and cover operating districts
that file a User-Friendly Budget. Profiles without a matching UFB row — for
example the Essex County Educational Services Commission, the missing 2025-26
comparative-cost row — simply do not render a budget section. The 2025-26 file has
595 UFB district rows, 566 with usable positive comparative cost per pupil.

### Fiscally-stressed districts get omitted from the statewide files

```toml ergo
[issue]
id = "omitted-distress-districts"
title = "NJDOE omits some fiscally-stressed districts from the statewide UFB revenue/recap files"
effect = "breaks"
type = "coverage"
status = "mitigated"
detection = "a district present in the tax-rate and per-pupil files is absent from rev/recap/enrollment for the same year; among 250 enrollment-1500+ districts only Lakewood and Jackson had gaps"
misuse = "Treating a missing district-year as no data or as $0, when a NJDOE per-district UFB PDF or the audited ACFR carries the figures."

[issue.scope]
entities = "Lakewood Twp (2520) FY2023-24 + FY2024-25; Jackson Twp (2360) FY2023-24"
```

The districts under the most fiscal stress are exactly the ones NJDOE leaves out
of the statewide CSVs — a systematic, not random, gap. Lakewood operates under a
state fiscal monitor and finances operations through annual DOE loans against
state aid; NJDOE explicitly does not post a FY2024-25 UFB for it ("not being
posted; contact the Public Information Office"). Jackson was in aid-loss turmoil.
`backfill_budget_gaps.py` fills these three district-years from Grade-A primary
sources — the NJDOE per-district UFB PDFs for the 2023-24 rows (budgeted basis,
matching the statewide data), and the audited ACFR (Exhibit C-1) for Lakewood
FY2024-25 (audited-actual basis, since no budget was published). Every backfilled
district-year and its basis/source is recorded in the `budget_backfill` table and
surfaced on the report so an audited-basis row is never silently mixed with
budgeted ones.

### The DOE loan against state aid is booked as state revenue, but it is debt

```toml ergo
[issue]
id = "doe-loan-in-state-sources"
title = "The Department of Education Loan Against State Aid sits inside Total State Sources but is debt, not aid"
effect = "misleads"
type = "definitional"
status = "mitigated"
detection = "acct 10-3199 (line 495) is a component of Total Revenues from State Sources (line 520); Lakewood's stored state revenue swings $34M-$147M year to year, tracking the loan, not its aid"
misuse = "Counting the loan as state aid inflates a district's state share and injects spurious volatility. Lakewood reads as 48% state-funded in 2025-26 with the loan, ~23% without it."

[issue.scope]
entities = "districts drawing a DOE loan against state aid — chronically Lakewood (2520); occasionally one or two others"
```

The builder captures the loan separately in `budget_summary.doe_loan_against_aid`
(from line 495). `state_operating_revenue` keeps NJDOE's total (loan included) for
fidelity to the source, but the budget-pressure signals net the loan out of both
the numerator (state aid) and the denominator (operating budget), so aid-share and
local-share measure real revenue composition rather than debt financing. This is
material for Lakewood only among the tracked districts.

## Validation

```toml ergo
[validation]
date = "2026-07-10"
method = "confirmed from real data — 2025-26 UFB warehouse load"
result = "595 UFB district rows in 2025-26, 566 with usable positive comparative cost per pupil; 738 district profile pages rebuilt, 587 with a budget report; statewide operating budget per UFB on-roll $24,735 (2019-20) rose to $25,851 (2025-26) after CPI-U adjustment"
```

```toml ergo
[validation]
date = "2026-07-10"
method = "cross-checked the published comparative-cost line against the NJDOE source PDF"
result = "the PDF confirms the published comparative-cost-per-pupil values are the reader-facing figures; the summary builder carries them as published rather than recomputing (ufb/do-not-recompute-comparative-cost)"
```

The reconciliation figures above are the same ones carried in the Storyline Leads
section; re-pull and re-record here when the warehouse is rebuilt.

## Changelog

```toml ergo
[change]
date = "2026-07-10"
note = "Converted to ergo format: 18 issues registered; the ~50-bullet gotcha list sorted into dataset issues vs. surface-methodology narrative with zero bullets lost."
```

```toml ergo
[change]
date = "2026-07-12"
note = "Two issues registered for the budget-pressure-signals report: omitted-distress-districts (Lakewood/Jackson backfilled from primary sources via backfill_budget_gaps.py) and doe-loan-in-state-sources (new budget_summary.doe_loan_against_aid column; signals net the DOE loan out of aid/local shares)."
```