# NJ ESEA Flexibility waiver accountability profiles, 2012–2016

```toml ergo
[dataset]
ergo = "0.4"
slug = "esea-waiver-accountability-profiles"
title = "New Jersey ESEA Flexibility waiver accountability profiles, 2012 to 2016"
publisher = "NJ Department of Education, Office of Title I"
subject = "https://www.nj.gov/education/title1/accountability/"
source_url = "https://www.nj.gov/education/title1/accountability/"
pitfall = "YES* never means the school met its published target. Through 2014 it marks a school that missed it and was credited within a 95 percent confidence interval; from 2015 it is a participation determination, in a different column, in editions that publish no performance target at all."
status = "live"
confidence = "A"
updated = "2026-07-30"
implementation = "https://github.com/lavallee/njschooldata"

[dataset.coverage]
period = "Five annual editions, 2012 through 2016. Nothing before 2012 and nothing between 2017 and 2023 exists at this path; the ESSA calculation workbooks begin with 2024-25."
grain = "academic determination = entity x edition x subject x student group; graduation determination = entity x edition x student group; attendance determination = entity x edition, never by student group"
entities = "2,325 school CDS codes and 679 district CDS codes across the five editions; 2,234 schools and 645 districts resolve to a current njschooldata slug"

[dataset.access]
keys = ["edition_year", "entity_level", "district_code", "school_code", "subject_conformed", "subgroup_reported"]

[dataset.missingness]
zero_is_missing = false
source_tokens = ["-", "--"]
```

Five annual archives holding the determinations New Jersey made under its ESEA
Flexibility waiver. Per school and per district, one row per student group per
subject: how many were enrolled, how many were not tested, whether participation
was met, how many valid scores there were, what share cleared the bar, the bar
the state set, and the state's determination. Beside them, per student group,
whether the state credited the entity on graduation rate, and per entity whether
it met the attendance bar.

It is the only place a reader can see **how the state judged this school before
ESSA** — and it is five different documents wearing one name.

The producer's page is the fuller registry: Aquifer's
`docs/data-sources/nj-esea-waiver-accountability-profiles.md` documents the
parse, the layout forms, the target trajectories and the acquisition. **The issue
ids on this page are that page's ids**, so an id resolves to the same fact in
either repo. What is registered below is the subset that changes what a reader
sees on a profile page.

## Consumer question and boundary

> For each New Jersey school and district, what determination did the state
> publish in each of the five editions 2012 to 2016 — per student group and
> subject on participation and, through 2014 only, on an individually set
> progress target; per student group on graduation rate; and per entity on
> attendance — without turning five differently-defined editions into one score?

Aquifer owns acquisition, the coordinate-bound parse of the Crystal Reports
PDFs, the layout forms, the per-edition semantics, the identity join, the
publisher-defect register and the immutable release. This repo owns the narrower
profile-page question, the eight-table consumer mart, the exact-code join, the
regime-band rendering and the rollback.

`progress_targets` — 1,017,348 rows of per-year target trajectories — is
deliberately **not** projected. The target a determination was made against is
already on the determination row, and no profile page asks what the 2016 target
would have been had one been set.

## Exact offline pin and mart

The committed pin is
`contracts/aquifer/nj-esea-waiver-accountability-profiles.lock.json`:

- release: `esea-waiver-2012-2016-aq042-indicator-block-v2`;
- manifest SHA-256:
  `54cb52300649045ae92cc13202be773a6bba49cfee1e7fa074e19beca0d05778`;
- SQLite SHA-256:
  `a1fb6afa3c5900f473f996b284d1e88c1e1832831c7131c4caee80097bf7c560`;
- byte count: `854,716,416`.

`tools/build_aquifer_esea_waiver.py` verifies the whole artifact before it
writes anything, then fail-closes on the source populations, the per-edition
determination vocabulary and state histogram, the asterisk column per edition,
the graduation bars and cohort years per edition and level, the attendance
grain, the identity histogram, the edition spread of every entity, and the
share of 2012–2014 school rows that carry a determination at all. Any of those
moving stops the build rather than changing a page.

| Consumer table | Rows | Deterministic row SHA-256 |
| --- | ---: | --- |
| `esea_waiver_academic_determination` | 288,160 | `967b6c8588e66784a6161f73b8d34fc049e645e31047f7f91987de18212a9896` |
| `esea_waiver_graduation_determination` | 115,630 | `d024805664bf73ff27f1214ab80eda42bab3b75513e568a49d9f87bd3f4a2134` |
| `esea_waiver_attendance_determination` | 9,167 | `eaf3ef63d6f5a661574df5c92c5b39bffb773b3c3cce08544cfeae3d209fb7a1` |
| `esea_waiver_year_semantics` | 5 | `764223dcdfa5cfd4b8645153b6ecc3b5e54fd7235751356de6ce8f501b57c3f5` |
| `esea_waiver_indicator_semantics` | 10 | `6bfe7a7057a7649ddfaf63f9c0ae8fed48f6764049d56465425797d3a680543a` |
| `esea_waiver_subgroup_vocabulary` | 50 | `0f01bc795587fea14211d3d14a96d9b31859856f41c6cd5a75753af5b1235747` |
| `esea_waiver_absent_year` | 3 | `2ce4e4b434697b4d8976b5dca5a09c68b8d5eabf22a7f04007cff4512d05c090` |
| `esea_waiver_publisher_defect` | 2,555 | `df7d2a84fc43b200d04570492cce13683fcf7b373ffad74e7c242859a526a70e` |

The four semantic tables are the reason the module can tell one kind of absence
from another. Drop them and every blank cell on the page looks like the same
blank cell.

## Issues

### 2015 and 2016 set no performance target at all

```toml ergo
[issue]
id = "no-performance-target-after-2014"
title = "2015 and 2016 publish no progress target and no performance determination, for every school in New Jersey"
effect = "breaks"
core = true
type = "coverage"
status = "mitigated"
detection = "All 116,040 rows in the 2015 and 2016 editions carry met_performance_state = 'not-published-this-edition' and target_state = 'not-published-this-edition', and year_semantics.publishes_performance_target is 0 for both. The 2012–2014 editions print seven columns; these print five."
misuse = "Rendering an empty performance column for 2015 and 2016, which reads as missing data about this school rather than as a decision the state made about the whole state. It is the most damaging misreading available on this page, and it covers two of the five years."

[issue.scope]
years = ["2015", "2016"]
tables = ["esea_waiver_academic_determination", "esea_waiver_year_semantics"]
columns = ["met_performance_state", "target_state", "performance_measure"]
```

The measure changes too: `% Proficient` on NJASK and HSPA becomes `% Meeting
Standards` on PARCC, defined on the profile page as performance levels 4 and 5.

The module renders those two editions with **no performance column at all** and
a sentence above them stating that New Jersey set no target and published no
determination — not for this entity and not for any school in the state. The
adapter fails the build if a single 2015 or 2016 performance determination ever
appears.

### `YES*` does not mean the school met its target

```toml ergo
[issue]
id = "the-asterisk-is-not-met"
title = "Through 2014, YES* marks a school that MISSED its target and was credited within a 95 percent confidence interval"
effect = "misleads"
core = true
type = "coding"
status = "mitigated"
detection = "24,894 rows across 2012–2014 carry met_performance_state = 'met-within-95-percent-confidence-interval'. 24,891 sit below the target printed beside them; none sits at or above it; three print no target at all. Each edition's key.pdf says 'Safe harbor no longer applies. A 95% confidence interval (CI) is applied to all proficiency measures ... as additional criteria in determining meeting Progress Targets.'"
misuse = "Rendering YES* as 'met', which reports that a group cleared a bar the record says it missed — and does so about exactly the group the finding is wrong about."

[issue.scope]
years = ["2012", "2013", "2014"]
tables = ["esea_waiver_academic_determination", "esea_waiver_year_semantics"]
columns = ["met_performance_literal", "met_performance_state", "determination_qualifier"]
```

In 2014 Emma C Attales in Absecon reads 77.0% proficient against a 79.7% target
for its schoolwide group, and the printed determination is `YES*`. The school
missed the target.

The module writes the cell as **“Missed its target, credited within a 95%
confidence interval”**, with the publisher's own `YES*` beside it, and the words
are generated from that edition's recorded `asterisk_meaning` rather than from a
constant.

### The asterisk changes column, and meaning, in 2015

```toml ergo
[issue]
id = "the-asterisk-changes-column-in-2015"
title = "From 2015 the asterisk moves to the participation column and means participation averaging, not a confidence-interval credit"
effect = "misleads"
core = true
type = "coding"
status = "mitigated"
detection = "year_semantics.asterisk_column is 'met_performance' for 2012–2014 and 'met_participation' for 2015–2016. 9,826 rows in the later editions carry met_participation_state = 'met-with-participation-averaging'; the printed legend on every page reads 'YES* Met Participation Rate (Participation Averaging applied)'."
misuse = "Carrying one asterisk meaning across the archive, which reports a 2015 school as having met a performance bar the state never set that year."

[issue.scope]
years = ["2015", "2016"]
tables = ["esea_waiver_academic_determination", "esea_waiver_year_semantics"]
columns = ["met_participation_literal", "met_participation_state", "asterisk_column"]
```

The renderer resolves the asterisk against the edition's recorded column. Where
the recorded column and the column a value arrived in disagree, it prints the
publisher's literal and claims nothing about it rather than borrowing another
edition's meaning.

### The double dash is a determination that appears in no legend

```toml ergo
[issue]
id = "the-double-dash-is-not-in-the-legend"
title = "A sixth token, --, appears in the performance column and is documented nowhere"
effect = "corrupts"
type = "coding"
status = "mitigated"
detection = "2,178 rows across 2012–2014 print `--` in the performance column, typed not-determined-no-progress-target-published, each with a filed defect row. Every page's legend covers MET GOAL, YES, YES*, NO and the single dash, and none covers this one. It always co-occurs with a printed target of `-` on a group at or above that edition's proficiency minimum of 30."
misuse = "Normalising `--` to `-`, which conflates a group large enough to score for which no target was ever set with a group too small to determine. Those are opposite facts about the same school."

[issue.scope]
years = ["2012", "2013", "2014"]
tables = ["esea_waiver_academic_determination", "esea_waiver_publisher_defect"]
columns = ["met_performance_literal", "met_performance_state"]
```

The module writes `--` as **“Large enough to score, but no target was ever set”**
and `-` as **“Too few students to determine”**, in different type, with the
publisher's glyph beside each. They are never the same cell.

### Graduation and attendance are a third table, on their own grains

```toml ergo
[issue]
id = "graduation-and-attendance-block-not-captured"
title = "Each profile carries a third table: graduation per student group and attendance per entity, printed in four of the five editions"
effect = "misleads"
type = "coverage"
status = "mitigated"
detection = "indicator_semantics records block_state = 'printed' for 2013–2016 at both levels and 'not-printed-this-edition' for 2012 at both. The four-year bar moves — 75% in 2013, 78% in 2014 and 2015, 81% in 2016 — against a flat 85% five-year and 90% attendance, and every cohort is lagged behind the edition that judges it. attendance_determinations carries grain = 'entity' on all 9,167 rows and has no student-group column."
misuse = "Reading the academic tables as the whole of what the state judged. For 2015 and 2016 that is the sharpest error: those editions publish no performance determination, so this block is the only outcome judgment New Jersey printed. The second error is slicing attendance by student group — the publisher prints it once per page, off the group grid, and it was never disaggregated."

[issue.scope]
years = ["2013", "2014", "2015", "2016"]
tables = ["esea_waiver_graduation_determination", "esea_waiver_attendance_determination", "esea_waiver_indicator_semantics"]
```

`OR` is the publisher's operator and is printed on the page: a group that
cleared the four-year bar **or** the five-year bar was credited on the
indicator. All three columns are carried separately and the indicator is never
derived from the two inputs, because which route the state credited is the thing
worth seeing. Columbia High School's 2015 row for Students with Disabilities is
the case — four-year `NO`, five-year `YES`, indicator `YES`.

2013's district profiles are a different table: per-group graduation **rates as
numbers**, no five-year column, no attendance, and the printed note *“5yr Cohort
Grad rate and Attendance rate will be included in Final District Profiles.”* No
final 2013 district profile was ever published. Those rates render in a numeric
column with that note attached and are never coerced into a determination
vocabulary.

### Preliminary is not final, and 2013 is both

```toml ergo
[issue]
id = "preliminary-is-not-final"
title = "Three editions are labelled preliminary, one final and one unmarked — and in 2013 the school profiles are final while the district profiles are preliminary"
effect = "misleads"
type = "revision"
status = "mitigated"
detection = "year_semantics carries school_determination_stage and district_determination_stage separately, and they differ in 2013 (final / preliminary). Every determination row carries its own determination_stage."
misuse = "Building a timeline that mixes preliminary and final determinations without saying which is which, or reading the 2014 unmarked edition as final. No final edition of 2012, 2015 or 2016 was ever published."

[issue.scope]
tables = ["esea_waiver_academic_determination", "esea_waiver_year_semantics"]
columns = ["determination_stage"]
```

Each year section states its own stage, from the row's own value, so a district
page can say *preliminary* for 2013 while the school pages in it say *final*.
Unmarked is kept as a third state: 2014 asserts nothing about itself and neither
does the page.

### The student-group labels drift, and one is renamed

```toml ergo
[issue]
id = "subgroup-labels-drift"
title = "2012 through 2015 print Limited English Proficiency; 2016 prints English Language Learners"
effect = "misleads"
type = "definitional"
status = "mitigated"
detection = "subgroup_vocabulary carries eleven printed literals across the archive against ten conformed keys, with renamed_from = 'Limited English Proficiency' on the 2016 english-learners rows."
misuse = "Presenting Limited English Proficiency and English Language Learners as one continuously defined group because they occupy the same row of the table. Neither label is 'Multilingual Learners', whatever the overlap in who is counted."

[issue.scope]
years = ["2016"]
tables = ["esea_waiver_subgroup_vocabulary", "esea_waiver_academic_determination"]
columns = ["subgroup_reported", "renamed_from"]
```

Every row keeps its own edition's printed label, and the 2016 table marks the
renamed row inline so a reader moving between the 2015 and 2016 sections sees a
rename rather than a smooth series. The printed subject label is treated the
same way — *Language Arts Literacy* in 2012, *English Language Arts* from 2013,
*English Language Arts/Literacy* in 2016 — and heads its own edition's column.

### An unmatched 2012–2016 entity is a correct answer

```toml ergo
[issue]
id = "many-entities-no-longer-exist"
title = "The CDS codes are 2012–2016 entities and the identity release is a current snapshot, so a real share of rows will never join"
effect = "context"
type = "identity"
status = "mitigated"
detection = "8,080 of 288,160 determination rows (2.8%) carry identity_match_state 'unmatched-cds' or 'unmatched-district'. 2,143 of 2,325 school CDS codes appear in all five editions; 182 do not. 2,234 school codes and 645 district codes resolve to a current njschooldata slug."
misuse = "Name-matching an unmatched historical school into a present-day organization to raise the join rate, or reading a missing edition as a data gap rather than as a school that did not exist, had no profile, or changed code that year."

[issue.scope]
tables = ["esea_waiver_academic_determination", "esea_waiver_graduation_determination", "esea_waiver_attendance_determination"]
columns = ["identity_match_state", "njdoe_district_code", "njdoe_school_code"]
```

One trap is structural: these profiles use school code **888** as the
district-level sentinel where the NJDOE code list uses **000**, so a district row
joins on its county and district codes and never on the CDS the page prints.

The loader takes only rows the release verified. A page renders the editions its
entity appears in and says nothing at all about the ones it does not — no
inferred year, no dash standing in for a school that was not there.

### The recoverable span is 2012 to 2016, and the boundary is checked

```toml ergo
[issue]
id = "the-recoverable-span-is-2012-to-2016"
title = "11.zip, 17.zip and 18.zip return 404, and the seven years between this record and the ESSA workbooks will not fill"
effect = "context"
type = "coverage"
status = "mitigated"
detection = "absent_year_observations records 2011, 2017 and 2018 as structural absences with http_status 404, probed on every acquisition. The ESSA accountability workbooks release this site holds begins with 2024-25."
misuse = "Rendering 2017–2023 as a skipped tick or an interpolated segment, or reading the absence as evidence that New Jersey judged nothing in those years. What is established here is that nothing at this path covers them and nothing this site holds does either."

[issue.scope]
years = ["2011", "2017", "2018"]
tables = ["esea_waiver_absent_year"]
```

The gap renders as its own band, between the 2024-25 score and the waiver
record, carrying the probe result and the reason. No axis, no scale and no mark
crosses it.

### Determinations printed against no target

```toml ergo
[issue]
id = "determinations-without-a-target"
title = "Some 2012 charter district rows carry a real determination against a printed target of `-`"
effect = "corrupts"
type = "entry"
status = "open"
detection = "359 rows are filed as determination-published-without-a-target and nine more as determination-below-the-minimum-group-size. Three are YES*, including one on a group with 0 enrolled and 0 valid scores; others are NO on groups well below the minimum size, such as a NO on six valid scores."
misuse = "Treating an asterisked determination as evidence a bar was cleared when the page prints no bar."

[issue.scope]
years = ["2012"]
tables = ["esea_waiver_publisher_defect", "esea_waiver_academic_determination"]
```

The publisher's intent is not recoverable from the archive, so this stays
**open**. The module prints the determination the state printed and prints no
target beside it, because there was none; a year section on an affected page
carries a note naming what NJDOE got wrong on it, and nothing is repaired.

### Producer-owned issues this page does not re-register

These are real and are documented on the producer's page under the same ids.
They are handled before the release this repo pins, and none of them changes
what a profile page renders:

| id | what it is |
| --- | --- |
| `five-years-five-layouts` | five printed forms, one per edition; a page whose title does not match its recorded form fails the release |
| `read-the-pdf-by-column-band` | values are bound to measured x-coordinate bands, not read by ordinal position |
| `the-2012-trajectory-clips-at-the-statewide-goal` | 148 of 72,118 reconciled target cells disagree, all 2012, all clipped at 90 |
| `no-key-pdf-after-2014` | 2015 and 2016 ship no `key.pdf`; the printed legend is the whole authority |
| `targets-are-absent-because-none-were-set` | `targets.zip` is missing from those two payloads because none were set |
| `negative-percent-not-tested` | two rows print a negative percent-not-tested; this consumer renders no such column |
| `entity-names-are-sometimes-not-printed` | a handful of pages identify their entity by CDS code alone |
| `target-tables-omit-subgroup-rows` | two target tables omit a student-group line entirely; the trajectories are not projected here |

## Practices

### Keep the regimes apart and draw the gap as a gap

```toml ergo
[practice]
id = "keep-the-two-regimes-apart"
title = "Keep the regimes apart and draw the gap as a gap"
question = "Can I put these years on one axis?"
authority = "project"
rule = "Present 2012–2014 and 2015–2016 as two bands, comparable within each and not across, put the 2017–2023 absence between them and the 2024-25 ESSA score as a third instrument, and carry the determination stage and the performance measure on every year so the discontinuity is visible rather than smoothed."
because = "There is no single comparable score across the waiver era and there never will be. The regime change is not noise — it is what the reader learns."
naive = "Deriving a composite score, or drawing a trend line, so the years fit on one chart."
```

Commission 043 put the form question to vizier and the answer was a table per
year in regime bands, with **no chart**. The cell values are qualitative labels
with no order: `YES*` is not a better `YES`, and `-` is a suppression rather than
a low value. A heat-grid would need the one palette vizier's own pattern text
rules out — qualitative multi-hue, which encodes no order — to shade a variable
with no magnitude over a grid that is 60% absence. Ten student groups all
carrying meaning is a table, not more colors. And across the 2017–2023 break the
documented benchmark is *no fair comparison available*, so the module says so
rather than drawing through it. The reasoning is recorded in `CHARTS.md` as
entry **P23**, a no-chart entry, so the next pass does not re-derive it.

### Ask what the glyph meant *that year*, from that year's own documents

```toml ergo
[practice]
id = "pin-the-glyph-per-edition"
title = "Ask what the glyph meant that year, from that year's own documents"
question = "What does this determination glyph mean in the edition it was printed in?"
authority = "publisher"
rule = "Resolve every determination glyph against the edition it appears in, using that edition's recorded column, meaning and authority, and generate the reader-facing words from those rather than from a lookup table shared across the archive."
because = "The same five characters encode a confidence-interval credit on a performance determination through 2014 and participation averaging on a participation determination from 2015. One constant would state a finding about a student group that the record does not support."
naive = "Building one glyph-to-meaning map for the whole archive because the glyphs are the same five characters."
```

## Static rendering

Both profile levels render the module, under **How the state has judged this
school** and **How the state has judged this district**. The section id stays
`accountability`; the school pages carry the 2024-25 ESSA band above the waiver
bands, and district pages carry the waiver bands alone, because the ESSA
calculation workbooks publish no district row.

- 2,234 of the 2,975 school rows in `data/entity_slugs.csv` match a verified
  waiver school record by exact `(district_code, school_code)`; 645 of the 744
  district rows match by district code. Exact-code nonmatches render no band
  rather than falling back to names.
- 2,143 school codes carry all five editions. 182 carry between one and four,
  and those pages render exactly the editions they have.
- Graduation determinations reach 386 to 418 school pages per edition and 297
  to 643 district pages; attendance reaches 7,411 school and 1,756 district
  entity rows in total. Elementary and middle schools carry attendance and no
  graduation; high schools carry graduation and, mostly, no attendance.

Columbia High School (`4900-030`) is the five-edition worked example: five year
sections in two bands, a 2015 Students-with-Disabilities row credited on the
five-year cohort after missing the four-year, a 2014 mathematics `YES*` for the
same group at 57.1% against a 64.4% target, and `MET GOAL` on four 2014 cells.
West Side High School in Newark (`3570-080`) appears in 2012, 2013 and 2014
only, renders those three, and renders neither a 2015–2016 band nor an absence
band, because there is nothing on the far side of the gap to bracket.

## Change log

```toml ergo
[change]
date = "2026-07-30"
note = "Adopted AQ-042 esea-waiver-2012-2016-aq042-indicator-block-v2 as an eight-table consumer mart and built the accountability timeline on it, per commission 043. Kept the four semantic tables beside the determinations because they are the only thing separating one kind of absence from another: 2015 and 2016 render with no performance column and a sentence saying the state published none, rather than an empty column that reads as missing data. The asterisk's words are generated per edition from the recorded semantics. Attendance stays at entity grain with no student-group column at any layer. Form ruled by vizier as a table per year in regime bands, no chart, recorded in CHARTS.md as P23."
```