NoticeForge - Road Reporter

Click any road. Instantly know who maintains it and how to report a hazard.

Priority coverage: Schenectady County + adjacent counties (Albany, Montgomery, Saratoga, Schoharie, Fulton). Full NYS RIS data loaded statewide.

Quick Start (local dev)

Prerequisites

Docker + Docker Compose (for PostGIS + ETL)
Node ≥ 18 + npm ≥ 9 (for frontend)
~20 GB free disk (RIS GDB + parcel data + PMTiles)

1 — Clone and configure

git clone https://github.com/your-org/nyroadreport.git
cd nyroadreport
cp .env.example .env

2 — Build ETL Docker image (~5 min first time; compiles tippecanoe)

make build-etl-image

3 — Run the full ETL pipeline

make run-etl

This will:

Start PostGIS in Docker
Download all authoritative data (RIS, parcels, NYPAD, PDFs)
Load to PostGIS, generate 75-metre road markers, spatial-join land info
Export roads.pmtiles, markers.pmtiles, land.pmtiles, contacts_lookup.json, and build_meta.json → frontend/public/data/

Expected time: 30–90 minutes (statewide data volume).
To test with a single county first, set a bounding box filter in etl/config.py (a DEBUG_BBOX variable can be added to 02_load_roads.py).

4 — Start the frontend dev server

make dev
# → http://localhost:5173

File Tree

nyroadreport/
├── .env.example
├── .gitignore
├── docker-compose.yml            # PostGIS + ETL containers
├── Makefile                      # Common dev commands
│
├── docker/
│   ├── Dockerfile.postgis        # PostGIS 16 image
│   └── Dockerfile.etl            # Ubuntu + GDAL + Python + tippecanoe
│
├── etl/
│   ├── config.py                 # All config / URLs / constants
│   ├── db_utils.py               # Shared DB helpers
│   ├── 00_schema.sql             # PostGIS schema (idempotent)
│   ├── 01_download.py            # Download + checksum all sources
│   ├── 02_load_roads.py          # RIS roads → road_segments
│   ├── 03_load_land.py           # Parcels + NYPAD → land_jurisdictions
│   ├── 04_parse_contacts.py      # PDF parsing → contacts (raw)
│   ├── 05_match_contacts.py      # Assign jurisdiction_keys to contacts
│   ├── 06_gen_markers.py         # road_markers @ 75m + land spatial join
│   ├── 07_export_pmtiles.py      # PostGIS → GeoJSONSeq → PMTiles
│   ├── 08_export_contacts_json.py # contacts_lookup.json + build_meta.json
│   ├── run_all.sh                # Full pipeline runner (inside container)
│   ├── requirements.txt
│   ├── data/
│   │   └── contact_overrides.json  ← EDIT THIS to patch bad contact matches
│   ├── downloads/                  (gitignored; cached raw downloads)
│   └── tests/
│       └── test_jurisdiction_keys.py
│
└── frontend/
    ├── index.html
    ├── package.json
    ├── tsconfig*.json
    ├── vite.config.ts
    ├── eslint.config.js
    ├── public/
    │   └── data/                   (gitignored; output of ETL)
    │       ├── roads.pmtiles
    │       ├── markers.pmtiles
    │       ├── land.pmtiles
    │       ├── contacts_lookup.json
    │       └── build_meta.json
    └── src/
        ├── main.tsx
        ├── App.tsx
        ├── store.ts                # Zustand state
        ├── types/index.ts          # Shared TS types
        ├── utils/
        │   ├── contacts.ts         # contacts_lookup.json loader + helpers
        │   └── nominatim.ts        # Address geocoding (rate-limited)
        ├── components/
        │   ├── Map.tsx             # MapLibre map, PMTiles sources, layer styling
        │   ├── Popup.tsx           # Jurisdiction + contact detail panel
        │   ├── BottomSheet.tsx     # Mobile bottom sheet / desktop side panel
        │   └── SearchBar.tsx       # Address search with Nominatim
        └── styles/
            └── index.css

Jurisdiction Key Strategy

Pattern	Example	Meaning
`RIS:<code>`	`RIS:04`	NYSDOT Region 4 (Capital)
`COUNTY:<fips5>`	`COUNTY:36093`	Schenectady County DPW
`MUNI:<fips5>:<TYPE>:<slug>`	`MUNI:36093:CITY:schenectady`	City of Schenectady
`STATE:DEC`	`STATE:DEC`	NYS DEC land
`STATE:OTHER:<slug>`	`STATE:OTHER:canal_corp`	Other state-owned
`PROTECTED:<id>`	`PROTECTED:12345`	NYPAD protected area

To patch a wrong match, edit etl/data/contact_overrides.json and commit.

Manual Contact Override Workflow

After running ETL, check frontend/public/data/build_meta.json → .unmatched_contacts — list of contacts with no jurisdiction_key.
Open etl/data/contact_overrides.json.

Add an entry:

{
  "COUNTY:36093": {
    "phone": "518-388-4300",
    "email": "dpw@schenectadycounty.com"
  }
}

Re-run only step 8: make run-etl-step STEP=etl/08_export_contacts_json.py
Commit the override file.

Production Deployment

Static frontend → Cloudflare Pages / Vercel / Netlify

make build-frontend
# dist/ → deploy to hosting provider

PMTiles → Cloudflare R2 (recommended)

# Install rclone or aws-cli configured for R2
aws s3 cp frontend/public/data/roads.pmtiles   s3://<bucket>/roads.pmtiles   --content-type application/octet-stream --cache-control "public, max-age=86400"
aws s3 cp frontend/public/data/markers.pmtiles s3://<bucket>/markers.pmtiles --content-type application/octet-stream --cache-control "public, max-age=86400"
aws s3 cp frontend/public/data/land.pmtiles    s3://<bucket>/land.pmtiles    --content-type application/octet-stream --cache-control "public, max-age=86400"
aws s3 cp frontend/public/data/contacts_lookup.json s3://<bucket>/contacts_lookup.json --content-type application/json --cache-control "public, max-age=86400"
aws s3 cp frontend/public/data/build_meta.json      s3://<bucket>/build_meta.json      --content-type application/json --cache-control "public, max-age=3600"

Set VITE_TILES_BASE_URL=https://your-r2-bucket.r2.dev before building.

R2 CORS policy (required for PMTiles range requests)

[{
  "AllowedOrigins": ["https://yourdomain.com"],
  "AllowedMethods": ["GET", "HEAD"],
  "AllowedHeaders": ["Range"],
  "ExposeHeaders": ["Content-Range", "Content-Length", "Accept-Ranges"],
  "MaxAgeSeconds": 86400
}]

Deployment

1 — Cloudflare Worker (hazard report endpoint)

cd worker
npm install

# Create KV namespaces (note the IDs printed and paste into wrangler.toml)
npx wrangler kv:namespace create REPORTS_KV
npx wrangler kv:namespace create REPORTS_KV --preview
npx wrangler kv:namespace create RATELIMIT_KV
npx wrangler kv:namespace create RATELIMIT_KV --preview

# Optionally: set ALLOWED_ORIGIN to your Pages domain to lock CORS
npx wrangler secret put ALLOWED_ORIGIN

# Deploy
npx wrangler deploy

Set VITE_REPORT_API_URL in your Pages environment variables (or .env) to the worker URL shown after wrangler deploy, e.g. https://nyroadreport-worker.YOUR-NAME.workers.dev.

2 — Frontend to Cloudflare Pages

cd frontend
npm install
npm run build          # output: frontend/dist/
# Then either:
npx wrangler pages deploy dist --project-name nyroadreport
# or connect the GitHub repo in the Cloudflare Pages dashboard.

Environment variables to set in Pages project settings:

Variable	Description
`VITE_TILES_BASE_URL`	Public R2/CDN URL for tile artifacts
`VITE_REPORT_API_URL`	Cloudflare Worker URL

3 — Tile artifacts to Cloudflare R2

# Install AWS CLI (used for S3-compat R2 access)
aws configure   # use R2 access key ID + secret

R2_ENDPOINT="https://<ACCOUNT_ID>.r2.cloudflarestorage.com"
aws s3 cp etl/data/artifacts/ s3://nyroadreport-tiles/tiles/ \
  --recursive --endpoint-url "$R2_ENDPOINT"

Then set the R2 bucket's CORS policy to allow GET from your Pages domain, and point VITE_TILES_BASE_URL at the public R2 URL (or a Cloudflare-fronted domain).

4 — GitHub Actions (automated)

Workflow	Trigger	Description
`annual_refresh.yml`	April 1 10:00 UTC + manual	Full ETL run + R2 upload
`deploy_frontend.yml`	Push to `main` or after refresh	Vite build + Pages deploy

Add the following repository secrets (Settings → Secrets and variables → Actions):

Secret	Used by
`R2_ACCOUNT_ID`	annual_refresh
`R2_ACCESS_KEY_ID`	annual_refresh
`R2_SECRET_ACCESS_KEY`	annual_refresh
`R2_BUCKET_NAME`	annual_refresh
`CF_ZONE_ID`	annual_refresh (optional cache purge)
`CF_API_TOKEN`	annual_refresh + deploy_frontend
`CF_ACCOUNT_ID`	deploy_frontend

And the following repository variables:

Variable	Used by
`CF_PAGES_PROJECT_NAME`	deploy_frontend
`VITE_TILES_BASE_URL`	deploy_frontend
`VITE_REPORT_API_URL`	deploy_frontend

Annual Data Refresh (manual)

make run-etl   # re-downloads, re-processes, re-exports everything
# Then re-deploy frontend/public/data/ to CDN or run deploy_frontend workflow

Data Sources

Layer	Source	License
Road Maintenance Jurisdiction	NYSDOT RIS (ArcGIS Hub)	Public domain (NY Gov)
State-Owned Parcels	NYS GIS (gisdata.ny.gov)	Public domain
Public Tax Parcel Centroids	NYS GIS (gis.ny.gov)	Public domain
Protected Areas	NYPAD (nypad.org)	Public domain
Municipal Contacts	NYSDOT PDF	Public domain
Highway Supt. Directory	NYSCHSA PDF	Public domain
Basemap	OpenFreeMap (OSM)	ODbL
Geocoding	Nominatim/OSM	ODbL

License

MIT. See LICENSE.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

NoticeForge - Road Reporter

Quick Start (local dev)

Prerequisites

1 — Clone and configure

2 — Build ETL Docker image (~5 min first time; compiles tippecanoe)

3 — Run the full ETL pipeline

4 — Start the frontend dev server

File Tree

Jurisdiction Key Strategy

Manual Contact Override Workflow

Production Deployment

Static frontend → Cloudflare Pages / Vercel / Netlify

PMTiles → Cloudflare R2 (recommended)

R2 CORS policy (required for PMTiles range requests)

Deployment

1 — Cloudflare Worker (hazard report endpoint)

2 — Frontend to Cloudflare Pages

3 — Tile artifacts to Cloudflare R2

4 — GitHub Actions (automated)

Annual Data Refresh (manual)

Data Sources

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
.github/workflows		.github/workflows
demo		demo
docker		docker
etl		etl
frontend		frontend
worker		worker
.env.example		.env.example
.gitignore		.gitignore
Makefile		Makefile
README.md		README.md
deploy.sh		deploy.sh
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation

NoticeForge - Road Reporter

Quick Start (local dev)

Prerequisites

1 — Clone and configure

2 — Build ETL Docker image (~5 min first time; compiles tippecanoe)

3 — Run the full ETL pipeline

4 — Start the frontend dev server

File Tree

Jurisdiction Key Strategy

Manual Contact Override Workflow

Production Deployment

Static frontend → Cloudflare Pages / Vercel / Netlify

PMTiles → Cloudflare R2 (recommended)

R2 CORS policy (required for PMTiles range requests)

Deployment

1 — Cloudflare Worker (hazard report endpoint)

2 — Frontend to Cloudflare Pages

3 — Tile artifacts to Cloudflare R2

4 — GitHub Actions (automated)

Annual Data Refresh (manual)

Data Sources

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages