Skip to content

Fix local pipeline database registration and empty village-ID handling#1122

Merged
pawangramvaani merged 4 commits into
feature/local-computefrom
fix/error_checks
Jul 21, 2026
Merged

Fix local pipeline database registration and empty village-ID handling#1122
pawangramvaani merged 4 commits into
feature/local-computefrom
fix/error_checks

Conversation

@amit-spatial

@amit-spatial amit-spatial commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Register locally generated Facilities, Antyodaya, and Livestock layers through the canonical save_layer_info_to_db and update_layer_sync_status functions after successful GeoServer publication.
  • Use not applicable: local compute GeoServer layer for asset_id, which is stored in gee_asset_path.
  • Pass only {"is_generated_locally": true} in misc.
  • Remove the shared register_layer abstraction and its package export so these first three consumers exercise the production database functions directly.
  • Report tehsils containing village geometries but no usable village IDs gracefully instead of raising a missing village_code KeyError.

Root cause

The shared wrapper introduced a second layer-registration implementation instead of using the established production functions directly.

Separately, an empty sidecar lookup returned a DataFrame without its configured key columns. For tehsils where every village ID was missing, downstream code then accessed a nonexistent village_code column.

Database behavior

Each successfully published layer now calls:

layer_id = save_layer_info_to_db(
    ...,
    asset_id="not applicable: local compute GeoServer layer",
    misc={"is_generated_locally": True},
)
update_layer_sync_status(layer_id=layer_id, sync_to_geoserver=True)

Registration and sync failures are raised explicitly rather than being silently reported as successful pipeline runs.

Validation

  • Reran Facilities, Antyodaya, and Livestock for Bihar/Jamui/Jamui through Django with use_pregenerated=false and live GeoServer publication.
  • Verified five database registrations through the canonical helpers: Facilities Proximity, two Facilities Points layers, Antyodaya 2020, and Livestock Census.
  • Verified the explanatory gee_asset_path, is_public_gee_asset=false, is_sync_to_geoserver=true, and misc.is_generated_locally=true.
  • Verified empty village-ID lookups preserve the configured village_code schema.
  • Passed py_compile, focused Ruff fatal-error checks, and git diff --check.

Scope

This PR contains four commits and changes only:

  • computing/misc/facilities/pipeline.py
  • computing/misc/antyodaya/pipeline.py
  • computing/misc/livestocks/pipeline.py
  • utilities/pipelines/__init__.py
  • utilities/pipelines/publish.py
  • utilities/pipelines/tabular.py

No changes are made to computing/utils.py, authentication, API permissions, or migrations. The CS admin-standard resource work remains isolated in PR #1084.

@amit-spatial
amit-spatial marked this pull request as ready for review July 20, 2026 12:01
@pawangramvaani
pawangramvaani merged commit c98a502 into feature/local-compute Jul 21, 2026
@pawangramvaani
pawangramvaani deleted the fix/error_checks branch July 21, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants