fix(cli): support $-prefixed pagination params for Socrata-style APIs#1204
fix(cli): support $-prefixed pagination params for Socrata-style APIs#1204youdidwhat-towho wants to merge 1 commit into
Conversation
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟠 require-ready-label-and-ciWaiting for
Waiting checks:
|
Greptile SummaryThis PR introduces an optional
Confidence Score: 4/5Safe for Socrata GET-only use cases, but the fix is incomplete: POST, PUT, and DELETE endpoints with query params will silently ignore url_name and emit the wrong key in the URL. All GET-based code paths are correctly updated and the spec/helper logic is sound. However, the same {{.Name}} interpolation that was fixed for GET still appears in every non-GET branch of command_endpoint.go.tmpl. internal/generator/templates/command_endpoint.go.tmpl — seven non-GET params[{{.Name}}] sites need the same paramWireName substitution applied in this PR Important Files Changed
|
Merge Queue Status
This pull request spent 25 minutes 26 seconds in the queue, with no time running CI. ReasonPull request #1204 has been dequeued The pull request rule doesn't match anymore HintYou should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it. |
|
@youdidwhat-towho resolve conflicts |
Adds an optional `url_name` field to spec.Param that, when set, overrides
the URL query-key while leaving the CLI flag, Go identifier, and JSON body
emission unchanged. Specs targeting Socrata-backed APIs (NYC OpenData and
~70 other major US municipal portals) can now declare:
params:
- name: limit
url_name: "$limit"
type: integer
and the generated handler emits params["$limit"] in the URL while keeping
--limit as the cobra flag.
Previously generated CLIs against Socrata returned HTTP 400
"Unrecognized arguments [limit]" because the wire-side key was the plain
Name. Specs without url_name are unaffected — verified by regenerating
bexar-cad-pp-cli and confirming params["f"], params["outFields"], etc.
remain plain.
Changes
- internal/spec/spec.go: add URLName field + Param.WireName() method
- internal/generator/flag_collision.go: add paramWireName helper, mirroring
paramIdent. Updated paramIdent comment to clarify URL emission goes
through paramWireName, not Name directly.
- internal/generator/generator.go: register paramWireName template helper
- internal/generator/templates/command_endpoint.go.tmpl: 6 emission sites
switched from {{.Name}} to {{paramWireName .}}
- internal/generator/templates/command_promoted.go.tmpl: 8 emission sites
switched from {{.Name}} to {{paramWireName .}}
- internal/generator/url_name_test.go: new file. Three tests cover
WireName() unit behavior, $-prefixed URL emission via url_name, and
the regression guard for plain specs.
Notes
- Path substitution, JSON body keys, and MCP tool bindings still use Name.
url_name affects URL query keys only.
- Pagination's limit_param/cursor_param already accepted user-supplied
strings (e.g. "$limit"), so the pagination loop alone was working;
this fix closes the remaining gap for non-pagination SoQL params like
$where, $select, $order.
- Secondary Socrata gap (id_field extraction needs $select=:id,*
auto-injection) is NOT addressed here. Filed as follow-up.
Live-tested
- nyc-liens-pp-cli regen with url_name on limit/where/select returns
valid Brooklyn water-only lien records from
https://data.cityofnewyork.us/resource/9rz4-mjek.json against borough=3
and water_debt_only=YES.
- bexar-cad-pp-cli regen unchanged — ArcGIS plain params preserved.
Signed-off-by: youdidwhat-towho <chris@lovephoenixhomes.com>
1e8bee0 to
af0731a
Compare
Problem
Press's generator emits paginated-GET handlers that put each Param's
Namedirectly into the URL query string. Socrata-backed APIs (NYC OpenData and ~70 other major US municipal portals) require a literal$prefix on pagination + SoQL params:$limit,$offset(pagination)$where,$select,$order,$group(SoQL passthrough)Generated CLIs against Socrata returned
HTTP 400 Unrecognized arguments [limit]on every live call.Fix (Option 2 — per-param
url_nameoverride)Adds an optional
url_namefield tospec.Paramthat overrides the URL query-key emission while leaving the CLI flag name, Go identifier, and JSON body emission unchanged.--limitstays the CLI flag,flagLimitstays the Go identifier, but the URL emits?$limit=N.Why this approach (vs auto-detect)
Auto-detecting
*.socrata.com/*.cityofnewyork.uswould couple the generator to URL patterns, which doesn't scale to other custom-prefix APIs. A spec-level opt-in is non-breaking and explicit — existing specs work unchanged, new Socrata specs declare the prefix where they declare every other param attribute.What's NOT touched
Name.url_nameaffects URL query keys only.limit_param/cursor_paramalready accepted user-supplied strings, so spec-level pagination config already worked. This fixes the remaining gap for non-pagination SoQL params ($where,$select, etc.) where the user declares them as ordinary params.url_namedeclared keeps emitting plainName.Tests
New file
internal/generator/url_name_test.go:TestParamWireNameUnit—spec.Param.WireName()returns URLName when set, Name when emptyTestParamURLNameOverridesWireKey— generated handler emitsparams["$limit"]for a param withurl_name: "$limit", plainparams["borough"]for a sibling param without override, and CLI flag identifiers stay clean (no$leak into Go)TestParamWithoutURLNameUnchanged— regression guard: specs withouturl_namekeep plainparams["limit"],params["owner"]emissionFull
internal/spec/+internal/generator/test suite green:Live verification
Before (working around the bug by stripping
limitfrom the spec):nyc-liens-pp-cli records query --borough 3 --water-debt-only YES # HTTP 400 if any of limit / where / select suppliedAfter (with
url_namedeclared):nyc-liens-pp-cli records query --borough 3 --water-debt-only YES --limit 2 # Returns 2 valid Brooklyn water-only lien records, structured JSONRegression check —
bexar-cad-pp-cli(ArcGIS, nourl_nameanywhere):Inspected generated handler for bexar-cad-test confirms plain emission preserved:
And for nyc-liens-test (mixed plain + url_name):
Diff size
6 files, +166 / -16. Five files modified, one new test file.
Follow-up — NOT addressed in this PR
Press's
id_fieldextraction fails on Socrata because Socrata only returns its:idrow identifier when explicitly$select-ed. Generated sync runs produceall_items_failed_id_extractionwarnings on Socrata datasets. The clean fix is to detectid_field: ":id"in the spec (or any field starting with:) and auto-inject$select=:id,*(or the union of explicit$select+:id) into the URL. Filing as a separate PR to keep this one scoped.Unlocks
This change unblocks every Socrata-backed CLI Factory target: