Skip to content

Conversation

@yvan-sraka
Copy link
Collaborator

@yvan-sraka yvan-sraka commented Dec 9, 2025

This change adds crane support to the existing pgrx extension builder, enabling better incremental build performance and caching for pg_jsonschema and other pgrx extensions. Crane separates dependency builds from main crate builds, allowing dependencies to be cached independently.

The unified buildPgrxExtension.nix now accepts an optional craneLib parameter. When provided, it uses crane's two-phase build (buildDepsOnly + buildPackage). Otherwise, it falls back to rustPlatform.buildRustPackage. Both paths share the same build and install logic, avoiding code duplication.

Crane requires Cargo.lock at the source root while rustPlatform accepts external lockFile paths. The builder handles this by ensuring external lock files are properly accessible during the build process. Crane's cargoVendorDir is only used for crane builds to avoid conflicts with rustPlatform's cargo handling.

Extensions opt into crane builds by passing useCrane = true to mkPgrxExtension. All existing build parameters remain compatible with both backends.

@yvan-sraka yvan-sraka requested a review from jfroche December 9, 2025 22:11
@yvan-sraka yvan-sraka self-assigned this Dec 9, 2025
@yvan-sraka yvan-sraka force-pushed the feat/use-crane branch 2 times, most recently from 906fbe3 to 8f0dffb Compare December 10, 2025 21:32
@yvan-sraka yvan-sraka closed this Dec 10, 2025
@yvan-sraka yvan-sraka reopened this Dec 10, 2025
@yvan-sraka yvan-sraka force-pushed the feat/use-crane branch 3 times, most recently from d95ecff to 38ead9e Compare December 10, 2025 22:27
@yvan-sraka yvan-sraka force-pushed the feat/use-crane branch 4 times, most recently from 4665a3f to 14906f4 Compare December 11, 2025 10:42
@yvan-sraka yvan-sraka force-pushed the update-nixpkgs branch 5 times, most recently from b48e4c3 to 3df6c4a Compare December 12, 2025 10:07
@yvan-sraka yvan-sraka changed the title [WIP] feat: migrate pgrx extensions to crane build system (WIP) feat: migrate pgrx extensions to crane build system Dec 12, 2025
@samrose
Copy link
Collaborator

samrose commented Dec 12, 2025

this one seems pretty important and would like to get to this one soon after nixpkgs update

pg_net requires an older version of curl (pinned from nixpkgs commit
a76c4553d7e741e17f289224eda135423de0491d) to resolve build compatibility issues.
Updated cargo-pgrx buildPgrxExtension with rustc wrapper to handle empty
postmaster_stub.rs files that cause duplicate symbol errors.
(first two filenames are `` and
`/build/tmp.*/postmaster_stubs/nix/store/*/bin/postmaster_stub.rs`)

pg_jsonschema cargo checks fails on specific versions (0.2.0, 0.3.1, 0.3.3).
Remove deprecated functions that were removed in newer pgrouting versions
(_pgr_betweennesscentrality and _pgr_contractionhierarchies).
Improves reproducibility and avoids evaluation-time network access by
using a proper flake input instead of fetchTarball for the older nixpkgs
needed by pg_net and plv8.
…64-darwin

Version 0.19.5 has a dangling pointer issue in src/core.c:177 that causes
compilation to fail on aarch64-darwin with newer clang versions. This adds
-Wno-error=dangling-assignment to allow compilation to proceed.

The flag is restricted to aarch64-darwin only since GCC on Linux doesn't
recognize this warning option.
Add -headerpad_max_install_names linker flag to prevent 'install_name_tool:
changing install names or rpaths can't be redone' errors on macOS. This
allocates sufficient header space for install_name_tool to modify the
shared library during post-install processing.
… extensions

Test assertions were inconsistent: expecting {version}.so and {extension_name}-{version}.so
patterns while actual libraries use {lib_name}-{version}.so (e.g. pg_partman_bgw-5.3.1.so,
timescaledb-loader-2.16.1.so). Updated PostgresExtensionTest to accept Optional[str] lib_name
parameter and use consistent {lib_name}-{version}.so assertions throughout.

Added missing timescaledb-loader.so symlink creation and updated the switch script to
maintain both symlinks when switching versions.

Fixes 'Expected timescaledb version 2.16.1, but found timescaledb-loader-2.16.1.so' and
'Expected pg_partman version 5.3.1, but found pg_partman_bgw-5.3.1.so' test failures.
Use overlay instead of passing full nixpkgs-oldstable to avoid exposing
entire old nixpkgs attribute set. Now provides specific packages
(curl_8_4, v8_oldstable) through overlay.
Keep revision support that was accidentally simplified during nixpkgs update.
Restores conditional logic to support building orioledb from specific git
revision hashes instead of only tags.
These files were moved to subdirectories and the old .nix files should be
removed since they were moved to new directory structure with default.nix files.
Remove nixpkgs-go124 dependency since current nixpkgs has Go 1.25.4
which is sufficient for packer. Remove nix-fast-build input as unused
and add follows directives to resolve duplicate dependencies.
Run flake-linter to eliminate duplicate dependencies, and remove unused
cargo-pgrx / rustc versions hashes.
Move pgrouting.nix to pgrouting/default.nix and groonga package to
pgroonga/groonga.nix. Update imports and fix relative paths for versions.json
and patch files.
Replace inline bash script generation with writeShellScriptBin for rustc
wrapper that filters empty postmaster_stub.rs arguments. Apply wrapper only
for pgrx < 0.12 since issue was fixed upstream in pgrx#1435 and pgrx#1441.
Move nixpkgs-oldstable import to a let binding to avoid importing it
three times...
Remove supabase-groonga from global flake packages and import it locally
in pgroonga extension instead. Other components access groonga via
pgroonga.passthru.groonga since groonga is only used by pgroonga.
The supabase-groonga package was moved to pgroonga extension in commit 1f0ed1f,
but Ansible was still trying to install it separately causing build failures.
This reverts commit 55474686405394826ad07a032cbcf0e36a924554.
This reverts commit cfa9bcb92219f391469ae743f95c8856b0a99561.
Remove references to darwin.apple_sdk.frameworks.{IOKit,CoreFoundation}
which have been deprecated in nixpkgs. Disable CGO to avoid Darwin
framework dependencies entirely.
@yvan-sraka
Copy link
Collaborator Author

pg_graphql (with useCrane = true) hits a tokio-postgres compilation error:

error: builder for '/nix/store/80sbdnbf97j3wjiy38vwi0w9y8qj41l2-pg_graphql-deps-deps-1.2.0.drv' failed with exit code 101;
       last 25 log lines:
       >     Checking tracing-error v0.2.1
       >    Compiling semver v0.11.0
       >     Checking tokio-util v0.7.4
       >     Checking pgx-sql-entity-graph v0.7.1
       >     Checking futures-executor v0.3.26
       >     Checking tokio-postgres v0.7.8
       >     Checking futures v0.3.26
       >    Compiling rustc_version v0.3.3
       >    Compiling atomic-traits v0.3.0
       >     Checking clap v4.1.6
       > error[E0277]: `DataRowBody` doesn't implement `std::fmt::Debug`
       >    --> /nix/store/v7hfg1lha0s198amava5m9gfqqpj9vk1-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/tokio-postgres-0.7.8/src/row.rs:202:5
       >     |
       > 199 | #[derive(Debug)]
       >     |          ----- in this derive macro expansion
       > ...
       > 202 |     body: DataRowBody,
       >     |     ^^^^^^^^^^^^^^^^^ `DataRowBody` cannot be formatted using `{:?}` because it doesn't implement `std::fmt::Debug`
       >     |
       >     = help: the trait `std::fmt::Debug` is not implemented for `DataRowBody`
       >     = note: this error originates in the derive macro `Debug` (in Nightly builds, run with -Z macro-backtrace for more info)
       >
       > For more information about this error, try `rustc --explain E0277`.
       > error: could not compile `tokio-postgres` (lib) due to 1 previous error
       > warning: build failed, waiting for other jobs to finish...
       For full logs, run 'nix log /nix/store/80sbdnbf97j3wjiy38vwi0w9y8qj41l2-pg_graphql-deps-deps-1.2.0.drv'.
error: 1 dependencies of derivation '/nix/store/f8gq25rp5z9g14k5kbgsil4k4clhbdv5-pg_graphql-1.2.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/07fiwlxlzk4vpa5b57b6kzyj9nw696ma-pg_graphql.drv' failed to build

Basic crane integration works correctly (pg_jsonschema builds successfully) but crane uses the external Cargo.lock files directly while rustPlatform was applying patches that contained e.g. fixed dependency versions. Here, the external Cargo.lock files have tokio-postgres 0.7.8 which has this debug trait issue.

To fix this, we need to either update the external Cargo-X.X.X.lock files with compatible dependency versions or apply the same patches that rustPlatform was using.

@yvan-sraka yvan-sraka requested a review from samrose December 14, 2025 01:22
This change adds [crane](https://crane.dev/) support to the existing `pgrx` extension builder, enabling better incremental build performance and caching for `pg_jsonschema` and other pgrx extensions. Crane separates dependency builds from main crate builds, allowing dependencies to be cached independently.

The unified `buildPgrxExtension.nix` now accepts an optional `craneLib` parameter. When provided, it uses crane's two-phase build (`buildDepsOnly` + `buildPackage`). Otherwise, it falls back to `rustPlatform.buildRustPackage`. Both paths share the same build and install logic, avoiding code duplication.

Crane requires `Cargo.lock` at the source root while `rustPlatform` accepts external `lockFile` paths. The builder handles this by ensuring external lock files are properly accessible during the build process. Crane's `cargoVendorDir` is only used for crane builds to avoid conflicts with rustPlatform's cargo handling.

Extensions opt into crane builds by passing `useCrane = true` to `mkPgrxExtension`. All existing build parameters remain compatible with both backends.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants