Use standard extension configuration for CPython 3.12+ - #1180
Conversation
2bc109e to
059aed0
Compare
jjhelmus
added
platform:darwin
jjhelmus
added
arch:all
Configure pkgconf to report the temporary directories of the prefix that is used on macOS during CPython builds.
Build standard-library extensions through CPython's configure and Setup.stdlib mechanism for Python 3.12 and newer. Default to statically linking these modules into libpython/python with a few overrides speficied in a Setup.local file created from extension-modules.yaml. This allows CPython's extension checks to run. Python 3.11 and 3.10 contrinue to use the existing process to configure and build extension modules.
Configuration correctly detect the SIMD support in the toolchain and applies the appropiate flags when building libhacl.
These extensions were not enabled when using a custom build configuration + build. Maintain this policy.
…tion Reverts commit 059aed0
|
With
Using the custom configuration the last three of these are disabled and not built. The first four are build as shared modules except for static builds where they are statically linked. Because of the behavior of Setup.stdlib.in theses three need to be included in Setup.local to restore the static linkage. This should be revisited as disabling these in the static builds may be more prudent. The existing behavior for 3.10 and 3.11 is to include a shared module, #1222 which is retained. |
…stdlib.in A handful of modules are marked as always building shared which causes these to be built as shared modules in the static builds.
5bcf925 to
a77eb2c
Compare
|
This is ready for a review. There is follow up work to construct This could be done in this PR, in a stacked PR, or as a fast follow. I'd prefer merging this and following with another PR to avoid a long running branch. The work for building |
| # ./configure tries to auto-detect whether it can build 128-bit and 256-bit SIMD helpers for HACL, | ||
| # but on x86-64 that requires v2 and v3 respectively, and on arm64 the performance is bad as noted | ||
| # in the comments, so just don't even try. (We should check if we can make this conditional) | ||
| if [[ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" ]]; then | ||
| patch -p1 -i "${ROOT}/patch-python-configure-hacl-no-simd.patch" | ||
| fi |
| # See https://github.com/python/cpython/issues/145810#issuecomment-4068139183 | ||
| replace_in_all("-LModules/_hacl", "") |
There was a problem hiding this comment.
| # CPython uses the macOS SDK's zlib without a zlib.pc file. Keep the -lz in | ||
| # Libs.private, but do not make pkg-config require the missing metadata. | ||
| if [[ "${PYBUILD_PLATFORM}" = macos* ]]; then | ||
| sed -i '' -e 's/ zlib >= 1.2.3//' tcl.pc.in |
There was a problem hiding this comment.
Should we be using a patch here instead?
| sed -i '/^Requires: /a\ | ||
| Requires.private: x11' tk.pc.in |
There was a problem hiding this comment.
2 participants
Switch CPython 3.12+ builds to the upstream Setup.stdlib extension-building mechanism while preserving the existing static/shared linkage behavior.
pkgconfon macOS to provide Tcl/Tk configuration to CPython.xxlimited_3_13extensions. c.f. Enable _testclinic, _testclinic_limited and xxlimited_3_13 #1185With this change,
extension-modules.ymlis still used for the following in CPython 3.12:Setup.stdlib.inPYTHON.jsonConstructing
PYTHON.jsonfor CPython 3.12+ from other sources is a needed follow up.Makes various patches unnecessary:
patch-checksharedmods-disable.patch#869patch-configure-disable-stdlib-mod-3.12.patch#875 (This patch is only applied on Python 3.11)patch-pwd-remove-conditional.patch#876 (this patch is only applied on Python 3.11)Configuration can now appropriately detect SIMD support in the toolchain and adjusts
libhaclflags accordingly.patch-python-configure-hacl-no-simd.patch#883This does not add shared extension suggested in #227. This could be done with a small change as a follow up.