refactor: change file off bindings in linux uclibc#5165
Draft
dybucc wants to merge 1 commit into
Draft
Conversation
The currently exposed bindings to file offset types and routines are
often found to be either lacking or incorrect in the current worktree.
This has been fixed by means of changing multiple type definitions
across child modules of the uClibc module under Linux, such that a more
cohesive interface is exposed.
Among those changes, the patch also switches the default definition of
types with LFS64 "variants," such that they use that definition by
default. This affects types such as `off_t` and `ino_t`, for which
support varied between platforms to either the regular definition
upstream or the LFS64 definition.
Beyond that, a few LFS64-specific routines have been deprecated in favor
of their unsuffixed variants. Much in the same vein as the prior
changes, this follows the trend towards supporting a single unsuffixed
type whose bitwidth is 64-bits. It has been assumed that this extends as
well to routines making use of such types.
Lastly, it must be noted that all changes submitted in this patch match
the definition of upstream uclibc-ng, but not necessarily of the
original uClibc (now unmaintained.) This was done by performing a regex
search through all header files under `**/{platform}/bits/*.h`, where
`{platform}` is replaced by one of: `common`, `common-generic`, `arm`,
`x86_64` or `mips`. The search term in question included any uses of the
`__USE_FILE_OFFSET64` or `__USE_LARGEFILE64` feature test macros. The
latter gates functionality that exposes 64-bit types, while the former
ensures the unsuffixed variants are defined in terms of the suffixed
types/routines. By default, upstream builds of uclibc-ng will have LFS
support added by default lest otherwise configured. This is shown in the
make configuration script generated by running make defconfig in the
root worktree.
dybucc
force-pushed
the
uclibc-file-offset-transition
branch
from
8755269 to
c883cf6
Compare
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR transitions the default types in uClibc from their 32-bit (non-LFS) variants to their 64-bit variants. Currently, the default build config when running the make scripts in upstream uclibc-ng will add support for LFS. This is configurable, but the
libccrate has no configuration option for it. Thus, this patch decides to expose the default behavior.The changes include a few modifications of the definitions for types involved in LFS64, such as
ino_tand whole records likedirent. A few routines are also affected.Because the
libccrate is actively attempting to provide a single unsuffixed variant that is solely 64-bits wide, multiple types that mirrored other types such as 64-bitoff_t, were deprecated. This patch does not target a stable release, as public fields of exposed records were altogether removed and sometimes reordered.This also means that the unsuffixed "variants", such as
off_t, now use theiroff64_tupstream definition by default. As a general rule, whenever a header file upstream checks for__USE_LARGEFILE64or__USE_FILE_OFFSET64, this patch defines the unsuffixed type in terms of the suffixed type.The only exceptions to this were records that are specifically not aliased in terms of their 64-bit "variants," but are rather both exposed in upstream uclibc-ng. Examples of this include
statandstatfs.There's one record I'm particularily doubtful of. In
libcwe choose to expose ever so slightly different definition for thestatvfsstructure depending on the target's endianness. This only affects whether a field appears before or after some padding, the latter of which depends on the target's word size. Other modules under theuclibcmodule do not seem to use this endian-dependent field ordering, so that has been removed as there's neither any indication in the upstream definition of there being need for that. Still, this is very much a change prone to reversion, if I'm, indeed, missing something. Input here is very much welcome.There were also some changes related to records that were straight up conflicting between builds for each of the supported target triples under
linux/uclibc. Among these,direnthad to be altogether removed from the uClibc interface because there were collisions while resolving items across the top-levellinuxandlinux_l4re_sharedmodules. Though the types of its fields still refer to the corresponding types in theuclibcchild module.Finally, it must be noted that there were quite a few architecture-specific definitions in upstream uclibc-ng that were not included in the exposed bindings. More specifically, no ISAs beyond those already present under the
uclibcmodule have been added. This means thelibccrate currently exposes the correct definitions for the header files common to all architectures, and those specific to MIPS32, MIPS64, x86_64 and 32-bit arm. Other architectures such as OpenRISC, Alpha, SPARC or i686 have not been included in this patch, even though they have specialized upstream types/routines. The policy here for the libc crate is unknown. Should support be provided solely for targets for which there's some tiered support level in upstream rustc, or should the provided bindings be as wide-ranging as possible? It must be noted that we support thex86_64module inlinux/uclibceven though there is only one tiered Rust target that uses uClibc on x86_64. That one uses L4Re and not Linux.Tests are pending on MIPS targets. This is a WIP.
Sources
Upstream uclibc-ng sources showing
ino_tdefined in terms of its 64-bit variant, and the definition for the 64-bit variant. There's a "chain" of types that leads to the final type with whichino64_tis defined. This chain is made most obvious by the final type alias, which appends a_TYPEand prepends two underscores to the type identifier in question. Notice how the__uquad_ttype is always defined as a 64-bit type, even on systems where that requires using a record type with a two-element array (32-bit per element).Upstream uclibc-ng sources showing
off_tdefined in terms of its 64-bit variant when LFS64 support and__USE_FILE_OFFSET64are enabled. Much like the above, there's a "chain" of types leading to the type whose bitwidth is made clear. The chain is shortened to the last few types.Upstream uclibc-ng sources showing
rlim_tdefined in terms of its 64-bit variant when LFS64 support and__USE_FILE_OFFSET64are enabled. This follows the same reasoning as the above.Upstream uclibc-ng sources showing
rlim_tdefined in terms of its 64-bit variant. This follows the same reasoning as the above.Upstream uclibc-ng sources showing
fsblckcnt_tandfsfilcnt_tdefined in terms of their 64-bit variants. This first of the below sources also includes references to the above types.Upstream uclibc-ng sources showing how
flockandflock64are independent, but when defined with__USE_FILE_OFFSET64, will end up having the same record layout. The second source shows how this definition can actually be unified across all child modules to theuclibcmodule because the fields that are conditionally compiled under MIPS will never be any different if__USE_FILE_OFFSET64is defined.Upstream uclibc-ng sources showing how
statvfsandstatvfs64are independent, but like the above, have equivalent definitions when__USE_FILE_OFFSET64is defined. This also shows how there does not seem to be a need for the fields to be reordered depending on the system's endianness.Upstream uclibc-ng sources showing how
statfsandstatfs64are independent, but like the above, have equivalent definitions when__USE_FILE_OFFSET64is defined. A bunch of field types were changed to both accurately reflect the upstream types used, and to add missing fields. The former has not been documented in full in the below sources because the types involved are already part of the sources cited in other list items. Some required special-casing in MIPS platforms has been necessary, as there were fields that had altogether different types. These were only ever so slightly tweaked for the same reasons as above.statfsin MIPS got unified as the definitions for both the o32 and N64/N32 ABIs are equivalent under__USE_FILE_OFFSET64.Upstream uclibc-ng sources showing the same as the above but for
statandstat64. There was also some special casing required in MIPS platforms, though the definitions for both x86_64 and arm converged. These have been kept in their corresponding modules because there's need for MIPS to have their own and it was decided against a top-level, field-specificcfg.Checklist
libc-test/semverhave been updated*LASTor*MAXare included (see #3131)cd libc-test && cargo test --target mytarget); especially relevant for platforms that may not be checked in CI