[pull] main from llvm:main - #1715
Merged
Merged
Conversation
InstCombine converts vXi1 logic reductions to bitcasted scalar integer ops - we should be testing that, not llvm.vector.reduce.and.v32i1 calls
Add support for `DenseResourceElementsAttr` in TOSA reshape folding.
Currently we copy construct `locale`s whenever we swap, which is quite expensive for locales since that does atomic increments and decrements. We can just swap the pointers instead.
…onicalizers (#207239) `InsertSliceOpConstantArgumentFolder` re-derives the refined source type via `ExtractSliceOp::inferCanonicalRankReducedResultType`, which copies the encoding of the passed-in "source template" - in this pattern, `insertSliceOp.getDestType()`. For a static, encoding-less destination this silently drops any encoding the actual source carried, which downstream dialects can use for mandatory metadata (upper bounds, layout, sparsity descriptors) lost during `--canonicalize`. Rebuild the refined source type carrying the original source's encoding. If the encoding implements `VerifiableTensorEncoding`, re-verify it against the refined shape and drop it if it no longer holds (e.g. a rank-dependent sparse encoding); otherwise treat it as opaque/shape-agnostic and propagate it unconditionally. Shape refinement (the `?` -> static direction) is unchanged and still satisfies `preservesStaticInformation`. lit created with Claude Opus 4.7 Signed-off-by: Dmitrii Makarenko <dmitrii.makarenko@intel.com>
…213730) These tests are never executed since they require C++20, and the C++03 frozen headers only run in C++03 mode.
…213732) The filter regex used `libcxx(abi?)`, which matches `libcxxab` and `libcxxabi`, but never just `libcxx`.
Based on local measurements, the associative containers are by far the slowest benchmarks to run. While there is value in benchmarking small and large sizes, we can significantly cut down benchmark times by stripping down to only 2 sizes (from 4). With this patch, the benchmarks go from 3124s to 1766s across the associative container benchmarks, which is -43% of their execution time, and corresponds to ~15% of the entire benchmark suite. Towards #173032
#213923) InstCombine converts vXi1 logic reductions to bitcasted scalar integer ops - we should be testing that, not llvm.vector.reduce.or.v3i1 calls Exposes some really poor scalarization on pre-AVX512 targets
olMemcpy currently disallows host-to-host copies with a queue, even though that is a desirable use case (e.g. to copy a buffer as a dependency for another operation) This PR adds support for asynchronous host-to-host copies by adding `GenericDeviceTy::dataMemcpy` and the corresponding required `dataMemcpyImpl` plugin operation. Unlike `dataSubmit`, `dataRetrieve`, and `dataExchange`, this operation does not assign a device direction to either pointer: - CUDA: Uses `cuMemcpyAsync`, which by itself determines the pointer types and preserves stream ordering. Dynamic CUDA symbol declarations and `_v2` lookup support are included. - Level Zero: Uses the existing queue `memoryCopy` operation (backed by `zeCommandListAppendMemoryCopy`), which appends a memory-copy command to the command list. - AMDGPU: No "enqueue a host-to-host copy" operation exists directly in that API, so we enqueue a `std::memcpy` callback through `pushHostCallback`. - Host: Call `memcpy` directly. The `olMemcpy` API documentation now states that queued host-to-host copies are ordered with other work in the queue. The public function signature is unchanged. A new unit test executes an `h2d -> d2h -> h2h -> h2d -> d2h` operation chain, which should fail if the queue order isn't respected. Assisted-by: Codex
…gets (#200134) I am working with a binary where some symbols are generated from linker scripts. They end up identified as `eSymbolInvalid` when loaded in LLDB. OpenOCD can try to fetch them. Some might even be hard-coded data, not going through an address in the binary (explaining the presence of the check and raw value return). Testing it showed that **GDB returns them despite them not being proper addresses**. These can also be generated by C++ static constexpr, such values could be accessed by a qSymbol query. 358cf1e introduced a divergence between GDB and LLDB where LLDB does not serve symbols of unknown type through GDB protocol command `qSymbol`. Per commit description, this is an expected behavior on MachO-based platforms, but it is not on ELF-based platforms, where LLDB should follow GDB. The changes introduced by said commit are now gated behind an architecture check.
… b, -0.0) A copyable fmul absorbed into fmuladd(a, b, -0.0) does not appear in the operand columns of its own node, so the operand scan never released the schedule data of the copyable instruction itself and scheduling ended with unscheduled bundles. Release it explicitly when the copyable element shows up in no operand column. Reviewers: Pull Request: #213937
Do addrspace cast for allocas that live in address space different from the datalayout's alloca address space, otherwise ASan will crash during use replacement
…s. (#212800) For SVE2 we can use top/bottom [S/U]ABAL[T/B] instruction. SVE2p3 can use just [S/U]ABAL.
A copyable single-use fadd a, b is modeled as fmuladd(1.0, a, b), which equals fadd a, b (the multiply by 1.0 is exact and preserves signed zeros), so the add dies instead of being computed and gathered. The addend/multiplicand assignment per lane matches the majority operand kinds of the non-copyable lanes. Same all-or-nothing and tie-break rules as for absorbed fmuls. Reviewers: hiraditya, bababuck, RKSimon Pull Request: #213786
This adds `[[gnu::warn_unused]]` to all container types currently implemented. In the future new containers should be added to the list, as well as other value types in the library. Clang has `-Wunused`, which is a suite of warnings about unused variables. However, that doesn't warn for a lot of class types, especially non-trivial ones. That is done to avoid false-positives for RAII-style types like `lock_guard`. `[[gnu::warn_unused]]` exists to tell Clang that a given class is a value type.
…3736) Since runtimes/ has been picked up by other subprojects, it doesn't make sense anymore for reviewers-libcxx to watch all changes to it.
`SPV_KHR_float_controls2` makes `FPFastMathMode` valid for all core instructions, so this patch enables translation of fast math flags for `OpFunctionCall`.
For calls to `remquo`, the return values are generally expected to be used, while it is rare to only use the value written via the pointer parameter. Moreover, some implementors already planned to apply `[[nodiscard]]` to `remquo`, see also https://llvm.org/PR171763. As a result, it is perhaps better to add `void` cast to the discarded call to `remquo` in the test file.
Replaces exp2f and exp10f with fully vectorised implementations. Includes refactoring of expf polynomial evalulation into expf_utils.h
fat-test.arm.dylib contains armv7, armv7s, and arm64 slices, so the test needs the AArch64 backend in addition to ARM.
…C) (#213949) This PR moves `populateLowerContractionToSVE*` entry points into the `arm_sve` namespace for consistency with their Arm Neon counterparts.
…213788) This is the final batch of skips. Together with a handful of local changes to LLDB and WAMR, the test suite now passes when targeting WebAssembly. Each test has a short comment explaining why it's skipped.
…vailable (#212616) The `Zbc` and `Zvbc` extensions in RISC-V are independent, thus some hardware may support `Zvbc` (`vclmul(h)`), but not `Zbc` (`clmul(h)`). Rather than forcing scalar `CLMUL(H)` to expand in this case, simply insert into a vector register, use `vclmul(h)`, and extract instead. For now, only allow this for RV64, where `clmul` and `vclmul` both operate on the same width. RV32 has the potential for similar tricks, but it would be more complicated, and is left as a FIXME for now.
"tail-pad-to-size"="n": Pad functions up to at least 'n' bytes. "tail-pad-value"="n": Pad using the value 'n' (default 0). This patch is to support dynamic debugging, RFC: https://discourse.llvm.org/t/90113 In order to transfer control from an optimized to an unoptimized function a debugger may patch the optimized version with a jump to the unoptimized one. This patch adds a mechanism to ensure there's enough padding between functions to avoid patches overwriting unrelated code. It adds padding to the end of functions smaller then 'n' bytes. The `.fill` directive expression was discussed here: https://discourse.llvm.org/t/89658 See the RFC and pull request for discussions of alternative approaches.
…213766) Migrate G_FNEG, G_FABS, G_FSQRT, G_FFLOOR, G_FLDEXP, G_STRICT_FLDEXP, G_FCMP, and G_IS_FPCLASS to extended LLTs. Remove redundant scalar clamps that request unsupported f128-to-f64 narrowing. Update the relevant MIR tests.
This fixes 962d16f (#211365). Buildkite error link: https://buildkite.com/llvm-project/upstream-bazel/builds?commit=962d16fd97793d22c42227e7fab397f62475c18b Co-authored-by: Google Bazel Bot <google-bazel-bot@google.com>
…213753) In #208510 I accidentally introduced a bug where I allowed sub-byte element types with atomicrmw elementwise (i.e. `<8 x i1>`). The LangRef states: ``` The access type must then be a fixed vector type whose total bit width is a power of two and whose element type is supported by the corresponding scalar atomic instruction. ``` The second part of this sentence rejects sub-byte element types. The fix is that elementwise atomics need to additionally call `checkAtomicMemAccessSize` on the vector element type (they already call it on the whole vector).
The <__locale> header was a collection of distinct utilities. This patch splits it up into granular headers under __locale_dir/. This patch however does not use the granular headers from users of <__locale>: this will be done in a separate patch. Assisted by Claude
…FC) (#213864) I refactored the `load atomic elementwise` check to have the same cleaner pattern as `atomicrmw elementwise` and remove the `ScalarTy` intermediate. The previous code had this segment: ``` if (VecTy) { checkAtomicMemAccessSize(ScalarTy, &LI); ScalarTy = VecTy->getElementType(); } ``` Which is confusing because it's checking the atomic access size on the whole vector and then assigning `ScalarTy` to the element type so that the second check does the check on the element type. It's cleaner to reverse this and have the first check operate on the element type so that the second check always checks the entire type.
XPASSing, see [here](#212384). Signed-off-by: Nick Sarnie <nick.sarnie@intel.com>
Reviewers: Pull Request: #213985
Swift async functions can miscompile on Windows ARM64 at `-O2`, when there's enough register pressure that a local gets scavenged into the callee-save area: the local ends up sharing an address with the saved caller x29, so the epilogue restores a value the function has already overwritten. swiftlang/swift#90920 has a reduced repro. `assignCalleeSavedSpillSlots` creates the swift async context object before the callee-save loop instead of inside it next to the FP slot. MachineFrameInfo ends up with it above the frame record while the prologue stores it below at FP-8, and the 8 byte disagreement leaves a hole in the middle of the callee-save area. PEI's scavenger hands that hole to the local. Only reproduces at -O2 and up since scavenging is gated on the opt level. ``` sub sp, sp, #112 str x19, [sp, #16] // 8-byte Spill str x21, [sp, #24] // 8-byte Spill stp x23, x24, [sp, #32] // 16-byte Folded Spill stp x25, x26, [sp, #48] // 16-byte Folded Spill stp x27, x28, [sp, #64] // 16-byte Folded Spill stp x29, x30, [sp, #88] // 16-byte Folded Spill str xzr, [sp, #80] add x29, sp, #88 ... str x7, [x29] // 8-byte Spill ... ldr x1, [x29] // 8-byte Reload ldp x29, x30, [sp, #88] // 16-byte Folded Reload ``` This creates the object inside the loop so the two agree. The other option was leaving the creation site alone and teaching MachineFrameInfo about the expanded 24 byte FP/LR slot, but that puts the layout in two places. Not sure which is preferred here, I don't know this code well. This also asserts the saved FP object resolves to FP+0, since nothing checks that today. Reverting the fix makes it fire on the same funclet. `store-swift-async-context-clobber-live-reg.ll` already miscompiles with `-regalloc=fast`, so this isn't Swift specific.
…212388) Explaining why we have `_LIBCPP_PREFERRED_OVERLOAD` and template versions.
Add a verifier for spirv.ExecutionMode enforcing the correct number of literal operands per execution mode and rejecting `<id>`-only modes and share the operand schema with spirv.ExecutionModeId
Reviewers: Pull Request: #213992
This updates the tests to match ctpop, how we test other operations. Some extra type coverage and cssc is added.
`VScaleAttr` creates an LLVM `VScaleRangeAttr`, but did not declare the LLVM dialect as a pass dependency. This aborts when the input does not otherwise load LLVM. Declare the dependency and remove the unused FIR-typed argument from the existing test. Parsing `!fir.ref` loads `FIROpsDialect`, which loads `LLVMDialect` as a dependency and previously masked the missing pass dependency. Signed-off-by: Keshav Vinayak Jha <keshavvinayakjha@gmail.com>
`llvm-mc` asserts on a FPCLASS category mask given as a symbol: ```asm vfpclassps $f0, %zmm1, %k1 ``` ``` Assertion failed: isImm() && "This is not an immediate", MCInst.h:85 ``` `printFPCLASSComments` reads the last operand with `getImm()` without checking it is one. The value is only known at link time, so there is no category to describe; return early and print no comment. Note this is a comment printer, so encoding is unaffected. `--filetype=obj` already succeeds today and emits a placeholder immediate plus an `R_X86_64_8` relocation, which is correct. The issues describe this as producing a wrong encoding in release builds, which I don't think is right for these two; the assert only fires when printing assembly. A literal category still prints its comment as before, covered in the test. Fixes #185364 Fixes #185365
…egalization (#213785) fixes #213783 Propagate result types through G_SHUFFLE_VECTOR and G_FATAN2 during post-legalizer type deduction. This prevents wide float vectors from producing integer-typed OpCompositeExtract instructions. Add generic float shuffle and wide atan2 regression coverage. assisted by Copilot (GPT-5.6-Sol)
The ACLE specifies the C intrinsic 'vshll_n to have a valid immediate range size of [0..eltsize] but it has a range of [0..((eltsize*2) - 1)]. Update the range check to match the specified behavior --------- Co-authored-by: Lukacma <Marian.Lukac@arm.com>
…cess in-between phi nodes (#211343) The pass can incorrectly place a load/store within the phi nodes at top of the basic block: https://godbolt.org/z/x5Ec4GWGT This is resolved by updating `replaceHandleWithIndices` to adjust the insertion point when the ptr comes from a phi node. Resolves, in part, #211121 Assisted by: Claude Opus 4.8
…13712) This adds an initial cost model for `masked.compressstore`. The tests are based on `masked_expand_load.ll`, which covers all configurations for the `expand`/`compact` instructions. Note: Right now, we only report valid costs for `masked.compressstore` operations that can be lowered with SVE1 (as that's all we handle ISEL for at the moment).
Replace the generic microtask dispatcher's handwritten function-pointer types and calls with a variadic-template helper and a macro for each supported argument count. The dispatcher still casts each microtask to the exact fixed-arity signature required by platforms such as WebAssembly. This is intended as a refactor only: the existing limit of 15 microtask arguments and the diagnostic for larger argument counts remain unchanged. This reduces the repetitive code and makes future changes to the supported argument range easier to review. Split out from #211071
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )