[pull] main from llvm:main - #1716
Merged
Merged
Conversation
Look through phis, chained selects and the loop back edge to the select itself and fold if no use can observe the sign of zero of the result. This restores the fold for loop-carried and unrolled running min/max values (fcsel -> fminnm on AArch64) lost to the signed zero handling fix in matchSelectPattern (#210077). Reviewers: nikic, dtcxzyw Pull Request: #213133
CallConvLowering classified each function once from its declared signature and reused that classification at every call site. An argument passed through an ellipsis has no entry in the callee's parameter list, so on x86_64 an ellipsis argument that needed an extension attribute, a register coercion, or a byval slot was emitted unchanged whenever the callee's declared parameters happened to need no rewrite, and reported NYI when they did. An ellipsis argument competes for the same registers as a declared one, so the same two-eightbyte record goes in a register pair early in the list and byval once the integer registers are gone. Variadic call sites under the x86_64 driver are now classified from the call's own operand types, and the declared parameter count reaches `llvm::abi::FunctionInfo::create` as its `NumRequired` argument, which is what lets the classifier tell a named argument from one passed through the ellipsis. Today that flag only decides whether a large vector goes in a register, and the CIR type bridge admits no vector types, so the boundary changes no output yet. `prependIndirectCallee` also rebuilt the callee pointer's pointee without `isVarArg`, dropping the ellipsis from a lowered indirect call and with it the vector-register count the x86_64 SysV ABI passes in AL. A void return and a return the ABI drops, such as an empty record, both classify as `Ignore` while needing opposite treatment, so `FunctionClassification` now records which one it was where the classification is produced. `needsRewrite` moves from the CIR rewriter to `mlir/include/mlir/ABI/ABIRewriteContext.h`, over a new per-argument `isPassThrough`, so the pass and the rewriter share one definition. Two things are newly diagnosed rather than newly broken. The operand-count check moves ahead of the pass-through early return, so a call whose operand count disagrees with a `no_proto` callee's declaration is reported instead of lowered with operands the classification never covered. And the indirect-call walk visits `cir::CIRCallOpInterface` rather than only `cir.call`, so a variadic indirect `cir.try_call` is accounted for, and one that needs a rewrite is reported instead of left as written while the signature around it is coerced. A non-variadic indirect `cir.try_call` is filtered out where the walk collects, since rewriting one is unimplemented either way and that gap has nothing to do with the ellipsis. Reported NYI: variadic indirect calls that need a rewrite, variadic calls under the other drivers, and an ellipsis argument whose ABI coercion the type bridge cannot represent.
Pre-RA, the ADDI can contain a frame index rather than a register, which causes `getReg()` to assert. These were missing the `CheckIsRegOperand` that most other fusions have.
This came up during self-build, we are spending a lot of time in some cases looking up using the global symbol table, which does no caching. Previously we'd propagated this in a few places, but this patch removes all uses of SymbolTable::lookup and adds the cache everywhere. This involved changing the tablegen to include it in each of our rewriters, plus the CirAttr lowering everywhere. The only thing we have to take care of is to make sure we invalidate the cache/update the cache whenever we add something with a name (see createLLVMFuncOpIfNotExist). This is NFC, as it isn't observable, other than being a build time improvement.
… reductions types before legalisation (#211043) Pre-AVX512 these will be legalized to wider vector types and might match against other reductions tables. Noticed while working on improving logic reductions, but hit a case with/without popcnt for bool sum reduction patterns - I've added a AVX2 test pass to check the (corrected) costs are still working
Currently, the `lldb_iter` helper is used for providing `__iter__` in
the Python bindings:
```python
def lldb_iter(obj, getsize, getelem):
"""A generator adaptor to support iteration for lldb container objects."""
size = getattr(obj, getsize)
elem = getattr(obj, getelem)
for i in range(size()):
yield elem(i)
```
A type checker or LSP can't see through this function. Currently, that's
no problem, because it doesn't know the return type of any Swig wrapper,
but when we add type annotations (hopefully with Swig 4.5 in #213463),
`__iter__` remains untyped. So iterating through the wrappers won't show
the correct type.
As the functionality is fairly simple, it's easier to inline it. That's
what this PR does. Then a type checker can infer the return type.
Instructions with the single user cost one extract per lane, so they are vectorized after all other roots in the function are exhausted, grouped by the key/subkey pairs. Loads, addresses, compares and the operations, folded away or contracted into the scalar FMA, are excluded. The rejected bundles and the members of the priced minimal nodes are not retried, unless the tree was dropped by the repacking overhead rather than by the members. Reviewers: hiraditya, RKSimon, bababuck Pull Request: #212579
…#213799) `cuda-omp-unsupported-debug-options.cu`'s COMMON-NOT: ...|lldb|... can match against the build directory path that may contain `lldb` in the pathname. Change the test to check the exact cc1 arguments. rdar://183501558
…213770) d3184bc ("[clang] Don't warn on stack_protector_ignore in system macros") added SuppressInSystemMacro to warn_stack_protection_ignore_attribute but only added an LLVM CodeGen test for the stack-protector metadata, so the diagnostic change itself was untested. Add a test using the self-include system_header idiom: the attribute from a system header macro does not warn under -fstack-protector-all, while one written in user code still does. rdar://183962985
A peeled subtract keeps the column sign for its first operand and flips it for the second, so mixed add/sub (fadd/fsub with reassoc) chains linearize into positive and negative operand columns combining as sum(positive) - sum(negative), with subtrahend columns reordered by value family. nuw is now also dropped on non-exact combines when a negated leaf exists, since a regrouped subtrahend sum can exceed the original running total. Reviewers: RKSimon, bababuck, hiraditya Pull Request: #213544
…types (#213733) GCC has special handling for an `sret` of a zero-sized type, match that handling. https://godbolt.org/z/Tcraao7rT It seems unlikely someone is actually relying on this, so I haven't added the abi compatibility logic. Rust however uses zero-sized types quite heavily, so having this work correctly is useful there.
This change replaces some locations where the Neon builtin handling was calling LLVM intrinsics directly for FMA and sqrt operations rather than using the CIR operations. Using the CIR operations will be necessary to get the best constrained FP handling. This will still require setting the fenv attribute on these operations when needed. This change is just the first of a few preliminary changes needed to get us in a good state to add the constrained FP handling. Assisted-by: Cursor / Grok 4.5
Because CIR handles global initializers using a region of the cir.global operation during initial code generation, these initializers don't automatically get the strictfp setup in CIRGenFunction::startFunction. This change adds an attribute to the cir.global operation to track floating-point constraints so that the `strictfp` attribute can be set on the initialization function when it is created during EHABI lowering. Assisted-by: Cursor / various models
DW_OP_LLVM_tag_offset doesn't emit anything into the DWARF expression, so look past it when deciding whether a register expression is complex or whether appendToStack needs to add a dereference. Tested with make check. Assisted with AI.
Fixing build bots after #203084.
…anning (#211408) After `CompilerInstanceWithContext`'s relocation to `DependencyScanningWorker.cpp`, we can use it freely as an implementation engine for TU scanning. This PR does that, and unifies the by-name scanning and TU scanning so they all go through `CompilerInstanceWithContext`. rdar://167034309 --- <sub>Stack created with <a href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
…1409) `DependencyScannerImpl.h/cpp` were created to share code between TU scanning and by-name scanning when they were on different code paths. After unifying the implementation through `CompilerInstanceWithContext`, we no longer need a separate header/cpp pair and all the code are moved into `DependencyScanningWorker.cpp`. This PR is mechnical and does not intend to introduce behavivor changes. --- <sub>Stack created with <a href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
This patch is to fix the following error on z/OS
```
llvm/include/llvm/Support/PerThreadBumpPtrAllocator.h:75:12: error: thread-local storage is not supported for the current target
75 | static thread_local std::vector<AllocatorTy *> Cache;
| ^
```
This helper can be useful in validateTargetOperandClass to get the MCRegisterClass for the MCK_* enum. Migrate the RVY logic to use this new generated function. Checking all register classes instead of just the RVY ones will marginally slow down validateTargetOperandClass, but since this is already a slow path it should not matter. If multiple RegisterOperands share a ParserMatchClass but map to different RegisterClasses, omit them from the generated getRegClassFromMatchKind mapping function. I originally tried to emit an error here, but it turns out there are many targets that rely on sharing the ParserMatchClass with different (but similar) RegisterClasses so ignoring them is safer. Reviewed By: lenary Pull Request: #200453
validateOperandClass() only remapped a RegClassByHwMode operand kind when the actual parsed operand was a register. When the operand was something else entirely (e.g. a bare immediate where a register was expected), this fell through to the generic "Kind <= MCK_LAST_REGISTER" diagnostic check, so we end up with a generic Match_InvalidOperand. No test changes here, but this is needed to avoid diagnostic regressions with the RVY load/store support (PR #177073). Pull Request: #213479
SI_USER is not defined on z/OS and is causing the following build failure. `error: use of undeclared identifier 'SI_USER'` This patch adds a guard to check whether SI_USER is defined before using it.
This fixes 04722a4 (#211409). Buildkite error link: https://buildkite.com/llvm-project/upstream-bazel/builds?commit=04722a4a32e6fec1897cc3cb5a54809fcd8c9cae Co-authored-by: Google Bazel Bot <google-bazel-bot@google.com>
…lling through (#214012) `buildConstructQueue` asserts that decomposition produced output: ```cpp ConstructDecomposition decompose(modOp, semaCtx, eval, compound, clauses); assert(!decompose.output.empty() && "Construct decomposition failed"); ``` Release builds have no check, so the empty queue falls through to the loop below and out to the caller. That is undefined behaviour, and it is reachable — semantics does not catch every directive/clause combination that cannot be decomposed. The result is an intermittent segfault rather than a diagnostic. #211430 is one way in: `allocate` is an OpenMP 5.0 clause, but most directives declare it in `OMP.td` without a minimum version, so semantics accepts it below 5.0 and decomposition then correctly refuses it. flang defaults to OpenMP 3.1, so no unusual flags are needed. The intermittency is what made that report hard to pin down — 40 trials at `-fopenmp-version=31`: | | segfaults | |---|---| | ASLR on | 25/40 | | ASLR off (`setarch -R`) | 0/40 | which is an uninitialized read. After this patch the same input fails deterministically with a located diagnostic, 40/40: ``` error: loc("repro.f90":6:11): OpenMP construct decomposition failed: a clause on 'target teams distribute parallel do' cannot be applied to any of its leaf constructs ``` `genCrashDiag=false` so it exits non-zero without a backtrace, rather than presenting as a compiler crash. This is a hardening fix for the lowering path and is independent of any particular clause: it covers every combination that decomposes empty. It does not make the `allocate` case a *good* diagnostic — that belongs in semantics, and #213980 is the separate change for it. `check-flang` is clean (4808 tests). No test is added: the only in-tree way to reach this path is through the `OMP.td` gap that #213980 fixes, so a test here would encode that gap as expected behaviour and have to be removed when it lands. Happy to add one if you'd prefer. --- Parts of this change were written or audited with Claude Code. I have reviewed all of it and take full responsibility for the contribution. See `llvm/docs/AIToolPolicy.md`.
Lowers peak memory usage on a llvm-dwp invocation from 12.37GB -> 11.19GB. This actually now beats out GNU dwp (11.35GB) at least when the output file is in tmpfs.
#214027) …-amdgpu-tracker-physreg-crash.ll Observed in https://ci.swift.org/job/llvm.org/job/clang-stage1-RA-expensive/job/main/1155 with LLVM_ENABLE_EXPENSIVE_CHECKS enabled, `CodeGen/AMDGPU/schedule-amdgpu-tracker-physreg-crash.ll` fails due to MachineVerifier running, causing an exit 1. Modified to always run the `-verify-machineinstrs`. rdar://184010766
…ts (#214044) The buildbot [lldb-remote-linux-win](https://lab.llvm.org/buildbot/#/builders/197) is red after #212861. #212861 cannot be reverted automatically now because of #212641. So disable TestProcessSaveCoreMinidumpSizeMismatch on Windows hosts to make the buildbot green again.
…fndef (#213045) Several Windows headers/TUs unconditionally `#define WIN32_LEAN_AND_MEAN`. When a build predefines the macro on the compiler command line (e.g. a toolchain that passes -DWIN32_LEAN_AND_MEAN, which clang treats as `#define WIN32_LEAN_AND_MEAN 1`), the differing token lists trigger -Wmacro-redefined, which becomes a hard error under -Werror. Guard the definitions with #ifndef, matching the adjacent NOMINMAX handling and the existing pattern in llvm/lib/Support/rpmalloc/rpmalloc.c. The macro is only a presence flag, so keeping an externally-provided definition is correct. Sites guarded: llvm/include/llvm/Support/Windows/WindowsSupport.h llvm/lib/WindowsDriver/MSVCPaths.cpp clang/lib/Driver/ToolChains/MSVC.cpp lldb/include/lldb/Host/windows/windows.h Meta ran into this internally building lldb on window after a sync with upstream from July commit, so adding guards would allow us to drop a workaround of ignoring the duplicate defines. Error: llvm\include\llvm/Support/Windows/WindowsSupport.h(29,9): error: 'WIN32_LEAN_AND_MEAN' macro redefined [-Werror,-Wmacro-redefined] 29 | #define WIN32_LEAN_AND_MEAN | ^ <command line>(10,9): note: previous definition is here 10 | #define WIN32_LEAN_AND_MEAN 1 | ^ 1 error generated.
When a setcc condition is shared between multiple vselects,
commuteSelect previously bailed out due to the m_OneUse constraint.
Extend it to check all users of the setcc and, if they all benefit from
commuting, invert the condition once with ReplaceAllUsesOfValueWith and
swap all vselect operands.
This fixes a write-mask fusion regression where masked min/max
reductions produced unfused vminps + vmovaps {%k} instead of a single
write-masked vminps {%k}.
…ble as uint8_t. NFC. (#214056) Much cleaner than using the MaskedValueIsZero check. Fix i64 -> xor(i32,i32) parity stage comment as well.
…es (#213806) fixes #213802 Apply the maximum-vector-size rule before the explicit PHI legality rules for type. This way if we see a size 16 vector we split it beforehand. In more precise words this preserves existing legality rules while ensuring shader PHIs wider than four lanes are split first. Assisted by Copilot (GPT 5.6 Sol)
Anchor checks for declare, define, and comdat to the start of lines or another fixture so paths in module headers cannot satisfy or invalidate them. Tested with make check. Assisted by AI.
…m` (#213566) The test had `CHECK` directives that were never executed because the `RUN` line did not pipe output to `FileCheck`. This also drops the first `CHECK` directive in favor of the one below it due to `libclang` now reporting top-level forward class declarations as `ObjCClassRef` cursors which is the modern representation of that same declaration.
#214041) …n 5.1- These two modifiers are structurally identical, but the latter has an exclusive property whereas the former does not. Also, this allows diagnostics to use the modifier name that appears in the spec that corresponds to the version in -fopenmp-version.
…_by_or_null` their own documentation (#212877) Previously the `counted_by`, `counted_by_or_null`, `sized_by`, and `sized_by_or_null` attributes all pointed at a single `Documentation` object (`CountedByDocs`). The generated `AttributeReference.rst` therefore folded all four into one entry whose text described only `counted_by` used on a C99 flexible array member. This was misleading because the attributes differ in important ways that went undocumented: * `counted_by` counts *elements* whereas `sized_by` counts *bytes*. * `counted_by` and `sized_by` require a null pointer to have a zero count/size, while the `_or_null` variants allow a null pointer regardless of the count or size. * `counted_by` may be applied to a flexible array member, but `sized_by`, `counted_by_or_null`, and `sized_by_or_null` apply to pointers only. As a result a reader looking up any of the three non-`counted_by` attributes was shown documentation for a different attribute with different behavior. This change gives `sized_by`, `counted_by_or_null`, and `sized_by_or_null` their own documentation so that each renders as its own entry describing its actual behavior. The substantive text lives in `counted_by` and `sized_by`; the `_or_null` variants are described in terms of their base attribute. While we're here, this patch also: * Restructures `counted_by`'s documentation to explain the attribute's meaning first, with the flexible-array-member material moved into its own sub-section rather than leading the entry. It made sense that FAMs were the leading entry when that was the only context the attribute was supported in, but it's also now supported on pointer fields in a struct. * Discusses how `counted_by` on `void *` is supported as a GNU extension. * Adds two sections on keeping the pointer and count/size value in sync. The two sections are practically identical other than the attribute they discuss and the associated count/size. * For `counted_by`, explains how a pointer to a forward-declared type is allowed as long as the type is complete at the use sites. * For `sized_by`, explains the motivation for using it on types that aren't byte-sized. Assisted-by: Claude Code rdar://136979310
) For cases like: ```c #pragma omp declare_mapper (default: S s) map(s.x, s.p[0:10]) S s1; ... #pragma omp target_enter_data map(present, alloc: s1) ``` After "mapper-expansion", the behavior of the above should be equivalent to: ``` #pragma omp target_enter_data map(present, alloc: s1.x) map(present, alloc: s.p[0:10]) ``` i.e. The `present` map-type needs to be propagated to the map for the pointee of `s.p`. That was not happening prior to this change. We try to limit this to entries that have their own "attach-ptr" (like `s.p[0:10]`) and thus occupy a separate storage block than the base variable for which the mapper is declared (like `s`). Note that we do this only with OpenMP 6.0+, since the wording in 5.2 implies that PRESENT applies only to `s.x` after mapper-expansion. TODO: `PRESENT` bit should also be propagated to non-pointee entries resulting from mapper-expansion, e.g. ```c struct S { int x; int y; int *p; }; S s1; #pragma omp declare_mapper map(s.x, s.y) ... #pragma omp target_enter_data map(present: s1) // present has to be propagated inwards to `s.x, s.y` so that it doesn't // assert that the full `s1` is "present". ``` However we cannot do that until we have removed the remaining uses of PTR_AND_OBJ maps map-type (which is used for byrefs in clang at the moment). --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The verifier was accepting a `cir.global` whose `sym_type` is a function. The LLVM dialect global takes one too, so it survives to LLVM IR translation and crashes instead of reporting an error. `GlobalOp::verify()` now rejects it.
This patch came out of self-build, any constructor temporary called during the construction of an array element was being improperly cleaned up (actually, no terminator?). This patch moves the RunCleanups RAII to do so immediately in the body. The OGCG/LLVM check lines are effectively equal, except OGCG uses a PHI and we are doing so with an iterator loop, but the 'dtor called immediately' is still correct.
This is just to reduce the number of scalar types in the MIR, as the value is between a constant and a store it should have little effect.
A step into a function ran past its prologue only when the pc was exactly the function's first address. A target whose entry point is not a function's first address (such as WebAssembly) is entered past it. This means that the check took every such call for one whose prologue had already run, and the step stopped on the opening brace instead of the first statement. What says the prologue has yet to run is the pc being inside it, which for a target that does enter at the first address is the condition that was there before.
When twiden != 0, LMUL, tail policy, and mask policy from the user are ignored. The tail policy and mask policy are always treated as agnostic. The normal RVV instruction will ignore the twiden parameter. This observation could allow the RVV instruction and xsfmm instruction to share the same configuration instruction. We need to make sure the AVL, SEW, and AltFmt is same between VSETVL and VSETVLTN. --------- Co-authored-by: Luke Lau <luke_lau@icloud.com>
…nsic ops (#211699) This PR introduces dedicated CIR operations for the coroutine intrinsics (`coro.intrinsic.id`, `coro.intrinsic.alloc`, `coro.intrinsic.begin`, `coro.intrinsic.free`, `coro.intrinsic.end`, `coro.intrinsic.size`) instead of emitting them as calls to hand-created builtin functions. LLVM IR already has its own dedicated ops for these cases, so this follows the same pattern at CIR. Along the way this also changes how coroutine builtins are handled in `CIRGenBuiltin`: previously the intrinsics emitted automatically as part of lowering `EmitCoroutineBody` and the ones triggered by an explicit `__builtin_coro_*` call written in user source code were going through the same path. This PR separates the two, since they're conceptually different emission points. New tests covering these builtins are included as well. Also added explicit `errorNYI` messages for the remaining `__builtin_coro_*` cases that aren't implemented yet (`coro_promise`, `coro_resume`, `coro_noop`, `coro_destroy`, `coro_done`, `coro_end`, `coro_suspend`, `coro_align`). @bcardosolopes The ClangIR Progress Report (July 2026) showed 33 hits on the generic coroutine NYI error, with no way to tell which specific builtin was missing these per case messages fix that going forward. A few things intentionally left out of scope for this PR, to keep it reviewable: - LLVM lowering for these new intrinsics follow-up PR. - The different `coro.size` variants for 32/64-bit follow-up PR. - `coro.end`'s current signature is wrong and needs fixing follow-up PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )