Skip to content

sync: rvck88: riscv: Add support for kernel mode vector#135

Merged
xmzzz merged 14 commits into
RVCK-Project:OLK-6.6from
xmzzz:pr-135-kernel-v
Nov 5, 2025
Merged

sync: rvck88: riscv: Add support for kernel mode vector#135
xmzzz merged 14 commits into
RVCK-Project:OLK-6.6from
xmzzz:pr-135-kernel-v

Conversation

@xmzzz
Copy link
Copy Markdown
Contributor

@xmzzz xmzzz commented Nov 2, 2025

Link: RVCK-Project/rvck#88

Updated openeuler_defconfig.

uestc-gr and others added 14 commits November 2, 2025 23:03
mainline inclusion
from mainline-v6.7-rc1
commit b18f7296fbfdb2ad0871f00f3042fc74663d52ac
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

For the sake of coherency, use local labels in assembly when
applicable. This also avoid kprobes being confused when applying a
kprobe since the size of function is computed by checking where the
next visible symbol is located. This might end up in computing some
function size to be way shorter than expected and thus failing to apply
kprobes to the specified offset.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Link: https://lore.kernel.org/r/20231024132655.730417-2-cleger@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7-rc1
commit 4cc0d8a3f109fbdd8100ed88fc9417203a5d5b4e
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

Some data were incorrectly annotated with SYM_FUNC_*() instead of
SYM_DATA_*() ones. Use the correct ones.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Link: https://lore.kernel.org/r/20231024132655.730417-4-cleger@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 	ecd2ada8a5e0b464dab54f71d4ba7bbf5708711f
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

Add kernel_vector_begin() and kernel_vector_end() function declarations
and corresponding definitions in kernel_mode_vector.c

These are needed to wrap uses of vector in kernel mode.

Co-developed-by: Vincent Chen <vincent.chen@sifive.com>
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Signed-off-by: Greentime Hu <greentime.hu@sifive.com>
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-2-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 	956895b9d8f74df015636288a81872c07c4fded3
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

The goal of this patch is to provide full support of Vector in kernel
softirq context. So that some of the crypto alogrithms won't need scalar
fallbacks.

By disabling bottom halves in active kernel-mode Vector, softirq will
not be able to nest on top of any kernel-mode Vector. So, softirq
context is able to use Vector whenever it runs.

After this patch, Vector context cannot start with irqs disabled.
Otherwise local_bh_enable() may run in a wrong context.

Disabling bh is not enough for RT-kernel to prevent preeemption. So
we must disable preemption, which also implies disabling bh on RT.

Related-to: commit 696207d ("arm64/sve: Make kernel FPU protection RT friendly")
Related-to: commit 66c3ec5 ("arm64: neon: Forbid when irqs are disabled")
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-3-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 	c5674d00cacdb1c47c72e19a552fbae401bc3532
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

This patch adds support for vector optimized XOR and it is tested in
qemu.

Co-developed-by: Han-Kuan Chen <hankuan.chen@sifive.com>
Signed-off-by: Han-Kuan Chen <hankuan.chen@sifive.com>
Signed-off-by: Greentime Hu <greentime.hu@sifive.com>
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-4-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 	7df56cbc27e4239807b5d8860f79a7350d63a741
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

User will use its Vector registers only after the kernel really returns
to the userspace. So we can delay restoring Vector registers as long as
we are still running in kernel mode. So, add a thread flag to indicates
the need of restoring Vector and do the restore at the last
arch-specific exit-to-user hook. This save the context restoring cost
when we switch over multiple processes that run V in kernel mode. For
example, if the kernel performs a context swicth from A->B->C, and
returns to C's userspace, then there is no need to restore B's
V-register.

Besides, this also prevents us from repeatedly restoring V context when
executing kernel-mode Vector multiple times.

The cost of this is that we must disable preemption and mark vector as
busy during vstate_{save,restore}. Because then the V context will not
get restored back immediately when a trap-causing context switch happens
in the middle of vstate_{save,restore}.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-5-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit c2a658d419246108c9bf065ec347355de5ba8a05
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

This patch utilizes Vector to perform copy_to_user/copy_from_user. If
Vector is available and the size of copy is large enough for Vector to
perform better than scalar, then direct the kernel to do Vector copies
for userspace. Though the best programming practice for users is to
reduce the copy, this provides a faster variant when copies are
inevitable.

The optimal size for using Vector, copy_to_user_thres, is only a
heuristic for now. We can add DT parsing if people feel the need of
customizing it.

The exception fixup code of the __asm_vector_usercopy must fallback to
the scalar one because accessing user pages might fault, and must be
sleepable. Current kernel-mode Vector does not allow tasks to be
preemptible, so we must disactivate Vector and perform a scalar fallback
in such case.

The original implementation of Vector operations comes from
https://github.com/sifive/sifive-libc, which we agree to contribute to
Linux kernel.

Co-developed-by: Jerry Shih <jerry.shih@sifive.com>
Signed-off-by: Jerry Shih <jerry.shih@sifive.com>
Co-developed-by: Nick Knight <nick.knight@sifive.com>
Signed-off-by: Nick Knight <nick.knight@sifive.com>
Suggested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-6-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit a93fdaf183125fea81f66b9bd756ef5a0c30859e
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

SR_SD summarizes the dirty status of FS/VS/XS. However, the current code
structure does not fully utilize it because each extension specific code
is divided into an individual segment. So remove the SR_SD check for
now.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Reviewed-by: Song Shuai <songshuaishuai@tinylab.org>
Reviewed-by: Guo Ren <guoren@kernel.org>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-7-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
…ore}()

mainline inclusion
from mainline-v6.7
commit d6c78f1ca3e8ec3fd1afa1bc567cdf083e7af9fe
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

riscv_v_vstate_{save,restore}() can operate only on the knowlege of
struct __riscv_v_ext_state, and struct pt_regs. Let the caller decides
which should be passed into the function. Meanwhile, the kernel-mode
Vector is going to introduce another vstate, so this also makes functions
potentially able to be reused.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-8-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 5b6048f2ff710196c85ce14373febe8be5115bbe
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

riscv_v_ctrl_set() should only touch bits within
PR_RISCV_V_VSTATE_CTRL_MASK. So, use the mask when we really set task's
vstate_ctrl.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-9-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit bd446f5df5afab212917f6732ba6442a5e8de85e
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

The allocation size of thread.vstate.datap is always riscv_v_vsize. So
it is possbile to use kmem_cache_* to manage the allocation. This gives
users more information regarding allocation of vector context via
/proc/slabinfo. And it potentially reduces the latency of the first-use
trap because of the allocation caches.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-10-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit 2080ff9493072a94e42b1856d59f5f1bffb761b7
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

Add kernel_vstate to keep track of kernel-mode Vector registers when
trap introduced context switch happens. Also, provide riscv_v_flags to
let context save/restore routine track context status. Context tracking
happens whenever the core starts its in-kernel Vector executions. An
active (dirty) kernel task's V contexts will be saved to memory whenever
a trap-introduced context switch happens. Or, when a softirq, which
happens to nest on top of it, uses Vector. Context retoring happens when
the execution transfer back to the original Kernel context where it
first enable preempt_v.

Also, provide a config CONFIG_RISCV_ISA_V_PREEMPTIVE to give users an
option to disable preemptible kernel-mode Vector at build time. Users
with constraint memory may want to disable this config as preemptible
kernel-mode Vector needs extra space for tracking of per thread's
kernel-mode V context. Or, users might as well want to disable it if all
kernel-mode Vector code is time sensitive and cannot tolerate context
switch overhead.

Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-11-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
mainline inclusion
from mainline-v6.7
commit c27fa53b858b4ee6552a719aa599c250cf98a586
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

The RISC-V Vector specification states in "Appendix D: Calling
Convention for Vector State" [1] that "Executing a system call causes
all caller-saved vector registers (v0-v31, vl, vtype) and vstart to
become unspecified.". In the RISC-V kernel this is called "discarding
the vstate".

Returning from a signal handler via the rt_sigreturn() syscall, vector
discard is also performed. However, this is not an issue since the
vector state should be restored from the sigcontext, and therefore not
care about the vector discard.

The "live state" is the actual vector register in the running context,
and the "vstate" is the vector state of the task. A dirty live state,
means that the vstate and live state are not in synch.

When vectorized user_from_copy() was introduced, an bug sneaked in at
the restoration code, related to the discard of the live state.

An example when this go wrong:

  1. A userland application is executing vector code
  2. The application receives a signal, and the signal handler is
     entered.
  3. The application returns from the signal handler, using the
     rt_sigreturn() syscall.
  4. The live vector state is discarded upon entering the
     rt_sigreturn(), and the live state is marked as "dirty", indicating
     that the live state need to be synchronized with the current
     vstate.
  5. rt_sigreturn() restores the vstate, except the Vector registers,
     from the sigcontext
  6. rt_sigreturn() restores the Vector registers, from the sigcontext,
     and now the vectorized user_from_copy() is used. The dirty live
     state from the discard is saved to the vstate, making the vstate
     corrupt.
  7. rt_sigreturn() returns to the application, which crashes due to
     corrupted vstate.

Note that the vectorized user_from_copy() is invoked depending on the
value of CONFIG_RISCV_ISA_V_UCOPY_THRESHOLD. Default is 768, which
means that vlen has to be larger than 128b for this bug to trigger.

The fix is simply to mark the live state as non-dirty/clean prior
performing the vstate restore.

Link: https://github.com/riscv/riscv-isa-manual/releases/download/riscv-isa-release-8abdb41-2024-03-26/unpriv-isa-asciidoc.pdf # [1]
Reported-by: Charlie Jenkins <charlie@rivosinc.com>
Reported-by: Vineet Gupta <vgupta@kernel.org>
Fixes: c2a658d41924 ("riscv: lib: vectorize copy_to_user/copy_from_user")
Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Reviewed-by: Andy Chiu <andy.chiu@sifive.com>
Tested-by: Vineet Gupta <vineetg@rivosinc.com>
Link: https://lore.kernel.org/r/20240403072638.567446-1-bjorn@kernel.org
Cc: stable@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Gao Rui <gao.rui@zte.com.cn>
Signed-off-by: Yanteng Si <si.yanteng@linux.dev>
Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
riscv inclusion
category: feature
bugzilla: RVCK-Project/rvck#86

--------------------------------

Enable the following configs:
CONFIG_RISCV_ISA_V

The following configs have been merged in the previous PR and are now
enabled together:
CONFIG_SERIAL_EARLYCON_RISCV_SBI
CONFIG_PARAVIRT
CONFIG_PARAVIRT_TIME_ACCOUNTING

Then update openeuler_defconfig:
make save_oedefconfig
make update_oedefconfig

Build and boot testing passed.

Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Nov 2, 2025


开始测试 log: https://github.com/RVCK-Project/rvck-olk/actions/runs/19014892493

参数解析结果
args value
repository RVCK-Project/rvck-olk
head ref pull/135/head
base ref OLK-6.6
LAVA repo RVCK-Project/lavaci
LAVA Template lava-job-template/qemu/qemu-ltp.yaml
Testcase path lava-testcases/common-test/ltp/ltp.yaml

测试完成

详细结果:

RVCK result

check result
kunit-test success
kernel-build failure
lava-trigger skipped
check-patch success

Kunit Test Result

kunit test failed

Kernel Build Result

Kernel build failed.

Check Patch Result

Total Errors 0
Total Warnings 4

测试完成

详细结果:

RVCK result

check result
kunit-test success
kernel-build failure
lava-trigger skipped
check-patch success

Kunit Test Result

kunit test failed

Kernel Build Result

Kernel build failed.

Check Patch Result

Total Errors 0
Total Warnings 4

测试完成

详细结果:

RVCK result

check result
kunit-test success
kernel-build abandoned
lava-trigger skipped
check-patch success

Kunit Test Result

kunit test failed

Check Patch Result

Total Errors 0
Total Warnings 4

测试完成

详细结果:

RVCK result

check result
kunit-test success
kernel-build success
lava-trigger success
check-patch success

Kunit Test Result

kunit test failed

Kernel Build Result

Kernel build succeeded: RVCK-Project/rvck-olk/135/

60fdf22b4a6439c0bfb77f270cad563d /srv/guix_result/3d3ac27d2217af7ca3a1763fd6a4698a702cbfda/Image
5de3d3bd8c5eac75cc9ece0065323793 /root/initramfs.img

LAVA Check

args:

result:

Lava check done! lava log: https://lava.oerv.ac.cn/scheduler/job/911

lava result count: [fail]: 17, [pass]: 1589, [skip]: 293

Check Patch Result

Total Errors 0
Total Warnings 4

@wangliu-iscas
Copy link
Copy Markdown
Collaborator

wangliu-iscas commented Nov 4, 2025

/check lava_template=lava-job-template/qemu/qemu-mmtests.yaml testcase_path=lava-testcases/common-test/MMTests/MMTests.yaml


开始测试 log: https://github.com/RVCK-Project/rvck-olk/actions/runs/19062535760

参数解析结果
args value
repository RVCK-Project/rvck-olk
head ref pull/135/head
base ref OLK-6.6
LAVA repo RVCK-Project/lavaci
LAVA Template lava-job-template/qemu/qemu-mmtests.yaml
Testcase path lava-testcases/common-test/MMTests/MMTests.yaml

测试完成

详细结果:

RVCK result

check result
kunit-test success
kernel-build success
lava-trigger cancelled
check-patch success

Kunit Test Result

kunit test failed

Kernel Build Result

Kernel build succeeded: RVCK-Project/rvck-olk/135_3484571858/

0232d94567dc385725913636b6c76308 /srv/guix_result/3d3ac27d2217af7ca3a1763fd6a4698a702cbfda/Image
5de3d3bd8c5eac75cc9ece0065323793 /root/initramfs.img

LAVA Check

args:

result:

Lava check done! lava log: https://lava.oerv.ac.cn/scheduler/job/914

lava result count: [fail]: 12, [pass]: 68

Check Patch Result

Total Errors 0
Total Warnings 4

@xmzzz xmzzz merged commit 83c30d7 into RVCK-Project:OLK-6.6 Nov 5, 2025
67 of 70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants