* [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations
@ 2026-10-16 8:10 Tiezhu Yang
2026-10-16 8:10 ` [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection Tiezhu Yang
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: Tiezhu Yang @ 2026-10-16 8:10 UTC (permalink / raw)
To: Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel
v3:
-- Add a prerequisite patch to detect CPUCFG2_DIV32 feature.
-- Fix a hardware compatibility bug by checking cpu_has_div32
before emitting native 32-bit div/mod instructions.
This version is based on v7.3-rc7 and has been tested on Loongson-3A6000
and Loongson-3A5000 physical machines.
All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32)
and Loongson-3A5000 (without DIV32), here are the test results:
$ sudo modprobe test_bpf
$ dmesg -t | grep Summary
test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed]
test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED
For the BPF selftests test_verifier and test_progs, there are no any
regressions.
Tiezhu Yang (3):
LoongArch: Add CPUCFG2 DIV32 feature detection
LoongArch: BPF: Clean up and decouple is32 parameter from move_imm()
LoongArch: BPF: Optimize 32-bit operations using native instructions
arch/loongarch/include/asm/cpu-features.h | 1 +
arch/loongarch/include/asm/cpu.h | 2 +
arch/loongarch/include/asm/inst.h | 6 +
arch/loongarch/include/uapi/asm/hwcap.h | 1 +
arch/loongarch/kernel/cpu-probe.c | 4 +
arch/loongarch/kernel/proc.c | 2 +
arch/loongarch/net/bpf_jit.c | 381 ++++++++++++----------
arch/loongarch/net/bpf_jit.h | 25 +-
8 files changed, 234 insertions(+), 188 deletions(-)
--
2.42.0
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection 2026-10-16 8:10 [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang @ 2026-10-16 8:10 ` Tiezhu Yang 2026-10-20 10:48 ` Huacai Chen 2026-10-16 8:10 ` [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() Tiezhu Yang ` (2 subsequent siblings) 3 siblings, 1 reply; 15+ messages in thread From: Tiezhu Yang @ 2026-10-16 8:10 UTC (permalink / raw) To: Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel The LoongArch Architecture Manual defines a hardware capability bit CPUCFG2_DIV32 (bit 26 of CPUCFG 0x2). When this bit is set to 1, it indicates that on 64-bit machines, the division and mod instructions (DIV.W[U] and MOD.W[U]) will ignore the upper 32 bits of the input registers and only compute based on the lower 32 bits. On certain CPU cores without this feature, these 32-bit instructions require both input operands to be properly sign-extended. Otherwise, the hardware execution results can be randomly meaningless values. Add CPU_FEATURE_DIV32 to detect and export this hardware capability via elf_hwcap and cpu_has_div32. This provides the infrastructure for subsystems like BPF JIT to safely choose either native 32-bit instructions or fallback instruction sequences based on the actual CPU hardware capabilities. Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> --- arch/loongarch/include/asm/cpu-features.h | 1 + arch/loongarch/include/asm/cpu.h | 2 ++ arch/loongarch/include/uapi/asm/hwcap.h | 1 + arch/loongarch/kernel/cpu-probe.c | 4 ++++ arch/loongarch/kernel/proc.c | 2 ++ 5 files changed, 10 insertions(+) diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h index 62059c5551b9..3e7cf1987171 100644 --- a/arch/loongarch/include/asm/cpu-features.h +++ b/arch/loongarch/include/asm/cpu-features.h @@ -69,5 +69,6 @@ #define cpu_has_msgint cpu_opt(LOONGARCH_CPU_MSGINT) #define cpu_has_avecint cpu_opt(LOONGARCH_CPU_AVECINT) #define cpu_has_redirectint cpu_opt(LOONGARCH_CPU_REDIRECTINT) +#define cpu_has_div32 cpu_opt(LOONGARCH_CPU_DIV32) #endif /* __ASM_CPU_FEATURES_H */ diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h index 91b96938861e..d1cf5dd5eceb 100644 --- a/arch/loongarch/include/asm/cpu.h +++ b/arch/loongarch/include/asm/cpu.h @@ -127,6 +127,7 @@ static inline char *id_to_core_name(unsigned int id) #define CPU_FEATURE_MSGINT 31 /* CPU has MSG interrupt */ #define CPU_FEATURE_AVECINT 32 /* CPU has AVEC interrupt */ #define CPU_FEATURE_REDIRECTINT 33 /* CPU has interrupt remapping */ +#define CPU_FEATURE_DIV32 34 /* CPU has DIV32 feature */ #define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG) #define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM) @@ -162,5 +163,6 @@ static inline char *id_to_core_name(unsigned int id) #define LOONGARCH_CPU_MSGINT BIT_ULL(CPU_FEATURE_MSGINT) #define LOONGARCH_CPU_AVECINT BIT_ULL(CPU_FEATURE_AVECINT) #define LOONGARCH_CPU_REDIRECTINT BIT_ULL(CPU_FEATURE_REDIRECTINT) +#define LOONGARCH_CPU_DIV32 BIT_ULL(CPU_FEATURE_DIV32) #endif /* _ASM_CPU_H */ diff --git a/arch/loongarch/include/uapi/asm/hwcap.h b/arch/loongarch/include/uapi/asm/hwcap.h index 90e96113ba51..00bf93683c74 100644 --- a/arch/loongarch/include/uapi/asm/hwcap.h +++ b/arch/loongarch/include/uapi/asm/hwcap.h @@ -20,5 +20,6 @@ #define HWCAP_LOONGARCH_LSPW (1 << 14) #define HWCAP_LOONGARCH_SCQ (1 << 15) #define HWCAP_LOONGARCH_LAM_BH (1 << 16) +#define HWCAP_LOONGARCH_DIV32 (1 << 17) #endif /* _UAPI_ASM_HWCAP_H */ diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c index 74d31f260dfd..c50fcba805f7 100644 --- a/arch/loongarch/kernel/cpu-probe.c +++ b/arch/loongarch/kernel/cpu-probe.c @@ -236,6 +236,10 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) elf_hwcap |= HWCAP_LOONGARCH_LBT_MIPS; } #endif + if (config & CPUCFG2_DIV32) { + c->options |= LOONGARCH_CPU_DIV32; + elf_hwcap |= HWCAP_LOONGARCH_DIV32; + } config = read_cpucfg(LOONGARCH_CPUCFG6); if (config & CPUCFG6_PMP) diff --git a/arch/loongarch/kernel/proc.c b/arch/loongarch/kernel/proc.c index d4ce5b585453..c8d6f397de55 100644 --- a/arch/loongarch/kernel/proc.c +++ b/arch/loongarch/kernel/proc.c @@ -94,6 +94,8 @@ static int show_cpuinfo(struct seq_file *m, void *v) seq_puts(m, " lbt_arm"); if (cpu_has_lbt_mips) seq_puts(m, " lbt_mips"); + if (cpu_has_div32) + seq_puts(m, " div32"); seq_puts(m, "\n"); seq_printf(m, "Hardware Watchpoint\t: %s", str_yes_no(cpu_has_watch)); -- 2.42.0 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection 2026-10-16 8:10 ` [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection Tiezhu Yang @ 2026-10-20 10:48 ` Huacai Chen 0 siblings, 0 replies; 15+ messages in thread From: Huacai Chen @ 2026-10-20 10:48 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Hengqi Chen, loongarch, bpf, linux-kernel This one is applied, thanks. Huacai On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > The LoongArch Architecture Manual defines a hardware capability bit > CPUCFG2_DIV32 (bit 26 of CPUCFG 0x2). When this bit is set to 1, it > indicates that on 64-bit machines, the division and mod instructions > (DIV.W[U] and MOD.W[U]) will ignore the upper 32 bits of the input > registers and only compute based on the lower 32 bits. > > On certain CPU cores without this feature, these 32-bit instructions > require both input operands to be properly sign-extended. Otherwise, > the hardware execution results can be randomly meaningless values. > > Add CPU_FEATURE_DIV32 to detect and export this hardware capability > via elf_hwcap and cpu_has_div32. This provides the infrastructure > for subsystems like BPF JIT to safely choose either native 32-bit > instructions or fallback instruction sequences based on the actual > CPU hardware capabilities. > > Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> > --- > arch/loongarch/include/asm/cpu-features.h | 1 + > arch/loongarch/include/asm/cpu.h | 2 ++ > arch/loongarch/include/uapi/asm/hwcap.h | 1 + > arch/loongarch/kernel/cpu-probe.c | 4 ++++ > arch/loongarch/kernel/proc.c | 2 ++ > 5 files changed, 10 insertions(+) > > diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h > index 62059c5551b9..3e7cf1987171 100644 > --- a/arch/loongarch/include/asm/cpu-features.h > +++ b/arch/loongarch/include/asm/cpu-features.h > @@ -69,5 +69,6 @@ > #define cpu_has_msgint cpu_opt(LOONGARCH_CPU_MSGINT) > #define cpu_has_avecint cpu_opt(LOONGARCH_CPU_AVECINT) > #define cpu_has_redirectint cpu_opt(LOONGARCH_CPU_REDIRECTINT) > +#define cpu_has_div32 cpu_opt(LOONGARCH_CPU_DIV32) > > #endif /* __ASM_CPU_FEATURES_H */ > diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h > index 91b96938861e..d1cf5dd5eceb 100644 > --- a/arch/loongarch/include/asm/cpu.h > +++ b/arch/loongarch/include/asm/cpu.h > @@ -127,6 +127,7 @@ static inline char *id_to_core_name(unsigned int id) > #define CPU_FEATURE_MSGINT 31 /* CPU has MSG interrupt */ > #define CPU_FEATURE_AVECINT 32 /* CPU has AVEC interrupt */ > #define CPU_FEATURE_REDIRECTINT 33 /* CPU has interrupt remapping */ > +#define CPU_FEATURE_DIV32 34 /* CPU has DIV32 feature */ > > #define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG) > #define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM) > @@ -162,5 +163,6 @@ static inline char *id_to_core_name(unsigned int id) > #define LOONGARCH_CPU_MSGINT BIT_ULL(CPU_FEATURE_MSGINT) > #define LOONGARCH_CPU_AVECINT BIT_ULL(CPU_FEATURE_AVECINT) > #define LOONGARCH_CPU_REDIRECTINT BIT_ULL(CPU_FEATURE_REDIRECTINT) > +#define LOONGARCH_CPU_DIV32 BIT_ULL(CPU_FEATURE_DIV32) > > #endif /* _ASM_CPU_H */ > diff --git a/arch/loongarch/include/uapi/asm/hwcap.h b/arch/loongarch/include/uapi/asm/hwcap.h > index 90e96113ba51..00bf93683c74 100644 > --- a/arch/loongarch/include/uapi/asm/hwcap.h > +++ b/arch/loongarch/include/uapi/asm/hwcap.h > @@ -20,5 +20,6 @@ > #define HWCAP_LOONGARCH_LSPW (1 << 14) > #define HWCAP_LOONGARCH_SCQ (1 << 15) > #define HWCAP_LOONGARCH_LAM_BH (1 << 16) > +#define HWCAP_LOONGARCH_DIV32 (1 << 17) > > #endif /* _UAPI_ASM_HWCAP_H */ > diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c > index 74d31f260dfd..c50fcba805f7 100644 > --- a/arch/loongarch/kernel/cpu-probe.c > +++ b/arch/loongarch/kernel/cpu-probe.c > @@ -236,6 +236,10 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) > elf_hwcap |= HWCAP_LOONGARCH_LBT_MIPS; > } > #endif > + if (config & CPUCFG2_DIV32) { > + c->options |= LOONGARCH_CPU_DIV32; > + elf_hwcap |= HWCAP_LOONGARCH_DIV32; > + } > > config = read_cpucfg(LOONGARCH_CPUCFG6); > if (config & CPUCFG6_PMP) > diff --git a/arch/loongarch/kernel/proc.c b/arch/loongarch/kernel/proc.c > index d4ce5b585453..c8d6f397de55 100644 > --- a/arch/loongarch/kernel/proc.c > +++ b/arch/loongarch/kernel/proc.c > @@ -94,6 +94,8 @@ static int show_cpuinfo(struct seq_file *m, void *v) > seq_puts(m, " lbt_arm"); > if (cpu_has_lbt_mips) > seq_puts(m, " lbt_mips"); > + if (cpu_has_div32) > + seq_puts(m, " div32"); > seq_puts(m, "\n"); > > seq_printf(m, "Hardware Watchpoint\t: %s", str_yes_no(cpu_has_watch)); > -- > 2.42.0 > ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() 2026-10-16 8:10 [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 2026-10-16 8:10 ` [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection Tiezhu Yang @ 2026-10-16 8:10 ` Tiezhu Yang 2026-10-27 1:36 ` Hengqi Chen 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang 2026-10-22 7:43 ` [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 3 siblings, 1 reply; 15+ messages in thread From: Tiezhu Yang @ 2026-10-16 8:10 UTC (permalink / raw) To: Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel The current move_imm() relies on the 'is32' parameter for ALU32 and JMP32, forcing the helper to internally clear the upper 32 bits. However, this is redundant for 64-bit immediate loads. Furthermore, it is even undesirable for load/store signed address offset computations. Clean up move_imm() by removing the 'is32' parameter which allows it to focus on loading values, while moving the zero-extension logic outward to the explicit callers. For 64-bit and load-store operations, just drop the parameter. This is preparation for later patch, no functional changes. Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> --- arch/loongarch/net/bpf_jit.c | 102 +++++++++++++++++++---------------- arch/loongarch/net/bpf_jit.h | 7 +-- 2 files changed, 58 insertions(+), 51 deletions(-) diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c index e78eb582c300..77cd96a6f84c 100644 --- a/arch/loongarch/net/bpf_jit.c +++ b/arch/loongarch/net/bpf_jit.c @@ -194,7 +194,7 @@ static void build_prologue(struct jit_ctx *ctx) ctx->stack_size = stack_adjust; if (ctx->arena_vm_start) - move_imm(ctx, REG_ARENA, ctx->arena_vm_start, false); + move_imm(ctx, REG_ARENA, ctx->arena_vm_start); } static void __build_epilogue(struct jit_ctx *ctx, bool is_tail_call) @@ -339,7 +339,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) static void emit_store_stack_imm64(struct jit_ctx *ctx, int reg, int stack_off, u64 imm64) { - move_imm(ctx, reg, imm64, false); + move_imm(ctx, reg, imm64); emit_insn(ctx, std, reg, LOONGARCH_GPR_FP, stack_off); } @@ -355,7 +355,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) const s32 imm = insn->imm; const bool isdw = BPF_SIZE(insn->code) == BPF_DW; - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, addd, t1, dst, t1); move_reg(ctx, t3, src); @@ -528,7 +528,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, ldbu, dst, src, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, ldxbu, dst, src, t1); } break; @@ -536,7 +536,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, ldhu, dst, src, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, ldxhu, dst, src, t1); } break; @@ -544,7 +544,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, ldwu, dst, src, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, ldxwu, dst, src, t1); } break; @@ -552,7 +552,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, ldd, dst, src, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, ldxd, dst, src, t1); } break; @@ -567,7 +567,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, stb, src, dst, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, stxb, src, dst, t1); } break; @@ -575,7 +575,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, sth, src, dst, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, stxh, src, dst, t1); } break; @@ -583,7 +583,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, stw, src, dst, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, stxw, src, dst, t1); } break; @@ -591,7 +591,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) if (is_signed_imm12(off)) { emit_insn(ctx, std, src, dst, off); } else { - move_imm(ctx, t1, off, false); + move_imm(ctx, t1, off); emit_insn(ctx, stxd, src, dst, t1); } break; @@ -716,7 +716,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (insn_is_cast_user(insn)) { move_reg(ctx, t1, src); emit_zext_32(ctx, t1, true); - move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32, false); + move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); emit_insn(ctx, or, t1, dst, t1); move_reg(ctx, dst, t1); @@ -753,7 +753,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = imm */ case BPF_ALU | BPF_MOV | BPF_K: case BPF_ALU64 | BPF_MOV | BPF_K: - move_imm(ctx, dst, imm, is32); + move_imm(ctx, dst, imm); + emit_zext_32(ctx, dst, is32); break; /* dst = dst + src */ @@ -769,7 +770,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_signed_imm12(imm)) { emit_insn(ctx, addid, dst, dst, imm); } else { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, addd, dst, dst, t1); } emit_zext_32(ctx, dst, is32); @@ -788,7 +790,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_signed_imm12(-imm)) { emit_insn(ctx, addid, dst, dst, -imm); } else { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, subd, dst, dst, t1); } emit_zext_32(ctx, dst, is32); @@ -804,7 +807,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst * imm */ case BPF_ALU | BPF_MUL | BPF_K: case BPF_ALU64 | BPF_MUL | BPF_K: - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, muld, dst, dst, t1); emit_zext_32(ctx, dst, is32); break; @@ -831,12 +835,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_ALU | BPF_DIV | BPF_K: case BPF_ALU64 | BPF_DIV | BPF_K: if (!off) { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_zext_32(ctx, dst, is32); emit_insn(ctx, divdu, dst, dst, t1); emit_zext_32(ctx, dst, is32); } else { - move_imm(ctx, t1, imm, false); + move_imm(ctx, t1, imm); emit_sext_32(ctx, t1, is32); emit_sext_32(ctx, dst, is32); emit_insn(ctx, divd, dst, dst, t1); @@ -866,12 +871,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_ALU | BPF_MOD | BPF_K: case BPF_ALU64 | BPF_MOD | BPF_K: if (!off) { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_zext_32(ctx, dst, is32); emit_insn(ctx, moddu, dst, dst, t1); emit_zext_32(ctx, dst, is32); } else { - move_imm(ctx, t1, imm, false); + move_imm(ctx, t1, imm); emit_sext_32(ctx, t1, is32); emit_sext_32(ctx, dst, is32); emit_insn(ctx, modd, dst, dst, t1); @@ -899,7 +905,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, andi, dst, dst, imm); } else { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, and, dst, dst, t1); } emit_zext_32(ctx, dst, is32); @@ -918,7 +925,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, ori, dst, dst, imm); } else { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, or, dst, dst, t1); } emit_zext_32(ctx, dst, is32); @@ -937,7 +945,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, xori, dst, dst, imm); } else { - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, xor, dst, dst, t1); } emit_zext_32(ctx, dst, is32); @@ -1097,7 +1106,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP32 | BPF_JSLE | BPF_K: jmp_offset = bpf2la_offset(i, off, ctx); if (imm) { - move_imm(ctx, t1, imm, false); + move_imm(ctx, t1, imm); tm = t1; } else { /* If imm is 0, simply use zero register. */ @@ -1129,7 +1138,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP | BPF_JSET | BPF_K: case BPF_JMP32 | BPF_JSET | BPF_K: jmp_offset = bpf2la_offset(i, off, ctx); - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); + emit_zext_32(ctx, t1, is32); emit_insn(ctx, and, t1, dst, t1); emit_zext_32(ctx, t1, is32); if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) @@ -1227,7 +1237,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (bpf_pseudo_func(insn)) move_addr(ctx, dst, imm64); else - move_imm(ctx, dst, imm64, is32); + move_imm(ctx, dst, imm64); return 1; } @@ -1268,7 +1278,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext else emit_insn(ctx, ldbu, dst, src, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); if (sign_extend) emit_insn(ctx, ldxb, dst, src, t1); else @@ -1282,7 +1292,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext else emit_insn(ctx, ldhu, dst, src, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); if (sign_extend) emit_insn(ctx, ldxh, dst, src, t1); else @@ -1296,7 +1306,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext else emit_insn(ctx, ldwu, dst, src, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); if (sign_extend) emit_insn(ctx, ldxw, dst, src, t1); else @@ -1304,7 +1314,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext } break; case BPF_DW: - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); emit_insn(ctx, ldxd, dst, src, t1); break; } @@ -1331,42 +1341,42 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext switch (BPF_SIZE(code)) { case BPF_B: - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); if (is_signed_imm12(off)) { emit_insn(ctx, stb, t1, dst, off); } else { - move_imm(ctx, t2, off, is32); + move_imm(ctx, t2, off); emit_insn(ctx, stxb, t1, dst, t2); } break; case BPF_H: - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); if (is_signed_imm12(off)) { emit_insn(ctx, sth, t1, dst, off); } else { - move_imm(ctx, t2, off, is32); + move_imm(ctx, t2, off); emit_insn(ctx, stxh, t1, dst, t2); } break; case BPF_W: - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); if (is_signed_imm12(off)) { emit_insn(ctx, stw, t1, dst, off); } else if (is_signed_imm14(off)) { emit_insn(ctx, stptrw, t1, dst, off); } else { - move_imm(ctx, t2, off, is32); + move_imm(ctx, t2, off); emit_insn(ctx, stxw, t1, dst, t2); } break; case BPF_DW: - move_imm(ctx, t1, imm, is32); + move_imm(ctx, t1, imm); if (is_signed_imm12(off)) { emit_insn(ctx, std, t1, dst, off); } else if (is_signed_imm14(off)) { emit_insn(ctx, stptrd, t1, dst, off); } else { - move_imm(ctx, t2, off, is32); + move_imm(ctx, t2, off); emit_insn(ctx, stxd, t1, dst, t2); } break; @@ -1397,7 +1407,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_signed_imm12(off)) { emit_insn(ctx, stb, src, dst, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); emit_insn(ctx, stxb, src, dst, t1); } break; @@ -1405,7 +1415,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_signed_imm12(off)) { emit_insn(ctx, sth, src, dst, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); emit_insn(ctx, stxh, src, dst, t1); } break; @@ -1415,7 +1425,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext } else if (is_signed_imm14(off)) { emit_insn(ctx, stptrw, src, dst, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); emit_insn(ctx, stxw, src, dst, t1); } break; @@ -1425,7 +1435,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext } else if (is_signed_imm14(off)) { emit_insn(ctx, stptrd, src, dst, off); } else { - move_imm(ctx, t1, off, is32); + move_imm(ctx, t1, off); emit_insn(ctx, stxd, src, dst, t1); } break; @@ -1537,7 +1547,7 @@ static int emit_jump_and_link(struct jit_ctx *ctx, u8 rd, u64 target) return -EFAULT; } - move_imm(ctx, LOONGARCH_GPR_T1, target, false); + move_imm(ctx, LOONGARCH_GPR_T1, target); emit_insn(ctx, jirl, rd, LOONGARCH_GPR_T1, 0); return 0; @@ -1734,7 +1744,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, -run_ctx_off + cookie_off); /* arg1: prog */ - move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p); /* arg2: &run_ctx */ emit_insn(ctx, addid, LOONGARCH_GPR_A1, LOONGARCH_GPR_FP, -run_ctx_off); ret = emit_call(ctx, (const u64)bpf_trampoline_enter(p)); @@ -1755,7 +1765,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, /* arg1: &args_off */ emit_insn(ctx, addid, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -args_off); if (!p->jited) - move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi, false); + move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi); ret = emit_call(ctx, (const u64)p->bpf_func); if (ret) return ret; @@ -1772,7 +1782,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, } /* arg1: prog */ - move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p); /* arg2: prog start time */ move_reg(ctx, LOONGARCH_GPR_A1, LOONGARCH_GPR_S1); /* arg3: &run_ctx */ diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h index bb58c42c2f2a..57a98f8a055b 100644 --- a/arch/loongarch/net/bpf_jit.h +++ b/arch/loongarch/net/bpf_jit.h @@ -137,7 +137,7 @@ static inline void move_addr(struct jit_ctx *ctx, enum loongarch_gpr rd, u64 add emit_insn(ctx, lu52id, rd, rd, imm_63_52); } -static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm, bool is32) +static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm) { long imm_11_0, imm_31_12, imm_51_32, imm_63_52, imm_51_0, imm_51_31; @@ -150,7 +150,7 @@ static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm /* addiw rd, $zero, imm_11_0 */ if (is_signed_imm12(imm)) { emit_insn(ctx, addiw, rd, LOONGARCH_GPR_ZERO, imm); - goto zext; + return; } /* ori rd, $zero, imm_11_0 */ @@ -195,9 +195,6 @@ static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm if (!is_signed_imm52(imm)) emit_insn(ctx, lu52id, rd, rd, imm_63_52); } - -zext: - emit_zext_32(ctx, rd, is32); } static inline void move_reg(struct jit_ctx *ctx, enum loongarch_gpr rd, -- 2.42.0 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() 2026-10-16 8:10 ` [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() Tiezhu Yang @ 2026-10-27 1:36 ` Hengqi Chen 0 siblings, 0 replies; 15+ messages in thread From: Hengqi Chen @ 2026-10-27 1:36 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Huacai Chen, loongarch, bpf, linux-kernel On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > The current move_imm() relies on the 'is32' parameter for ALU32 and JMP32, > forcing the helper to internally clear the upper 32 bits. However, this is > redundant for 64-bit immediate loads. Furthermore, it is even undesirable > for load/store signed address offset computations. > > Clean up move_imm() by removing the 'is32' parameter which allows it to > focus on loading values, while moving the zero-extension logic outward to > the explicit callers. For 64-bit and load-store operations, just drop the > parameter. > > This is preparation for later patch, no functional changes. > > Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> > --- Acked-by: Hengqi Chen <hengqi.chen@gmail.com> > arch/loongarch/net/bpf_jit.c | 102 +++++++++++++++++++---------------- > arch/loongarch/net/bpf_jit.h | 7 +-- > 2 files changed, 58 insertions(+), 51 deletions(-) > > diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c > index e78eb582c300..77cd96a6f84c 100644 > --- a/arch/loongarch/net/bpf_jit.c > +++ b/arch/loongarch/net/bpf_jit.c > @@ -194,7 +194,7 @@ static void build_prologue(struct jit_ctx *ctx) > ctx->stack_size = stack_adjust; > > if (ctx->arena_vm_start) > - move_imm(ctx, REG_ARENA, ctx->arena_vm_start, false); > + move_imm(ctx, REG_ARENA, ctx->arena_vm_start); > } > > static void __build_epilogue(struct jit_ctx *ctx, bool is_tail_call) > @@ -339,7 +339,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) > > static void emit_store_stack_imm64(struct jit_ctx *ctx, int reg, int stack_off, u64 imm64) > { > - move_imm(ctx, reg, imm64, false); > + move_imm(ctx, reg, imm64); > emit_insn(ctx, std, reg, LOONGARCH_GPR_FP, stack_off); > } > > @@ -355,7 +355,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > const s32 imm = insn->imm; > const bool isdw = BPF_SIZE(insn->code) == BPF_DW; > > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, addd, t1, dst, t1); > move_reg(ctx, t3, src); > > @@ -528,7 +528,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, ldbu, dst, src, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, ldxbu, dst, src, t1); > } > break; > @@ -536,7 +536,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, ldhu, dst, src, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, ldxhu, dst, src, t1); > } > break; > @@ -544,7 +544,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, ldwu, dst, src, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, ldxwu, dst, src, t1); > } > break; > @@ -552,7 +552,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, ldd, dst, src, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, ldxd, dst, src, t1); > } > break; > @@ -567,7 +567,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, stb, src, dst, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxb, src, dst, t1); > } > break; > @@ -575,7 +575,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, sth, src, dst, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxh, src, dst, t1); > } > break; > @@ -583,7 +583,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, stw, src, dst, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxw, src, dst, t1); > } > break; > @@ -591,7 +591,7 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > if (is_signed_imm12(off)) { > emit_insn(ctx, std, src, dst, off); > } else { > - move_imm(ctx, t1, off, false); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxd, src, dst, t1); > } > break; > @@ -716,7 +716,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (insn_is_cast_user(insn)) { > move_reg(ctx, t1, src); > emit_zext_32(ctx, t1, true); > - move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32, false); > + move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); > emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); > emit_insn(ctx, or, t1, dst, t1); > move_reg(ctx, dst, t1); > @@ -753,7 +753,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = imm */ > case BPF_ALU | BPF_MOV | BPF_K: > case BPF_ALU64 | BPF_MOV | BPF_K: > - move_imm(ctx, dst, imm, is32); > + move_imm(ctx, dst, imm); > + emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst + src */ > @@ -769,7 +770,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_signed_imm12(imm)) { > emit_insn(ctx, addid, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, addd, dst, dst, t1); > } > emit_zext_32(ctx, dst, is32); > @@ -788,7 +790,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_signed_imm12(-imm)) { > emit_insn(ctx, addid, dst, dst, -imm); > } else { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, subd, dst, dst, t1); > } > emit_zext_32(ctx, dst, is32); > @@ -804,7 +807,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst * imm */ > case BPF_ALU | BPF_MUL | BPF_K: > case BPF_ALU64 | BPF_MUL | BPF_K: > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, muld, dst, dst, t1); > emit_zext_32(ctx, dst, is32); > break; > @@ -831,12 +835,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU | BPF_DIV | BPF_K: > case BPF_ALU64 | BPF_DIV | BPF_K: > if (!off) { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_zext_32(ctx, dst, is32); > emit_insn(ctx, divdu, dst, dst, t1); > emit_zext_32(ctx, dst, is32); > } else { > - move_imm(ctx, t1, imm, false); > + move_imm(ctx, t1, imm); > emit_sext_32(ctx, t1, is32); > emit_sext_32(ctx, dst, is32); > emit_insn(ctx, divd, dst, dst, t1); > @@ -866,12 +871,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU | BPF_MOD | BPF_K: > case BPF_ALU64 | BPF_MOD | BPF_K: > if (!off) { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_zext_32(ctx, dst, is32); > emit_insn(ctx, moddu, dst, dst, t1); > emit_zext_32(ctx, dst, is32); > } else { > - move_imm(ctx, t1, imm, false); > + move_imm(ctx, t1, imm); > emit_sext_32(ctx, t1, is32); > emit_sext_32(ctx, dst, is32); > emit_insn(ctx, modd, dst, dst, t1); > @@ -899,7 +905,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, andi, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, and, dst, dst, t1); > } > emit_zext_32(ctx, dst, is32); > @@ -918,7 +925,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, ori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, or, dst, dst, t1); > } > emit_zext_32(ctx, dst, is32); > @@ -937,7 +945,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, xori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, xor, dst, dst, t1); > } > emit_zext_32(ctx, dst, is32); > @@ -1097,7 +1106,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP32 | BPF_JSLE | BPF_K: > jmp_offset = bpf2la_offset(i, off, ctx); > if (imm) { > - move_imm(ctx, t1, imm, false); > + move_imm(ctx, t1, imm); > tm = t1; > } else { > /* If imm is 0, simply use zero register. */ > @@ -1129,7 +1138,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP | BPF_JSET | BPF_K: > case BPF_JMP32 | BPF_JSET | BPF_K: > jmp_offset = bpf2la_offset(i, off, ctx); > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > + emit_zext_32(ctx, t1, is32); > emit_insn(ctx, and, t1, dst, t1); > emit_zext_32(ctx, t1, is32); > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > @@ -1227,7 +1237,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (bpf_pseudo_func(insn)) > move_addr(ctx, dst, imm64); > else > - move_imm(ctx, dst, imm64, is32); > + move_imm(ctx, dst, imm64); > return 1; > } > > @@ -1268,7 +1278,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > else > emit_insn(ctx, ldbu, dst, src, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > if (sign_extend) > emit_insn(ctx, ldxb, dst, src, t1); > else > @@ -1282,7 +1292,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > else > emit_insn(ctx, ldhu, dst, src, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > if (sign_extend) > emit_insn(ctx, ldxh, dst, src, t1); > else > @@ -1296,7 +1306,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > else > emit_insn(ctx, ldwu, dst, src, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > if (sign_extend) > emit_insn(ctx, ldxw, dst, src, t1); > else > @@ -1304,7 +1314,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > } > break; > case BPF_DW: > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > emit_insn(ctx, ldxd, dst, src, t1); > break; > } > @@ -1331,42 +1341,42 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > switch (BPF_SIZE(code)) { > case BPF_B: > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > if (is_signed_imm12(off)) { > emit_insn(ctx, stb, t1, dst, off); > } else { > - move_imm(ctx, t2, off, is32); > + move_imm(ctx, t2, off); > emit_insn(ctx, stxb, t1, dst, t2); > } > break; > case BPF_H: > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > if (is_signed_imm12(off)) { > emit_insn(ctx, sth, t1, dst, off); > } else { > - move_imm(ctx, t2, off, is32); > + move_imm(ctx, t2, off); > emit_insn(ctx, stxh, t1, dst, t2); > } > break; > case BPF_W: > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > if (is_signed_imm12(off)) { > emit_insn(ctx, stw, t1, dst, off); > } else if (is_signed_imm14(off)) { > emit_insn(ctx, stptrw, t1, dst, off); > } else { > - move_imm(ctx, t2, off, is32); > + move_imm(ctx, t2, off); > emit_insn(ctx, stxw, t1, dst, t2); > } > break; > case BPF_DW: > - move_imm(ctx, t1, imm, is32); > + move_imm(ctx, t1, imm); > if (is_signed_imm12(off)) { > emit_insn(ctx, std, t1, dst, off); > } else if (is_signed_imm14(off)) { > emit_insn(ctx, stptrd, t1, dst, off); > } else { > - move_imm(ctx, t2, off, is32); > + move_imm(ctx, t2, off); > emit_insn(ctx, stxd, t1, dst, t2); > } > break; > @@ -1397,7 +1407,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_signed_imm12(off)) { > emit_insn(ctx, stb, src, dst, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxb, src, dst, t1); > } > break; > @@ -1405,7 +1415,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_signed_imm12(off)) { > emit_insn(ctx, sth, src, dst, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxh, src, dst, t1); > } > break; > @@ -1415,7 +1425,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > } else if (is_signed_imm14(off)) { > emit_insn(ctx, stptrw, src, dst, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxw, src, dst, t1); > } > break; > @@ -1425,7 +1435,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > } else if (is_signed_imm14(off)) { > emit_insn(ctx, stptrd, src, dst, off); > } else { > - move_imm(ctx, t1, off, is32); > + move_imm(ctx, t1, off); > emit_insn(ctx, stxd, src, dst, t1); > } > break; > @@ -1537,7 +1547,7 @@ static int emit_jump_and_link(struct jit_ctx *ctx, u8 rd, u64 target) > return -EFAULT; > } > > - move_imm(ctx, LOONGARCH_GPR_T1, target, false); > + move_imm(ctx, LOONGARCH_GPR_T1, target); > emit_insn(ctx, jirl, rd, LOONGARCH_GPR_T1, 0); > > return 0; > @@ -1734,7 +1744,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, > emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, -run_ctx_off + cookie_off); > > /* arg1: prog */ > - move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); > + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p); > /* arg2: &run_ctx */ > emit_insn(ctx, addid, LOONGARCH_GPR_A1, LOONGARCH_GPR_FP, -run_ctx_off); > ret = emit_call(ctx, (const u64)bpf_trampoline_enter(p)); > @@ -1755,7 +1765,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, > /* arg1: &args_off */ > emit_insn(ctx, addid, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -args_off); > if (!p->jited) > - move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi, false); > + move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi); > ret = emit_call(ctx, (const u64)p->bpf_func); > if (ret) > return ret; > @@ -1772,7 +1782,7 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im, > } > > /* arg1: prog */ > - move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); > + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p); > /* arg2: prog start time */ > move_reg(ctx, LOONGARCH_GPR_A1, LOONGARCH_GPR_S1); > /* arg3: &run_ctx */ > diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h > index bb58c42c2f2a..57a98f8a055b 100644 > --- a/arch/loongarch/net/bpf_jit.h > +++ b/arch/loongarch/net/bpf_jit.h > @@ -137,7 +137,7 @@ static inline void move_addr(struct jit_ctx *ctx, enum loongarch_gpr rd, u64 add > emit_insn(ctx, lu52id, rd, rd, imm_63_52); > } > > -static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm, bool is32) > +static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm) > { > long imm_11_0, imm_31_12, imm_51_32, imm_63_52, imm_51_0, imm_51_31; > > @@ -150,7 +150,7 @@ static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm > /* addiw rd, $zero, imm_11_0 */ > if (is_signed_imm12(imm)) { > emit_insn(ctx, addiw, rd, LOONGARCH_GPR_ZERO, imm); > - goto zext; > + return; > } > > /* ori rd, $zero, imm_11_0 */ > @@ -195,9 +195,6 @@ static inline void move_imm(struct jit_ctx *ctx, enum loongarch_gpr rd, long imm > if (!is_signed_imm52(imm)) > emit_insn(ctx, lu52id, rd, rd, imm_63_52); > } > - > -zext: > - emit_zext_32(ctx, rd, is32); > } > > static inline void move_reg(struct jit_ctx *ctx, enum loongarch_gpr rd, > -- > 2.42.0 > ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-16 8:10 [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 2026-10-16 8:10 ` [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection Tiezhu Yang 2026-10-16 8:10 ` [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() Tiezhu Yang @ 2026-10-16 8:10 ` Tiezhu Yang 2026-10-16 15:18 ` Alexei Starovoitov ` (2 more replies) 2026-10-22 7:43 ` [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 3 siblings, 3 replies; 15+ messages in thread From: Tiezhu Yang @ 2026-10-16 8:10 UTC (permalink / raw) To: Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel Currently, the LoongArch BPF JIT emits numerous redundant zero and sign extensions across 32-bit operations, bloating the final JIT image size and also leading to runtime overhead, particularly in the division and modulo operations. Thus, it is necessary to optimize 32-bit operations by emitting LoongArch native 32-bit word instructions. For general ALU32 operations, utilize native 32-bit word instructions for core calculations. Given that the BPF spec requires the upper 32 bits of the destination register to be zero-extended upon completion, use a unified 'bstrpickd' instruction to clear the upper 32 bits at the common exit of build_insn(), while immediate loads are optimized via (u32) casting to bypass the trailing extension. According to the LoongArch Reference Manual, native 32-bit division and modulo instructions (DIV.W[U] and MOD.W[U]) require the upper bits [63:31] of both input operands to be all 0s or all 1s (0x0 or 0x1ffffffff), that is to say, they need to be properly sign-extended by the software on CPU cores without the DIV32 feature. Otherwise, the 32-bit division and modulo instructions could trigger undefined hardware behaviors and return unexpected values. To ensure both performance and hardware compatibility, leverage the newly introduced cpu_has_div32 helper: (1) on newer CPUs with DIV32, emit native 32-bit div/mod instructions directly for optimal performance; (2) on older CPUs without DIV32, sign-extend the destination register in place and the source register into a temporary register via 'addw' before executing the native 32-bit div/mod instructions. Note that the JIT compiler must never alter the state of the src register because it remains read-only for the duration of the instruction. For JMP32 conditional branches, the source operands may originate from 64-bit contexts where the upper 32 bits contain arbitrary data, so use 6 independent case structures to prevent 64-bit comparison inversions. Furthermore, use 'bstrpickd' to mask out upper bits for unsigned JMP32, and 'addw' with GPR_ZERO to re-align sign bits for signed JMP32. With this patch, the emit_{zext, sext}_32() and is_signed_bpf_cond() helpers are completely wiped out from the header and source files, delivering a native execution stream with a reduced JIT image size, especially for division and mod operations. All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) and Loongson-3A5000 (without DIV32), here are the test results: $ sudo modprobe test_bpf $ dmesg -t | grep Summary test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED For the BPF selftests test_verifier and test_progs, there are no any regressions. Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> --- arch/loongarch/include/asm/inst.h | 6 + arch/loongarch/net/bpf_jit.c | 313 +++++++++++++++++------------- arch/loongarch/net/bpf_jit.h | 18 -- 3 files changed, 183 insertions(+), 154 deletions(-) diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h index 585667e361c2..b55aae325300 100644 --- a/arch/loongarch/include/asm/inst.h +++ b/arch/loongarch/include/asm/inst.h @@ -752,11 +752,17 @@ static inline void emit_##NAME(union loongarch_instruction *insn, \ DEF_EMIT_REG3_FORMAT(addw, addw_op) DEF_EMIT_REG3_FORMAT(addd, addd_op) +DEF_EMIT_REG3_FORMAT(subw, subw_op) DEF_EMIT_REG3_FORMAT(subd, subd_op) +DEF_EMIT_REG3_FORMAT(mulw, mulw_op) DEF_EMIT_REG3_FORMAT(muld, muld_op) +DEF_EMIT_REG3_FORMAT(divw, divw_op) DEF_EMIT_REG3_FORMAT(divd, divd_op) +DEF_EMIT_REG3_FORMAT(modw, modw_op) DEF_EMIT_REG3_FORMAT(modd, modd_op) +DEF_EMIT_REG3_FORMAT(divwu, divwu_op) DEF_EMIT_REG3_FORMAT(divdu, divdu_op) +DEF_EMIT_REG3_FORMAT(modwu, modwu_op) DEF_EMIT_REG3_FORMAT(moddu, moddu_op) DEF_EMIT_REG3_FORMAT(maskeqz, maskeqz_op) DEF_EMIT_REG3_FORMAT(masknez, masknez_op) diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c index 77cd96a6f84c..1a7c8effc37b 100644 --- a/arch/loongarch/net/bpf_jit.c +++ b/arch/loongarch/net/bpf_jit.c @@ -288,7 +288,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) * if (index >= array->map.max_entries) * goto out; */ - emit_zext_32(ctx, a2, true); + emit_insn(ctx, bstrpickd, a2, a2, 31, 0); off = offsetof(struct bpf_array, map.max_entries); emit_insn(ctx, ldwu, t1, a1, off); @@ -412,7 +412,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) return -EINVAL; } emit_insn(ctx, amadddbb, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_H: if (!cpu_has_lam_bh) { @@ -420,23 +419,23 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) return -EINVAL; } emit_insn(ctx, amadddbh, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_W: emit_insn(ctx, amadddbw, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_DW: emit_insn(ctx, amadddbd, src, t1, t3); break; } + if (!isdw) + emit_insn(ctx, bstrpickd, src, src, 31, 0); break; case BPF_AND | BPF_FETCH: if (isdw) { emit_insn(ctx, amanddbd, src, t1, t3); } else { emit_insn(ctx, amanddbw, src, t1, t3); - emit_zext_32(ctx, src, true); + emit_insn(ctx, bstrpickd, src, src, 31, 0); } break; case BPF_OR | BPF_FETCH: @@ -444,7 +443,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) emit_insn(ctx, amordbd, src, t1, t3); } else { emit_insn(ctx, amordbw, src, t1, t3); - emit_zext_32(ctx, src, true); + emit_insn(ctx, bstrpickd, src, src, 31, 0); } break; case BPF_XOR | BPF_FETCH: @@ -452,7 +451,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) emit_insn(ctx, amxordbd, src, t1, t3); } else { emit_insn(ctx, amxordbw, src, t1, t3); - emit_zext_32(ctx, src, true); + emit_insn(ctx, bstrpickd, src, src, 31, 0); } break; /* src = atomic_xchg(dst + off, src); */ @@ -464,7 +463,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) return -EINVAL; } emit_insn(ctx, amswapdbb, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_H: if (!cpu_has_lam_bh) { @@ -472,16 +470,16 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) return -EINVAL; } emit_insn(ctx, amswapdbh, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_W: emit_insn(ctx, amswapdbw, src, t1, t3); - emit_zext_32(ctx, src, true); break; case BPF_DW: emit_insn(ctx, amswapdbd, src, t1, t3); break; } + if (!isdw) + emit_insn(ctx, bstrpickd, src, src, 31, 0); break; /* r0 = atomic_cmpxchg(dst + off, r0, src); */ case BPF_CMPXCHG: @@ -494,13 +492,13 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -4); } else { emit_insn(ctx, llw, r0, t1, 0); - emit_zext_32(ctx, t2, true); - emit_zext_32(ctx, r0, true); + emit_insn(ctx, bstrpickd, t2, t2, 31, 0); + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); emit_insn(ctx, bne, t2, r0, 4); move_reg(ctx, t3, src); emit_insn(ctx, scw, t3, t1, 0); emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -6); - emit_zext_32(ctx, r0, true); + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); } emit_insn(ctx, dbar, DBAR_LLSC_MB); break; @@ -605,12 +603,6 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) return 0; } -static bool is_signed_bpf_cond(u8 cond) -{ - return cond == BPF_JSGT || cond == BPF_JSLT || - cond == BPF_JSGE || cond == BPF_JSLE; -} - #define BPF_FIXUP_REG_MASK GENMASK(31, 27) #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) #define REG_DONT_CLEAR_MARKER 0 @@ -707,7 +699,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext u8 dst = regmap[insn->dst_reg]; const s16 off = insn->off; const s32 imm = insn->imm; - const bool is32 = BPF_CLASS(insn->code) == BPF_ALU || BPF_CLASS(insn->code) == BPF_JMP32; + const bool is32 = BPF_CLASS(insn->code) == BPF_ALU && BPF_OP(insn->code) != BPF_END; switch (code) { /* dst = src */ @@ -715,7 +707,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_ALU64 | BPF_MOV | BPF_X: if (insn_is_cast_user(insn)) { move_reg(ctx, t1, src); - emit_zext_32(ctx, t1, true); + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); emit_insn(ctx, or, t1, dst, t1); @@ -734,15 +726,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext switch (off) { case 0: move_reg(ctx, dst, src); - emit_zext_32(ctx, dst, is32); break; case 8: emit_insn(ctx, extwb, dst, src); - emit_zext_32(ctx, dst, is32); break; case 16: emit_insn(ctx, extwh, dst, src); - emit_zext_32(ctx, dst, is32); break; case 32: emit_insn(ctx, addw, dst, src, LOONGARCH_GPR_ZERO); @@ -752,151 +741,188 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = imm */ case BPF_ALU | BPF_MOV | BPF_K: + move_imm(ctx, dst, (u32)imm); + return 0; case BPF_ALU64 | BPF_MOV | BPF_K: move_imm(ctx, dst, imm); - emit_zext_32(ctx, dst, is32); break; /* dst = dst + src */ case BPF_ALU | BPF_ADD | BPF_X: + emit_insn(ctx, addw, dst, dst, src); + break; case BPF_ALU64 | BPF_ADD | BPF_X: emit_insn(ctx, addd, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst + imm */ case BPF_ALU | BPF_ADD | BPF_K: + if (is_signed_imm12(imm)) { + emit_insn(ctx, addiw, dst, dst, imm); + } else { + move_imm(ctx, t1, (u32)imm); + emit_insn(ctx, addw, dst, dst, t1); + } + break; case BPF_ALU64 | BPF_ADD | BPF_K: if (is_signed_imm12(imm)) { emit_insn(ctx, addid, dst, dst, imm); } else { move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); emit_insn(ctx, addd, dst, dst, t1); } - emit_zext_32(ctx, dst, is32); break; /* dst = dst - src */ case BPF_ALU | BPF_SUB | BPF_X: + emit_insn(ctx, subw, dst, dst, src); + break; case BPF_ALU64 | BPF_SUB | BPF_X: emit_insn(ctx, subd, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst - imm */ case BPF_ALU | BPF_SUB | BPF_K: + if (is_signed_imm12(-imm)) { + emit_insn(ctx, addiw, dst, dst, -imm); + } else { + move_imm(ctx, t1, (u32)imm); + emit_insn(ctx, subw, dst, dst, t1); + } + break; case BPF_ALU64 | BPF_SUB | BPF_K: if (is_signed_imm12(-imm)) { emit_insn(ctx, addid, dst, dst, -imm); } else { move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); emit_insn(ctx, subd, dst, dst, t1); } - emit_zext_32(ctx, dst, is32); break; /* dst = dst * src */ case BPF_ALU | BPF_MUL | BPF_X: + emit_insn(ctx, mulw, dst, dst, src); + break; case BPF_ALU64 | BPF_MUL | BPF_X: emit_insn(ctx, muld, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst * imm */ case BPF_ALU | BPF_MUL | BPF_K: + move_imm(ctx, t1, (u32)imm); + emit_insn(ctx, mulw, dst, dst, t1); + break; case BPF_ALU64 | BPF_MUL | BPF_K: move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); emit_insn(ctx, muld, dst, dst, t1); - emit_zext_32(ctx, dst, is32); break; /* dst = dst / src */ case BPF_ALU | BPF_DIV | BPF_X: - case BPF_ALU64 | BPF_DIV | BPF_X: - if (!off) { - emit_zext_32(ctx, dst, is32); - move_reg(ctx, t1, src); - emit_zext_32(ctx, t1, is32); - emit_insn(ctx, divdu, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } else { - emit_sext_32(ctx, dst, is32); - move_reg(ctx, t1, src); - emit_sext_32(ctx, t1, is32); - emit_insn(ctx, divd, dst, dst, t1); - emit_zext_32(ctx, dst, is32); + if (cpu_has_div32) { + if (!off) + emit_insn(ctx, divwu, dst, dst, src); + else + emit_insn(ctx, divw, dst, dst, src); + break; } + /* + * Legacy fallback for !div32: src must be extended into a + * temporary register to keep the original src register intact. + */ + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); + if (!off) + emit_insn(ctx, divwu, dst, dst, t1); + else + emit_insn(ctx, divw, dst, dst, t1); + break; + case BPF_ALU64 | BPF_DIV | BPF_X: + if (!off) + emit_insn(ctx, divdu, dst, dst, src); + else + emit_insn(ctx, divd, dst, dst, src); break; /* dst = dst / imm */ case BPF_ALU | BPF_DIV | BPF_K: + move_imm(ctx, t1, (u32)imm); + if (!cpu_has_div32) { + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); + } + if (!off) + emit_insn(ctx, divwu, dst, dst, t1); + else + emit_insn(ctx, divw, dst, dst, t1); + break; case BPF_ALU64 | BPF_DIV | BPF_K: - if (!off) { - move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); - emit_zext_32(ctx, dst, is32); + move_imm(ctx, t1, imm); + if (!off) emit_insn(ctx, divdu, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } else { - move_imm(ctx, t1, imm); - emit_sext_32(ctx, t1, is32); - emit_sext_32(ctx, dst, is32); + else emit_insn(ctx, divd, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } break; /* dst = dst % src */ case BPF_ALU | BPF_MOD | BPF_X: - case BPF_ALU64 | BPF_MOD | BPF_X: - if (!off) { - emit_zext_32(ctx, dst, is32); - move_reg(ctx, t1, src); - emit_zext_32(ctx, t1, is32); - emit_insn(ctx, moddu, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } else { - emit_sext_32(ctx, dst, is32); - move_reg(ctx, t1, src); - emit_sext_32(ctx, t1, is32); - emit_insn(ctx, modd, dst, dst, t1); - emit_zext_32(ctx, dst, is32); + if (cpu_has_div32) { + if (!off) + emit_insn(ctx, modwu, dst, dst, src); + else + emit_insn(ctx, modw, dst, dst, src); + break; } + /* + * Legacy fallback for !div32: src must be extended into a + * temporary register to keep the original src register intact. + */ + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); + if (!off) + emit_insn(ctx, modwu, dst, dst, t1); + else + emit_insn(ctx, modw, dst, dst, t1); + break; + case BPF_ALU64 | BPF_MOD | BPF_X: + if (!off) + emit_insn(ctx, moddu, dst, dst, src); + else + emit_insn(ctx, modd, dst, dst, src); break; /* dst = dst % imm */ case BPF_ALU | BPF_MOD | BPF_K: + move_imm(ctx, t1, (u32)imm); + if (!cpu_has_div32) { + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); + } + if (!off) + emit_insn(ctx, modwu, dst, dst, t1); + else + emit_insn(ctx, modw, dst, dst, t1); + break; case BPF_ALU64 | BPF_MOD | BPF_K: - if (!off) { - move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); - emit_zext_32(ctx, dst, is32); + move_imm(ctx, t1, imm); + if (!off) emit_insn(ctx, moddu, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } else { - move_imm(ctx, t1, imm); - emit_sext_32(ctx, t1, is32); - emit_sext_32(ctx, dst, is32); + else emit_insn(ctx, modd, dst, dst, t1); - emit_zext_32(ctx, dst, is32); - } break; /* dst = -dst */ case BPF_ALU | BPF_NEG: + emit_insn(ctx, subw, dst, LOONGARCH_GPR_ZERO, dst); + break; case BPF_ALU64 | BPF_NEG: emit_insn(ctx, subd, dst, LOONGARCH_GPR_ZERO, dst); - emit_zext_32(ctx, dst, is32); break; /* dst = dst & src */ case BPF_ALU | BPF_AND | BPF_X: case BPF_ALU64 | BPF_AND | BPF_X: emit_insn(ctx, and, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst & imm */ @@ -905,18 +931,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, andi, dst, dst, imm); } else { - move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); + if (is32) + move_imm(ctx, t1, (u32)imm); + else + move_imm(ctx, t1, imm); emit_insn(ctx, and, dst, dst, t1); } - emit_zext_32(ctx, dst, is32); break; /* dst = dst | src */ case BPF_ALU | BPF_OR | BPF_X: case BPF_ALU64 | BPF_OR | BPF_X: emit_insn(ctx, or, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst | imm */ @@ -925,18 +951,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, ori, dst, dst, imm); } else { - move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); + if (is32) + move_imm(ctx, t1, (u32)imm); + else + move_imm(ctx, t1, imm); emit_insn(ctx, or, dst, dst, t1); } - emit_zext_32(ctx, dst, is32); break; /* dst = dst ^ src */ case BPF_ALU | BPF_XOR | BPF_X: case BPF_ALU64 | BPF_XOR | BPF_X: emit_insn(ctx, xor, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; /* dst = dst ^ imm */ @@ -945,19 +971,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext if (is_unsigned_imm12(imm)) { emit_insn(ctx, xori, dst, dst, imm); } else { - move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); + if (is32) + move_imm(ctx, t1, (u32)imm); + else + move_imm(ctx, t1, imm); emit_insn(ctx, xor, dst, dst, t1); } - emit_zext_32(ctx, dst, is32); break; /* dst = dst << src (logical) */ case BPF_ALU | BPF_LSH | BPF_X: emit_insn(ctx, sllw, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_LSH | BPF_X: emit_insn(ctx, slld, dst, dst, src); break; @@ -965,9 +990,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst << imm (logical) */ case BPF_ALU | BPF_LSH | BPF_K: emit_insn(ctx, slliw, dst, dst, imm); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_LSH | BPF_K: emit_insn(ctx, sllid, dst, dst, imm); break; @@ -975,9 +998,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst >> src (logical) */ case BPF_ALU | BPF_RSH | BPF_X: emit_insn(ctx, srlw, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_RSH | BPF_X: emit_insn(ctx, srld, dst, dst, src); break; @@ -985,9 +1006,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst >> imm (logical) */ case BPF_ALU | BPF_RSH | BPF_K: emit_insn(ctx, srliw, dst, dst, imm); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_RSH | BPF_K: emit_insn(ctx, srlid, dst, dst, imm); break; @@ -995,9 +1014,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst >> src (arithmetic) */ case BPF_ALU | BPF_ARSH | BPF_X: emit_insn(ctx, sraw, dst, dst, src); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_ARSH | BPF_X: emit_insn(ctx, srad, dst, dst, src); break; @@ -1005,9 +1022,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext /* dst = dst >> imm (arithmetic) */ case BPF_ALU | BPF_ARSH | BPF_K: emit_insn(ctx, sraiw, dst, dst, imm); - emit_zext_32(ctx, dst, is32); break; - case BPF_ALU64 | BPF_ARSH | BPF_K: emit_insn(ctx, sraid, dst, dst, imm); break; @@ -1016,12 +1031,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_ALU | BPF_END | BPF_FROM_LE: switch (imm) { case 16: - /* zero-extend 16 bits into 64 bits */ emit_insn(ctx, bstrpickd, dst, dst, 15, 0); break; case 32: - /* zero-extend 32 bits into 64 bits */ - emit_zext_32(ctx, dst, is32); + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); break; case 64: /* do nothing */ @@ -1034,13 +1047,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext switch (imm) { case 16: emit_insn(ctx, revb2h, dst, dst); - /* zero-extend 16 bits into 64 bits */ emit_insn(ctx, bstrpickd, dst, dst, 15, 0); break; case 32: emit_insn(ctx, revb2w, dst, dst); - /* clear the upper 32 bits */ - emit_zext_32(ctx, dst, true); + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); break; case 64: emit_insn(ctx, revbd, dst, dst); @@ -1048,7 +1059,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext } break; - /* PC += off if dst cond src */ + /* PC += off if dst cond src (64-bit) */ case BPF_JMP | BPF_JEQ | BPF_X: case BPF_JMP | BPF_JNE | BPF_X: case BPF_JMP | BPF_JGT | BPF_X: @@ -1059,31 +1070,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP | BPF_JSGE | BPF_X: case BPF_JMP | BPF_JSLT | BPF_X: case BPF_JMP | BPF_JSLE | BPF_X: + jmp_offset = bpf2la_offset(i, off, ctx); + if (emit_cond_jmp(ctx, cond, dst, src, jmp_offset) < 0) + goto toofar; + break; + + /* PC += off if dst cond src (32-bit unsigned) */ case BPF_JMP32 | BPF_JEQ | BPF_X: case BPF_JMP32 | BPF_JNE | BPF_X: case BPF_JMP32 | BPF_JGT | BPF_X: case BPF_JMP32 | BPF_JGE | BPF_X: case BPF_JMP32 | BPF_JLT | BPF_X: case BPF_JMP32 | BPF_JLE | BPF_X: + jmp_offset = bpf2la_offset(i, off, ctx); + emit_insn(ctx, bstrpickd, t1, dst, 31, 0); + emit_insn(ctx, bstrpickd, t2, src, 31, 0); + if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) + goto toofar; + break; + + /* PC += off if dst cond src (32-bit signed) */ case BPF_JMP32 | BPF_JSGT | BPF_X: case BPF_JMP32 | BPF_JSGE | BPF_X: case BPF_JMP32 | BPF_JSLT | BPF_X: case BPF_JMP32 | BPF_JSLE | BPF_X: jmp_offset = bpf2la_offset(i, off, ctx); - move_reg(ctx, t1, dst); - move_reg(ctx, t2, src); - if (is_signed_bpf_cond(BPF_OP(code))) { - emit_sext_32(ctx, t1, is32); - emit_sext_32(ctx, t2, is32); - } else { - emit_zext_32(ctx, t1, is32); - emit_zext_32(ctx, t2, is32); - } + emit_insn(ctx, addw, t1, dst, LOONGARCH_GPR_ZERO); + emit_insn(ctx, addw, t2, src, LOONGARCH_GPR_ZERO); if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) goto toofar; break; - /* PC += off if dst cond imm */ + /* PC += off if dst cond imm (64-bit) */ case BPF_JMP | BPF_JEQ | BPF_K: case BPF_JMP | BPF_JNE | BPF_K: case BPF_JMP | BPF_JGT | BPF_K: @@ -1094,12 +1112,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP | BPF_JSGE | BPF_K: case BPF_JMP | BPF_JSLT | BPF_K: case BPF_JMP | BPF_JSLE | BPF_K: + jmp_offset = bpf2la_offset(i, off, ctx); + if (imm) { + move_imm(ctx, t1, imm); + tm = t1; + } else { + tm = LOONGARCH_GPR_ZERO; + } + if (emit_cond_jmp(ctx, cond, dst, tm, jmp_offset) < 0) + goto toofar; + break; + + /* PC += off if dst cond imm (32-bit unsigned) */ case BPF_JMP32 | BPF_JEQ | BPF_K: case BPF_JMP32 | BPF_JNE | BPF_K: case BPF_JMP32 | BPF_JGT | BPF_K: case BPF_JMP32 | BPF_JGE | BPF_K: case BPF_JMP32 | BPF_JLT | BPF_K: case BPF_JMP32 | BPF_JLE | BPF_K: + jmp_offset = bpf2la_offset(i, off, ctx); + if (imm) { + move_imm(ctx, t1, imm); + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); + tm = t1; + } else { + tm = LOONGARCH_GPR_ZERO; + } + emit_insn(ctx, bstrpickd, t2, dst, 31, 0); + if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) + goto toofar; + break; + + /* PC += off if dst cond imm (32-bit signed) */ case BPF_JMP32 | BPF_JSGT | BPF_K: case BPF_JMP32 | BPF_JSGE | BPF_K: case BPF_JMP32 | BPF_JSLT | BPF_K: @@ -1107,19 +1151,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext jmp_offset = bpf2la_offset(i, off, ctx); if (imm) { move_imm(ctx, t1, imm); + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); tm = t1; } else { - /* If imm is 0, simply use zero register. */ tm = LOONGARCH_GPR_ZERO; } - move_reg(ctx, t2, dst); - if (is_signed_bpf_cond(BPF_OP(code))) { - emit_sext_32(ctx, tm, is32); - emit_sext_32(ctx, t2, is32); - } else { - emit_zext_32(ctx, tm, is32); - emit_zext_32(ctx, t2, is32); - } + emit_insn(ctx, addw, t2, dst, LOONGARCH_GPR_ZERO); if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) goto toofar; break; @@ -1129,7 +1166,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP32 | BPF_JSET | BPF_X: jmp_offset = bpf2la_offset(i, off, ctx); emit_insn(ctx, and, t1, dst, src); - emit_zext_32(ctx, t1, is32); + if (BPF_CLASS(code) == BPF_JMP32) + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) goto toofar; break; @@ -1139,9 +1177,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext case BPF_JMP32 | BPF_JSET | BPF_K: jmp_offset = bpf2la_offset(i, off, ctx); move_imm(ctx, t1, imm); - emit_zext_32(ctx, t1, is32); emit_insn(ctx, and, t1, dst, t1); - emit_zext_32(ctx, t1, is32); + if (BPF_CLASS(code) == BPF_JMP32) + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) goto toofar; break; @@ -1468,6 +1506,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext return -EINVAL; } + if (is32) + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); + return 0; toofar: diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h index 57a98f8a055b..2bd0dc5b911c 100644 --- a/arch/loongarch/net/bpf_jit.h +++ b/arch/loongarch/net/bpf_jit.h @@ -72,24 +72,6 @@ static inline int epilogue_offset(const struct jit_ctx *ctx) return (to - from); } -/* Zero-extend 32 bits into 64 bits */ -static inline void emit_zext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) -{ - if (!is32) - return; - - emit_insn(ctx, lu32id, reg, 0); -} - -/* Signed-extend 32 bits into 64 bits */ -static inline void emit_sext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) -{ - if (!is32) - return; - - emit_insn(ctx, addiw, reg, reg, 0); -} - /* Emit proper extension according to ABI requirements. * Note that it requires a value of size `size` already resides in register `reg`. */ -- 2.42.0 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang @ 2026-10-16 15:18 ` Alexei Starovoitov 2026-10-17 10:58 ` Tiezhu Yang 2026-10-27 2:02 ` Hengqi Chen 2026-10-28 4:19 ` Huacai Chen 2 siblings, 1 reply; 15+ messages in thread From: Alexei Starovoitov @ 2026-10-16 15:18 UTC (permalink / raw) To: Tiezhu Yang, Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel On Fri, Oct 16, 2026 at 04:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > and Loongson-3A5000 (without DIV32), here are the test results: fyi test_bpf is gone. See commit 4532e525b286 ("selftests/bpf: Remove the old test_bpf module"). Pls run test_progs ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-16 15:18 ` Alexei Starovoitov @ 2026-10-17 10:58 ` Tiezhu Yang 0 siblings, 0 replies; 15+ messages in thread From: Tiezhu Yang @ 2026-10-17 10:58 UTC (permalink / raw) To: Alexei Starovoitov, Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel On 2026/10/16 下午11:18, Alexei Starovoitov wrote: > On Fri, Oct 16, 2026 at 04:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > >> All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) >> and Loongson-3A5000 (without DIV32), here are the test results: > > fyi test_bpf is gone. See > commit 4532e525b286 ("selftests/bpf: Remove the old test_bpf module"). > Pls run test_progs I tested this series based on bpf-next tree with "./test_progs -a jit_insns" and "./test_progs -a jit_insns_blinded", there are 3 pre-existing failures: (1) #191/1 jit_insns/arena access with zero offset:FAIL (2) #191/25 jit_insns/ldst_offsets:FAIL (3) #192/11 jit_insns_blinded/ldst_offsets:FAIL These 3 failures are entirely unrelated to this series; they still fail on the clean, original bpf-next tree without this series. The root cause is that LoongArch BPF JIT support for arena atomics and sign-extending loads from arena is still a work-in-progress. IIUC, there are already two patches to fix them, but need revisions: https://lore.kernel.org/loongarch/20260702022322.51033-8-dongtai.guo@linux.dev/ https://lore.kernel.org/loongarch/20260702022322.51033-7-dongtai.guo@linux.dev/ Bypassing these known architectural gaps using the blacklist (-b) option allows the remaining subtests under jit_insns and jit_insns_blinded to pass: $ sudo ./test_progs -a jit_insns -b "jit_insns/arena access with zero offset,jit_insns/ldst_offsets" #191/2 jit_insns/cast to a user address:OK #191/3 jit_insns/alu_imm:OK #191/4 jit_insns/alu_reg:OK #191/5 jit_insns/alu_imm_regs:OK #191/6 jit_insns/div_const:OK #191/7 jit_insns/alu_reg_pairs:OK #191/8 jit_insns/alu_edges:OK #191/9 jit_insns/shifts:OK #191/10 jit_insns/shift_counts:OK #191/11 jit_insns/atomics:OK #191/12 jit_insns/atomic_reg_pairs:OK #191/13 jit_insns/xadd_chain:OK #191/14 jit_insns/ld_imm64:OK #191/15 jit_insns/jmp_imm:OK #191/16 jit_insns/jmp_reg:OK #191/17 jit_insns/jmp_reg_pairs:OK #191/18 jit_insns/jmp_edges:OK #191/19 jit_insns/staggered_jumps:OK #191/20 jit_insns/max_jmp:OK #191/21 jit_insns/convergence:OK #191/22 jit_insns/calls:OK #191/23 jit_insns/helper_calls:OK #191/24 jit_insns/ldst_zero_off:OK #191/26 jit_insns/acq_rel:OK #191/27 jit_insns/movsx:OK #191/28 jit_insns/jset_imm:OK #191/29 jit_insns/cmp_imm:OK #191/30 jit_insns/alu_consts:OK #191/31 jit_insns/bswap:OK #191/32 jit_insns/probe_mem:OK #191 jit_insns:OK Summary: 1/30 PASSED, 0 SKIPPED, 0/0 FAILED $ sudo ./test_progs -a jit_insns_blinded -b jit_insns_blinded/ldst_offsets #192/1 jit_insns_blinded/alu_imm:OK #192/2 jit_insns_blinded/alu_imm_regs:OK #192/3 jit_insns_blinded/alu_edges:OK #192/4 jit_insns_blinded/div_const:OK #192/5 jit_insns_blinded/shifts:OK #192/6 jit_insns_blinded/xadd_chain:OK #192/7 jit_insns_blinded/ld_imm64:OK #192/8 jit_insns_blinded/jmp_imm:OK #192/9 jit_insns_blinded/jmp_edges:OK #192/10 jit_insns_blinded/ldst_zero_off:OK #192/12 jit_insns_blinded/jset_imm:OK #192/13 jit_insns_blinded/cmp_imm:OK #192/14 jit_insns_blinded/alu_consts:OK #192 jit_insns_blinded:OK Summary: 1/13 PASSED, 0 SKIPPED, 0/0 FAILED Thanks, Tiezhu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang 2026-10-16 15:18 ` Alexei Starovoitov @ 2026-10-27 2:02 ` Hengqi Chen 2026-10-27 4:10 ` Tiezhu Yang 2026-10-28 4:19 ` Huacai Chen 2 siblings, 1 reply; 15+ messages in thread From: Hengqi Chen @ 2026-10-27 2:02 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Huacai Chen, loongarch, bpf, linux-kernel On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > Currently, the LoongArch BPF JIT emits numerous redundant zero and sign > extensions across 32-bit operations, bloating the final JIT image size > and also leading to runtime overhead, particularly in the division and > modulo operations. Thus, it is necessary to optimize 32-bit operations > by emitting LoongArch native 32-bit word instructions. > > For general ALU32 operations, utilize native 32-bit word instructions > for core calculations. Given that the BPF spec requires the upper 32 > bits of the destination register to be zero-extended upon completion, > use a unified 'bstrpickd' instruction to clear the upper 32 bits at > the common exit of build_insn(), while immediate loads are optimized > via (u32) casting to bypass the trailing extension. > > According to the LoongArch Reference Manual, native 32-bit division > and modulo instructions (DIV.W[U] and MOD.W[U]) require the upper > bits [63:31] of both input operands to be all 0s or all 1s (0x0 or > 0x1ffffffff), that is to say, they need to be properly sign-extended > by the software on CPU cores without the DIV32 feature. Otherwise, > the 32-bit division and modulo instructions could trigger undefined > hardware behaviors and return unexpected values. > > To ensure both performance and hardware compatibility, leverage the > newly introduced cpu_has_div32 helper: > (1) on newer CPUs with DIV32, emit native 32-bit div/mod instructions > directly for optimal performance; > (2) on older CPUs without DIV32, sign-extend the destination register > in place and the source register into a temporary register via 'addw' > before executing the native 32-bit div/mod instructions. Note that the > JIT compiler must never alter the state of the src register because it > remains read-only for the duration of the instruction. > > For JMP32 conditional branches, the source operands may originate from > 64-bit contexts where the upper 32 bits contain arbitrary data, so use > 6 independent case structures to prevent 64-bit comparison inversions. > Furthermore, use 'bstrpickd' to mask out upper bits for unsigned JMP32, > and 'addw' with GPR_ZERO to re-align sign bits for signed JMP32. > > With this patch, the emit_{zext, sext}_32() and is_signed_bpf_cond() > helpers are completely wiped out from the header and source files, > delivering a native execution stream with a reduced JIT image size, > especially for division and mod operations. > > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > and Loongson-3A5000 (without DIV32), here are the test results: > > $ sudo modprobe test_bpf > $ dmesg -t | grep Summary > test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] > test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] > test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED > > For the BPF selftests test_verifier and test_progs, there are no any > regressions. > > Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> > --- > arch/loongarch/include/asm/inst.h | 6 + > arch/loongarch/net/bpf_jit.c | 313 +++++++++++++++++------------- > arch/loongarch/net/bpf_jit.h | 18 -- > 3 files changed, 183 insertions(+), 154 deletions(-) > > diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h > index 585667e361c2..b55aae325300 100644 > --- a/arch/loongarch/include/asm/inst.h > +++ b/arch/loongarch/include/asm/inst.h > @@ -752,11 +752,17 @@ static inline void emit_##NAME(union loongarch_instruction *insn, \ > > DEF_EMIT_REG3_FORMAT(addw, addw_op) > DEF_EMIT_REG3_FORMAT(addd, addd_op) > +DEF_EMIT_REG3_FORMAT(subw, subw_op) > DEF_EMIT_REG3_FORMAT(subd, subd_op) > +DEF_EMIT_REG3_FORMAT(mulw, mulw_op) > DEF_EMIT_REG3_FORMAT(muld, muld_op) > +DEF_EMIT_REG3_FORMAT(divw, divw_op) > DEF_EMIT_REG3_FORMAT(divd, divd_op) > +DEF_EMIT_REG3_FORMAT(modw, modw_op) > DEF_EMIT_REG3_FORMAT(modd, modd_op) > +DEF_EMIT_REG3_FORMAT(divwu, divwu_op) > DEF_EMIT_REG3_FORMAT(divdu, divdu_op) > +DEF_EMIT_REG3_FORMAT(modwu, modwu_op) > DEF_EMIT_REG3_FORMAT(moddu, moddu_op) > DEF_EMIT_REG3_FORMAT(maskeqz, maskeqz_op) > DEF_EMIT_REG3_FORMAT(masknez, masknez_op) > diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c > index 77cd96a6f84c..1a7c8effc37b 100644 > --- a/arch/loongarch/net/bpf_jit.c > +++ b/arch/loongarch/net/bpf_jit.c > @@ -288,7 +288,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) > * if (index >= array->map.max_entries) > * goto out; > */ > - emit_zext_32(ctx, a2, true); > + emit_insn(ctx, bstrpickd, a2, a2, 31, 0); > > off = offsetof(struct bpf_array, map.max_entries); > emit_insn(ctx, ldwu, t1, a1, off); > @@ -412,7 +412,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amadddbb, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_H: > if (!cpu_has_lam_bh) { > @@ -420,23 +419,23 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amadddbh, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_W: > emit_insn(ctx, amadddbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_DW: > emit_insn(ctx, amadddbd, src, t1, t3); > break; > } > + if (!isdw) > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > break; > case BPF_AND | BPF_FETCH: > if (isdw) { > emit_insn(ctx, amanddbd, src, t1, t3); > } else { > emit_insn(ctx, amanddbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > case BPF_OR | BPF_FETCH: > @@ -444,7 +443,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, amordbd, src, t1, t3); > } else { > emit_insn(ctx, amordbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > case BPF_XOR | BPF_FETCH: > @@ -452,7 +451,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, amxordbd, src, t1, t3); > } else { > emit_insn(ctx, amxordbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > /* src = atomic_xchg(dst + off, src); */ > @@ -464,7 +463,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amswapdbb, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_H: > if (!cpu_has_lam_bh) { > @@ -472,16 +470,16 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amswapdbh, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_W: > emit_insn(ctx, amswapdbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_DW: > emit_insn(ctx, amswapdbd, src, t1, t3); > break; > } > + if (!isdw) > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > break; > /* r0 = atomic_cmpxchg(dst + off, r0, src); */ > case BPF_CMPXCHG: > @@ -494,13 +492,13 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -4); > } else { > emit_insn(ctx, llw, r0, t1, 0); > - emit_zext_32(ctx, t2, true); > - emit_zext_32(ctx, r0, true); > + emit_insn(ctx, bstrpickd, t2, t2, 31, 0); > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > emit_insn(ctx, bne, t2, r0, 4); > move_reg(ctx, t3, src); > emit_insn(ctx, scw, t3, t1, 0); > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -6); > - emit_zext_32(ctx, r0, true); > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > } > emit_insn(ctx, dbar, DBAR_LLSC_MB); > break; > @@ -605,12 +603,6 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > return 0; > } > > -static bool is_signed_bpf_cond(u8 cond) > -{ > - return cond == BPF_JSGT || cond == BPF_JSLT || > - cond == BPF_JSGE || cond == BPF_JSLE; > -} > - > #define BPF_FIXUP_REG_MASK GENMASK(31, 27) > #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) > #define REG_DONT_CLEAR_MARKER 0 > @@ -707,7 +699,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > u8 dst = regmap[insn->dst_reg]; > const s16 off = insn->off; > const s32 imm = insn->imm; > - const bool is32 = BPF_CLASS(insn->code) == BPF_ALU || BPF_CLASS(insn->code) == BPF_JMP32; > + const bool is32 = BPF_CLASS(insn->code) == BPF_ALU && BPF_OP(insn->code) != BPF_END; > > switch (code) { > /* dst = src */ > @@ -715,7 +707,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU64 | BPF_MOV | BPF_X: > if (insn_is_cast_user(insn)) { > move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, true); > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); > emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); > emit_insn(ctx, or, t1, dst, t1); > @@ -734,15 +726,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > switch (off) { > case 0: > move_reg(ctx, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 8: > emit_insn(ctx, extwb, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 16: > emit_insn(ctx, extwh, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 32: > emit_insn(ctx, addw, dst, src, LOONGARCH_GPR_ZERO); > @@ -752,151 +741,188 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = imm */ > case BPF_ALU | BPF_MOV | BPF_K: > + move_imm(ctx, dst, (u32)imm); > + return 0; > case BPF_ALU64 | BPF_MOV | BPF_K: > move_imm(ctx, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst + src */ > case BPF_ALU | BPF_ADD | BPF_X: > + emit_insn(ctx, addw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_ADD | BPF_X: > emit_insn(ctx, addd, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst + imm */ > case BPF_ALU | BPF_ADD | BPF_K: > + if (is_signed_imm12(imm)) { > + emit_insn(ctx, addiw, dst, dst, imm); > + } else { > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, addw, dst, dst, t1); > + } > + break; > case BPF_ALU64 | BPF_ADD | BPF_K: > if (is_signed_imm12(imm)) { > emit_insn(ctx, addid, dst, dst, imm); > } else { > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, addd, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst - src */ > case BPF_ALU | BPF_SUB | BPF_X: > + emit_insn(ctx, subw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_SUB | BPF_X: > emit_insn(ctx, subd, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst - imm */ > case BPF_ALU | BPF_SUB | BPF_K: > + if (is_signed_imm12(-imm)) { > + emit_insn(ctx, addiw, dst, dst, -imm); > + } else { > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, subw, dst, dst, t1); > + } > + break; > case BPF_ALU64 | BPF_SUB | BPF_K: > if (is_signed_imm12(-imm)) { > emit_insn(ctx, addid, dst, dst, -imm); > } else { > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, subd, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst * src */ > case BPF_ALU | BPF_MUL | BPF_X: > + emit_insn(ctx, mulw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_MUL | BPF_X: > emit_insn(ctx, muld, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst * imm */ > case BPF_ALU | BPF_MUL | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, mulw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_MUL | BPF_K: > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, muld, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst / src */ > case BPF_ALU | BPF_DIV | BPF_X: > - case BPF_ALU64 | BPF_DIV | BPF_X: > - if (!off) { > - emit_zext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, is32); > - emit_insn(ctx, divdu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - emit_sext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_sext_32(ctx, t1, is32); > - emit_insn(ctx, divd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > + if (cpu_has_div32) { > + if (!off) > + emit_insn(ctx, divwu, dst, dst, src); > + else > + emit_insn(ctx, divw, dst, dst, src); For ALU32 SDIV, off != 0, it seems like there is a overflow in S32_MIN / -1. verifier_sdiv.c has related test cases but annotate with __arch_x86_64, so it is not run on LoongArch. > + break; > } > + /* > + * Legacy fallback for !div32: src must be extended into a > + * temporary register to keep the original src register intact. > + */ > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > + if (!off) > + emit_insn(ctx, divwu, dst, dst, t1); > + else > + emit_insn(ctx, divw, dst, dst, t1); > + break; > + case BPF_ALU64 | BPF_DIV | BPF_X: > + if (!off) > + emit_insn(ctx, divdu, dst, dst, src); > + else > + emit_insn(ctx, divd, dst, dst, src); > break; > > /* dst = dst / imm */ > case BPF_ALU | BPF_DIV | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + if (!cpu_has_div32) { > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > + } > + if (!off) > + emit_insn(ctx, divwu, dst, dst, t1); > + else > + emit_insn(ctx, divw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_DIV | BPF_K: > - if (!off) { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, dst, is32); > + move_imm(ctx, t1, imm); > + if (!off) > emit_insn(ctx, divdu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - move_imm(ctx, t1, imm); > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, dst, is32); > + else > emit_insn(ctx, divd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } > break; > > /* dst = dst % src */ > case BPF_ALU | BPF_MOD | BPF_X: > - case BPF_ALU64 | BPF_MOD | BPF_X: > - if (!off) { > - emit_zext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, is32); > - emit_insn(ctx, moddu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - emit_sext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_sext_32(ctx, t1, is32); > - emit_insn(ctx, modd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > + if (cpu_has_div32) { > + if (!off) > + emit_insn(ctx, modwu, dst, dst, src); > + else > + emit_insn(ctx, modw, dst, dst, src); > + break; > } > + /* > + * Legacy fallback for !div32: src must be extended into a > + * temporary register to keep the original src register intact. > + */ > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > + if (!off) > + emit_insn(ctx, modwu, dst, dst, t1); > + else > + emit_insn(ctx, modw, dst, dst, t1); > + break; > + case BPF_ALU64 | BPF_MOD | BPF_X: > + if (!off) > + emit_insn(ctx, moddu, dst, dst, src); > + else > + emit_insn(ctx, modd, dst, dst, src); > break; > > /* dst = dst % imm */ > case BPF_ALU | BPF_MOD | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + if (!cpu_has_div32) { > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > + } > + if (!off) > + emit_insn(ctx, modwu, dst, dst, t1); > + else > + emit_insn(ctx, modw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_MOD | BPF_K: > - if (!off) { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, dst, is32); > + move_imm(ctx, t1, imm); > + if (!off) > emit_insn(ctx, moddu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - move_imm(ctx, t1, imm); > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, dst, is32); > + else > emit_insn(ctx, modd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } > break; > > /* dst = -dst */ > case BPF_ALU | BPF_NEG: > + emit_insn(ctx, subw, dst, LOONGARCH_GPR_ZERO, dst); > + break; > case BPF_ALU64 | BPF_NEG: > emit_insn(ctx, subd, dst, LOONGARCH_GPR_ZERO, dst); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst & src */ > case BPF_ALU | BPF_AND | BPF_X: > case BPF_ALU64 | BPF_AND | BPF_X: > emit_insn(ctx, and, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst & imm */ > @@ -905,18 +931,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, andi, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, and, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst | src */ > case BPF_ALU | BPF_OR | BPF_X: > case BPF_ALU64 | BPF_OR | BPF_X: > emit_insn(ctx, or, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst | imm */ > @@ -925,18 +951,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, ori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, or, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst ^ src */ > case BPF_ALU | BPF_XOR | BPF_X: > case BPF_ALU64 | BPF_XOR | BPF_X: > emit_insn(ctx, xor, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst ^ imm */ > @@ -945,19 +971,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, xori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, xor, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst << src (logical) */ > case BPF_ALU | BPF_LSH | BPF_X: > emit_insn(ctx, sllw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_LSH | BPF_X: > emit_insn(ctx, slld, dst, dst, src); > break; > @@ -965,9 +990,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst << imm (logical) */ > case BPF_ALU | BPF_LSH | BPF_K: > emit_insn(ctx, slliw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_LSH | BPF_K: > emit_insn(ctx, sllid, dst, dst, imm); > break; > @@ -975,9 +998,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> src (logical) */ > case BPF_ALU | BPF_RSH | BPF_X: > emit_insn(ctx, srlw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_RSH | BPF_X: > emit_insn(ctx, srld, dst, dst, src); > break; > @@ -985,9 +1006,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> imm (logical) */ > case BPF_ALU | BPF_RSH | BPF_K: > emit_insn(ctx, srliw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_RSH | BPF_K: > emit_insn(ctx, srlid, dst, dst, imm); > break; > @@ -995,9 +1014,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> src (arithmetic) */ > case BPF_ALU | BPF_ARSH | BPF_X: > emit_insn(ctx, sraw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_ARSH | BPF_X: > emit_insn(ctx, srad, dst, dst, src); > break; > @@ -1005,9 +1022,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> imm (arithmetic) */ > case BPF_ALU | BPF_ARSH | BPF_K: > emit_insn(ctx, sraiw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_ARSH | BPF_K: > emit_insn(ctx, sraid, dst, dst, imm); > break; > @@ -1016,12 +1031,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU | BPF_END | BPF_FROM_LE: > switch (imm) { > case 16: > - /* zero-extend 16 bits into 64 bits */ > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > break; > case 32: > - /* zero-extend 32 bits into 64 bits */ > - emit_zext_32(ctx, dst, is32); > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > break; > case 64: > /* do nothing */ > @@ -1034,13 +1047,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > switch (imm) { > case 16: > emit_insn(ctx, revb2h, dst, dst); > - /* zero-extend 16 bits into 64 bits */ > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > break; > case 32: > emit_insn(ctx, revb2w, dst, dst); > - /* clear the upper 32 bits */ > - emit_zext_32(ctx, dst, true); > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > break; > case 64: > emit_insn(ctx, revbd, dst, dst); > @@ -1048,7 +1059,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > } > break; > > - /* PC += off if dst cond src */ > + /* PC += off if dst cond src (64-bit) */ > case BPF_JMP | BPF_JEQ | BPF_X: > case BPF_JMP | BPF_JNE | BPF_X: > case BPF_JMP | BPF_JGT | BPF_X: > @@ -1059,31 +1070,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP | BPF_JSGE | BPF_X: > case BPF_JMP | BPF_JSLT | BPF_X: > case BPF_JMP | BPF_JSLE | BPF_X: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (emit_cond_jmp(ctx, cond, dst, src, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond src (32-bit unsigned) */ > case BPF_JMP32 | BPF_JEQ | BPF_X: > case BPF_JMP32 | BPF_JNE | BPF_X: > case BPF_JMP32 | BPF_JGT | BPF_X: > case BPF_JMP32 | BPF_JGE | BPF_X: > case BPF_JMP32 | BPF_JLT | BPF_X: > case BPF_JMP32 | BPF_JLE | BPF_X: > + jmp_offset = bpf2la_offset(i, off, ctx); > + emit_insn(ctx, bstrpickd, t1, dst, 31, 0); > + emit_insn(ctx, bstrpickd, t2, src, 31, 0); > + if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond src (32-bit signed) */ > case BPF_JMP32 | BPF_JSGT | BPF_X: > case BPF_JMP32 | BPF_JSGE | BPF_X: > case BPF_JMP32 | BPF_JSLT | BPF_X: > case BPF_JMP32 | BPF_JSLE | BPF_X: > jmp_offset = bpf2la_offset(i, off, ctx); > - move_reg(ctx, t1, dst); > - move_reg(ctx, t2, src); > - if (is_signed_bpf_cond(BPF_OP(code))) { > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, t2, is32); > - } else { > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, t2, is32); > - } > + emit_insn(ctx, addw, t1, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t2, src, LOONGARCH_GPR_ZERO); > if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > goto toofar; > break; > > - /* PC += off if dst cond imm */ > + /* PC += off if dst cond imm (64-bit) */ > case BPF_JMP | BPF_JEQ | BPF_K: > case BPF_JMP | BPF_JNE | BPF_K: > case BPF_JMP | BPF_JGT | BPF_K: > @@ -1094,12 +1112,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP | BPF_JSGE | BPF_K: > case BPF_JMP | BPF_JSLT | BPF_K: > case BPF_JMP | BPF_JSLE | BPF_K: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (imm) { > + move_imm(ctx, t1, imm); > + tm = t1; > + } else { > + tm = LOONGARCH_GPR_ZERO; > + } > + if (emit_cond_jmp(ctx, cond, dst, tm, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond imm (32-bit unsigned) */ > case BPF_JMP32 | BPF_JEQ | BPF_K: > case BPF_JMP32 | BPF_JNE | BPF_K: > case BPF_JMP32 | BPF_JGT | BPF_K: > case BPF_JMP32 | BPF_JGE | BPF_K: > case BPF_JMP32 | BPF_JLT | BPF_K: > case BPF_JMP32 | BPF_JLE | BPF_K: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (imm) { > + move_imm(ctx, t1, imm); > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > + tm = t1; > + } else { > + tm = LOONGARCH_GPR_ZERO; > + } > + emit_insn(ctx, bstrpickd, t2, dst, 31, 0); > + if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond imm (32-bit signed) */ > case BPF_JMP32 | BPF_JSGT | BPF_K: > case BPF_JMP32 | BPF_JSGE | BPF_K: > case BPF_JMP32 | BPF_JSLT | BPF_K: > @@ -1107,19 +1151,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > jmp_offset = bpf2la_offset(i, off, ctx); > if (imm) { > move_imm(ctx, t1, imm); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > tm = t1; > } else { > - /* If imm is 0, simply use zero register. */ > tm = LOONGARCH_GPR_ZERO; > } > - move_reg(ctx, t2, dst); > - if (is_signed_bpf_cond(BPF_OP(code))) { > - emit_sext_32(ctx, tm, is32); > - emit_sext_32(ctx, t2, is32); > - } else { > - emit_zext_32(ctx, tm, is32); > - emit_zext_32(ctx, t2, is32); > - } > + emit_insn(ctx, addw, t2, dst, LOONGARCH_GPR_ZERO); > if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > goto toofar; > break; > @@ -1129,7 +1166,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP32 | BPF_JSET | BPF_X: > jmp_offset = bpf2la_offset(i, off, ctx); > emit_insn(ctx, and, t1, dst, src); > - emit_zext_32(ctx, t1, is32); > + if (BPF_CLASS(code) == BPF_JMP32) > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > goto toofar; > break; > @@ -1139,9 +1177,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP32 | BPF_JSET | BPF_K: > jmp_offset = bpf2la_offset(i, off, ctx); > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, and, t1, dst, t1); > - emit_zext_32(ctx, t1, is32); > + if (BPF_CLASS(code) == BPF_JMP32) > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > goto toofar; > break; > @@ -1468,6 +1506,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > return -EINVAL; > } > > + if (is32) > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > + > return 0; > > toofar: > diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h > index 57a98f8a055b..2bd0dc5b911c 100644 > --- a/arch/loongarch/net/bpf_jit.h > +++ b/arch/loongarch/net/bpf_jit.h > @@ -72,24 +72,6 @@ static inline int epilogue_offset(const struct jit_ctx *ctx) > return (to - from); > } > > -/* Zero-extend 32 bits into 64 bits */ > -static inline void emit_zext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > -{ > - if (!is32) > - return; > - > - emit_insn(ctx, lu32id, reg, 0); > -} > - > -/* Signed-extend 32 bits into 64 bits */ > -static inline void emit_sext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > -{ > - if (!is32) > - return; > - > - emit_insn(ctx, addiw, reg, reg, 0); > -} > - > /* Emit proper extension according to ABI requirements. > * Note that it requires a value of size `size` already resides in register `reg`. > */ > -- > 2.42.0 > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-27 2:02 ` Hengqi Chen @ 2026-10-27 4:10 ` Tiezhu Yang 2026-10-27 8:13 ` Hengqi Chen 0 siblings, 1 reply; 15+ messages in thread From: Tiezhu Yang @ 2026-10-27 4:10 UTC (permalink / raw) To: Hengqi Chen; +Cc: Huacai Chen, loongarch, bpf, linux-kernel On 2026/10/27 上午10:02, Hengqi Chen wrote: > On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: >> >> Currently, the LoongArch BPF JIT emits numerous redundant zero and sign >> extensions across 32-bit operations, bloating the final JIT image size >> and also leading to runtime overhead, particularly in the division and >> modulo operations. Thus, it is necessary to optimize 32-bit operations >> by emitting LoongArch native 32-bit word instructions. ... >> + if (cpu_has_div32) { >> + if (!off) >> + emit_insn(ctx, divwu, dst, dst, src); >> + else >> + emit_insn(ctx, divw, dst, dst, src); > > For ALU32 SDIV, off != 0, it seems like there is a overflow in S32_MIN / -1. > verifier_sdiv.c has related test cases but annotate with > __arch_x86_64, so it is not run on LoongArch. After enabling the signed division and modulo overflow tests for LoongArch, here are the test results: $ sudo ./test_progs -t verifier_sdiv | grep "MIN/-1" #731/121 verifier_sdiv/SDIV64, overflow r/r, LLONG_MIN/-1:OK #731/123 verifier_sdiv/SDIV64, overflow r/i, LLONG_MIN/-1:OK #731/125 verifier_sdiv/SDIV32, overflow r/r, INT_MIN/-1:OK #731/127 verifier_sdiv/SDIV32, overflow r/i, INT_MIN/-1:OK #731/129 verifier_sdiv/SMOD64, overflow r/r, LLONG_MIN/-1:OK #731/131 verifier_sdiv/SMOD64, overflow r/i, LLONG_MIN/-1:OK #731/133 verifier_sdiv/SMOD32, overflow r/r, INT_MIN/-1:OK #731/135 verifier_sdiv/SMOD32, overflow r/i, INT_MIN/-1:OK That is to say, the current jit code is no problem. Thanks, Tiezhu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-27 4:10 ` Tiezhu Yang @ 2026-10-27 8:13 ` Hengqi Chen 0 siblings, 0 replies; 15+ messages in thread From: Hengqi Chen @ 2026-10-27 8:13 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Huacai Chen, loongarch, bpf, linux-kernel On Tue, Oct 27, 2026 at 12:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > On 2026/10/27 上午10:02, Hengqi Chen wrote: > > On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > >> > >> Currently, the LoongArch BPF JIT emits numerous redundant zero and sign > >> extensions across 32-bit operations, bloating the final JIT image size > >> and also leading to runtime overhead, particularly in the division and > >> modulo operations. Thus, it is necessary to optimize 32-bit operations > >> by emitting LoongArch native 32-bit word instructions. > > ... > > >> + if (cpu_has_div32) { > >> + if (!off) > >> + emit_insn(ctx, divwu, dst, dst, src); > >> + else > >> + emit_insn(ctx, divw, dst, dst, src); > > > > For ALU32 SDIV, off != 0, it seems like there is a overflow in S32_MIN / -1. > > verifier_sdiv.c has related test cases but annotate with > > __arch_x86_64, so it is not run on LoongArch. > > After enabling the signed division and modulo overflow tests for > LoongArch, here are the test results: > > $ sudo ./test_progs -t verifier_sdiv | grep "MIN/-1" > #731/121 verifier_sdiv/SDIV64, overflow r/r, LLONG_MIN/-1:OK > #731/123 verifier_sdiv/SDIV64, overflow r/i, LLONG_MIN/-1:OK > #731/125 verifier_sdiv/SDIV32, overflow r/r, INT_MIN/-1:OK > #731/127 verifier_sdiv/SDIV32, overflow r/i, INT_MIN/-1:OK > #731/129 verifier_sdiv/SMOD64, overflow r/r, LLONG_MIN/-1:OK > #731/131 verifier_sdiv/SMOD64, overflow r/i, LLONG_MIN/-1:OK > #731/133 verifier_sdiv/SMOD32, overflow r/r, INT_MIN/-1:OK > #731/135 verifier_sdiv/SMOD32, overflow r/i, INT_MIN/-1:OK > > That is to say, the current jit code is no problem. > Great. Acked-by: Hengqi Chen <hengqi.chen@gmail.com> > Thanks, > Tiezhu > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang 2026-10-16 15:18 ` Alexei Starovoitov 2026-10-27 2:02 ` Hengqi Chen @ 2026-10-28 4:19 ` Huacai Chen 2026-10-28 6:55 ` Hengqi Chen 2 siblings, 1 reply; 15+ messages in thread From: Huacai Chen @ 2026-10-28 4:19 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Hengqi Chen, loongarch, bpf, linux-kernel Hi, Hengqi, On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > Currently, the LoongArch BPF JIT emits numerous redundant zero and sign > extensions across 32-bit operations, bloating the final JIT image size > and also leading to runtime overhead, particularly in the division and > modulo operations. Thus, it is necessary to optimize 32-bit operations > by emitting LoongArch native 32-bit word instructions. > > For general ALU32 operations, utilize native 32-bit word instructions > for core calculations. Given that the BPF spec requires the upper 32 > bits of the destination register to be zero-extended upon completion, > use a unified 'bstrpickd' instruction to clear the upper 32 bits at > the common exit of build_insn(), while immediate loads are optimized > via (u32) casting to bypass the trailing extension. > > According to the LoongArch Reference Manual, native 32-bit division > and modulo instructions (DIV.W[U] and MOD.W[U]) require the upper > bits [63:31] of both input operands to be all 0s or all 1s (0x0 or > 0x1ffffffff), that is to say, they need to be properly sign-extended > by the software on CPU cores without the DIV32 feature. Otherwise, > the 32-bit division and modulo instructions could trigger undefined > hardware behaviors and return unexpected values. > > To ensure both performance and hardware compatibility, leverage the > newly introduced cpu_has_div32 helper: > (1) on newer CPUs with DIV32, emit native 32-bit div/mod instructions > directly for optimal performance; > (2) on older CPUs without DIV32, sign-extend the destination register > in place and the source register into a temporary register via 'addw' > before executing the native 32-bit div/mod instructions. Note that the > JIT compiler must never alter the state of the src register because it > remains read-only for the duration of the instruction. > > For JMP32 conditional branches, the source operands may originate from > 64-bit contexts where the upper 32 bits contain arbitrary data, so use > 6 independent case structures to prevent 64-bit comparison inversions. > Furthermore, use 'bstrpickd' to mask out upper bits for unsigned JMP32, > and 'addw' with GPR_ZERO to re-align sign bits for signed JMP32. > > With this patch, the emit_{zext, sext}_32() and is_signed_bpf_cond() > helpers are completely wiped out from the header and source files, > delivering a native execution stream with a reduced JIT image size, > especially for division and mod operations. > > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > and Loongson-3A5000 (without DIV32), here are the test results: > > $ sudo modprobe test_bpf > $ dmesg -t | grep Summary > test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] > test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] > test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED > > For the BPF selftests test_verifier and test_progs, there are no any > regressions. > > Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> > --- > arch/loongarch/include/asm/inst.h | 6 + > arch/loongarch/net/bpf_jit.c | 313 +++++++++++++++++------------- > arch/loongarch/net/bpf_jit.h | 18 -- > 3 files changed, 183 insertions(+), 154 deletions(-) > > diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h > index 585667e361c2..b55aae325300 100644 > --- a/arch/loongarch/include/asm/inst.h > +++ b/arch/loongarch/include/asm/inst.h > @@ -752,11 +752,17 @@ static inline void emit_##NAME(union loongarch_instruction *insn, \ > > DEF_EMIT_REG3_FORMAT(addw, addw_op) > DEF_EMIT_REG3_FORMAT(addd, addd_op) > +DEF_EMIT_REG3_FORMAT(subw, subw_op) > DEF_EMIT_REG3_FORMAT(subd, subd_op) > +DEF_EMIT_REG3_FORMAT(mulw, mulw_op) > DEF_EMIT_REG3_FORMAT(muld, muld_op) > +DEF_EMIT_REG3_FORMAT(divw, divw_op) > DEF_EMIT_REG3_FORMAT(divd, divd_op) > +DEF_EMIT_REG3_FORMAT(modw, modw_op) > DEF_EMIT_REG3_FORMAT(modd, modd_op) > +DEF_EMIT_REG3_FORMAT(divwu, divwu_op) > DEF_EMIT_REG3_FORMAT(divdu, divdu_op) > +DEF_EMIT_REG3_FORMAT(modwu, modwu_op) > DEF_EMIT_REG3_FORMAT(moddu, moddu_op) > DEF_EMIT_REG3_FORMAT(maskeqz, maskeqz_op) > DEF_EMIT_REG3_FORMAT(masknez, masknez_op) > diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c > index 77cd96a6f84c..1a7c8effc37b 100644 > --- a/arch/loongarch/net/bpf_jit.c > +++ b/arch/loongarch/net/bpf_jit.c > @@ -288,7 +288,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) > * if (index >= array->map.max_entries) > * goto out; > */ > - emit_zext_32(ctx, a2, true); > + emit_insn(ctx, bstrpickd, a2, a2, 31, 0); > > off = offsetof(struct bpf_array, map.max_entries); > emit_insn(ctx, ldwu, t1, a1, off); > @@ -412,7 +412,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amadddbb, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_H: > if (!cpu_has_lam_bh) { > @@ -420,23 +419,23 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amadddbh, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_W: > emit_insn(ctx, amadddbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_DW: > emit_insn(ctx, amadddbd, src, t1, t3); > break; > } > + if (!isdw) > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > break; > case BPF_AND | BPF_FETCH: > if (isdw) { > emit_insn(ctx, amanddbd, src, t1, t3); > } else { > emit_insn(ctx, amanddbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > case BPF_OR | BPF_FETCH: > @@ -444,7 +443,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, amordbd, src, t1, t3); > } else { > emit_insn(ctx, amordbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > case BPF_XOR | BPF_FETCH: > @@ -452,7 +451,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, amxordbd, src, t1, t3); > } else { > emit_insn(ctx, amxordbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > } > break; > /* src = atomic_xchg(dst + off, src); */ > @@ -464,7 +463,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amswapdbb, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_H: > if (!cpu_has_lam_bh) { > @@ -472,16 +470,16 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > return -EINVAL; > } > emit_insn(ctx, amswapdbh, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_W: > emit_insn(ctx, amswapdbw, src, t1, t3); > - emit_zext_32(ctx, src, true); > break; > case BPF_DW: > emit_insn(ctx, amswapdbd, src, t1, t3); > break; > } > + if (!isdw) > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > break; > /* r0 = atomic_cmpxchg(dst + off, r0, src); */ > case BPF_CMPXCHG: > @@ -494,13 +492,13 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -4); > } else { > emit_insn(ctx, llw, r0, t1, 0); > - emit_zext_32(ctx, t2, true); > - emit_zext_32(ctx, r0, true); > + emit_insn(ctx, bstrpickd, t2, t2, 31, 0); > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > emit_insn(ctx, bne, t2, r0, 4); > move_reg(ctx, t3, src); > emit_insn(ctx, scw, t3, t1, 0); > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -6); > - emit_zext_32(ctx, r0, true); > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > } > emit_insn(ctx, dbar, DBAR_LLSC_MB); > break; > @@ -605,12 +603,6 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > return 0; > } > > -static bool is_signed_bpf_cond(u8 cond) > -{ > - return cond == BPF_JSGT || cond == BPF_JSLT || > - cond == BPF_JSGE || cond == BPF_JSLE; > -} > - > #define BPF_FIXUP_REG_MASK GENMASK(31, 27) > #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) > #define REG_DONT_CLEAR_MARKER 0 > @@ -707,7 +699,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > u8 dst = regmap[insn->dst_reg]; > const s16 off = insn->off; > const s32 imm = insn->imm; > - const bool is32 = BPF_CLASS(insn->code) == BPF_ALU || BPF_CLASS(insn->code) == BPF_JMP32; > + const bool is32 = BPF_CLASS(insn->code) == BPF_ALU && BPF_OP(insn->code) != BPF_END; > > switch (code) { > /* dst = src */ > @@ -715,7 +707,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU64 | BPF_MOV | BPF_X: > if (insn_is_cast_user(insn)) { > move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, true); > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); > emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); > emit_insn(ctx, or, t1, dst, t1); > @@ -734,15 +726,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > switch (off) { > case 0: > move_reg(ctx, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 8: > emit_insn(ctx, extwb, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 16: > emit_insn(ctx, extwh, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > case 32: > emit_insn(ctx, addw, dst, src, LOONGARCH_GPR_ZERO); > @@ -752,151 +741,188 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = imm */ > case BPF_ALU | BPF_MOV | BPF_K: > + move_imm(ctx, dst, (u32)imm); > + return 0; > case BPF_ALU64 | BPF_MOV | BPF_K: > move_imm(ctx, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; dst = imm is the simplest case, I think the 64bit version can also "return 0" here. If you agree, I will modify it when apply. Huacai > > /* dst = dst + src */ > case BPF_ALU | BPF_ADD | BPF_X: > + emit_insn(ctx, addw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_ADD | BPF_X: > emit_insn(ctx, addd, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst + imm */ > case BPF_ALU | BPF_ADD | BPF_K: > + if (is_signed_imm12(imm)) { > + emit_insn(ctx, addiw, dst, dst, imm); > + } else { > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, addw, dst, dst, t1); > + } > + break; > case BPF_ALU64 | BPF_ADD | BPF_K: > if (is_signed_imm12(imm)) { > emit_insn(ctx, addid, dst, dst, imm); > } else { > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, addd, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst - src */ > case BPF_ALU | BPF_SUB | BPF_X: > + emit_insn(ctx, subw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_SUB | BPF_X: > emit_insn(ctx, subd, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst - imm */ > case BPF_ALU | BPF_SUB | BPF_K: > + if (is_signed_imm12(-imm)) { > + emit_insn(ctx, addiw, dst, dst, -imm); > + } else { > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, subw, dst, dst, t1); > + } > + break; > case BPF_ALU64 | BPF_SUB | BPF_K: > if (is_signed_imm12(-imm)) { > emit_insn(ctx, addid, dst, dst, -imm); > } else { > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, subd, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst * src */ > case BPF_ALU | BPF_MUL | BPF_X: > + emit_insn(ctx, mulw, dst, dst, src); > + break; > case BPF_ALU64 | BPF_MUL | BPF_X: > emit_insn(ctx, muld, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst * imm */ > case BPF_ALU | BPF_MUL | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + emit_insn(ctx, mulw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_MUL | BPF_K: > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, muld, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst / src */ > case BPF_ALU | BPF_DIV | BPF_X: > - case BPF_ALU64 | BPF_DIV | BPF_X: > - if (!off) { > - emit_zext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, is32); > - emit_insn(ctx, divdu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - emit_sext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_sext_32(ctx, t1, is32); > - emit_insn(ctx, divd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > + if (cpu_has_div32) { > + if (!off) > + emit_insn(ctx, divwu, dst, dst, src); > + else > + emit_insn(ctx, divw, dst, dst, src); > + break; > } > + /* > + * Legacy fallback for !div32: src must be extended into a > + * temporary register to keep the original src register intact. > + */ > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > + if (!off) > + emit_insn(ctx, divwu, dst, dst, t1); > + else > + emit_insn(ctx, divw, dst, dst, t1); > + break; > + case BPF_ALU64 | BPF_DIV | BPF_X: > + if (!off) > + emit_insn(ctx, divdu, dst, dst, src); > + else > + emit_insn(ctx, divd, dst, dst, src); > break; > > /* dst = dst / imm */ > case BPF_ALU | BPF_DIV | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + if (!cpu_has_div32) { > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > + } > + if (!off) > + emit_insn(ctx, divwu, dst, dst, t1); > + else > + emit_insn(ctx, divw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_DIV | BPF_K: > - if (!off) { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, dst, is32); > + move_imm(ctx, t1, imm); > + if (!off) > emit_insn(ctx, divdu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - move_imm(ctx, t1, imm); > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, dst, is32); > + else > emit_insn(ctx, divd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } > break; > > /* dst = dst % src */ > case BPF_ALU | BPF_MOD | BPF_X: > - case BPF_ALU64 | BPF_MOD | BPF_X: > - if (!off) { > - emit_zext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_zext_32(ctx, t1, is32); > - emit_insn(ctx, moddu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - emit_sext_32(ctx, dst, is32); > - move_reg(ctx, t1, src); > - emit_sext_32(ctx, t1, is32); > - emit_insn(ctx, modd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > + if (cpu_has_div32) { > + if (!off) > + emit_insn(ctx, modwu, dst, dst, src); > + else > + emit_insn(ctx, modw, dst, dst, src); > + break; > } > + /* > + * Legacy fallback for !div32: src must be extended into a > + * temporary register to keep the original src register intact. > + */ > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > + if (!off) > + emit_insn(ctx, modwu, dst, dst, t1); > + else > + emit_insn(ctx, modw, dst, dst, t1); > + break; > + case BPF_ALU64 | BPF_MOD | BPF_X: > + if (!off) > + emit_insn(ctx, moddu, dst, dst, src); > + else > + emit_insn(ctx, modd, dst, dst, src); > break; > > /* dst = dst % imm */ > case BPF_ALU | BPF_MOD | BPF_K: > + move_imm(ctx, t1, (u32)imm); > + if (!cpu_has_div32) { > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > + } > + if (!off) > + emit_insn(ctx, modwu, dst, dst, t1); > + else > + emit_insn(ctx, modw, dst, dst, t1); > + break; > case BPF_ALU64 | BPF_MOD | BPF_K: > - if (!off) { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, dst, is32); > + move_imm(ctx, t1, imm); > + if (!off) > emit_insn(ctx, moddu, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } else { > - move_imm(ctx, t1, imm); > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, dst, is32); > + else > emit_insn(ctx, modd, dst, dst, t1); > - emit_zext_32(ctx, dst, is32); > - } > break; > > /* dst = -dst */ > case BPF_ALU | BPF_NEG: > + emit_insn(ctx, subw, dst, LOONGARCH_GPR_ZERO, dst); > + break; > case BPF_ALU64 | BPF_NEG: > emit_insn(ctx, subd, dst, LOONGARCH_GPR_ZERO, dst); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst & src */ > case BPF_ALU | BPF_AND | BPF_X: > case BPF_ALU64 | BPF_AND | BPF_X: > emit_insn(ctx, and, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst & imm */ > @@ -905,18 +931,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, andi, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, and, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst | src */ > case BPF_ALU | BPF_OR | BPF_X: > case BPF_ALU64 | BPF_OR | BPF_X: > emit_insn(ctx, or, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst | imm */ > @@ -925,18 +951,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, ori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, or, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst ^ src */ > case BPF_ALU | BPF_XOR | BPF_X: > case BPF_ALU64 | BPF_XOR | BPF_X: > emit_insn(ctx, xor, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst ^ imm */ > @@ -945,19 +971,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > if (is_unsigned_imm12(imm)) { > emit_insn(ctx, xori, dst, dst, imm); > } else { > - move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > + if (is32) > + move_imm(ctx, t1, (u32)imm); > + else > + move_imm(ctx, t1, imm); > emit_insn(ctx, xor, dst, dst, t1); > } > - emit_zext_32(ctx, dst, is32); > break; > > /* dst = dst << src (logical) */ > case BPF_ALU | BPF_LSH | BPF_X: > emit_insn(ctx, sllw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_LSH | BPF_X: > emit_insn(ctx, slld, dst, dst, src); > break; > @@ -965,9 +990,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst << imm (logical) */ > case BPF_ALU | BPF_LSH | BPF_K: > emit_insn(ctx, slliw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_LSH | BPF_K: > emit_insn(ctx, sllid, dst, dst, imm); > break; > @@ -975,9 +998,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> src (logical) */ > case BPF_ALU | BPF_RSH | BPF_X: > emit_insn(ctx, srlw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_RSH | BPF_X: > emit_insn(ctx, srld, dst, dst, src); > break; > @@ -985,9 +1006,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> imm (logical) */ > case BPF_ALU | BPF_RSH | BPF_K: > emit_insn(ctx, srliw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_RSH | BPF_K: > emit_insn(ctx, srlid, dst, dst, imm); > break; > @@ -995,9 +1014,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> src (arithmetic) */ > case BPF_ALU | BPF_ARSH | BPF_X: > emit_insn(ctx, sraw, dst, dst, src); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_ARSH | BPF_X: > emit_insn(ctx, srad, dst, dst, src); > break; > @@ -1005,9 +1022,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > /* dst = dst >> imm (arithmetic) */ > case BPF_ALU | BPF_ARSH | BPF_K: > emit_insn(ctx, sraiw, dst, dst, imm); > - emit_zext_32(ctx, dst, is32); > break; > - > case BPF_ALU64 | BPF_ARSH | BPF_K: > emit_insn(ctx, sraid, dst, dst, imm); > break; > @@ -1016,12 +1031,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_ALU | BPF_END | BPF_FROM_LE: > switch (imm) { > case 16: > - /* zero-extend 16 bits into 64 bits */ > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > break; > case 32: > - /* zero-extend 32 bits into 64 bits */ > - emit_zext_32(ctx, dst, is32); > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > break; > case 64: > /* do nothing */ > @@ -1034,13 +1047,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > switch (imm) { > case 16: > emit_insn(ctx, revb2h, dst, dst); > - /* zero-extend 16 bits into 64 bits */ > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > break; > case 32: > emit_insn(ctx, revb2w, dst, dst); > - /* clear the upper 32 bits */ > - emit_zext_32(ctx, dst, true); > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > break; > case 64: > emit_insn(ctx, revbd, dst, dst); > @@ -1048,7 +1059,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > } > break; > > - /* PC += off if dst cond src */ > + /* PC += off if dst cond src (64-bit) */ > case BPF_JMP | BPF_JEQ | BPF_X: > case BPF_JMP | BPF_JNE | BPF_X: > case BPF_JMP | BPF_JGT | BPF_X: > @@ -1059,31 +1070,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP | BPF_JSGE | BPF_X: > case BPF_JMP | BPF_JSLT | BPF_X: > case BPF_JMP | BPF_JSLE | BPF_X: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (emit_cond_jmp(ctx, cond, dst, src, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond src (32-bit unsigned) */ > case BPF_JMP32 | BPF_JEQ | BPF_X: > case BPF_JMP32 | BPF_JNE | BPF_X: > case BPF_JMP32 | BPF_JGT | BPF_X: > case BPF_JMP32 | BPF_JGE | BPF_X: > case BPF_JMP32 | BPF_JLT | BPF_X: > case BPF_JMP32 | BPF_JLE | BPF_X: > + jmp_offset = bpf2la_offset(i, off, ctx); > + emit_insn(ctx, bstrpickd, t1, dst, 31, 0); > + emit_insn(ctx, bstrpickd, t2, src, 31, 0); > + if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond src (32-bit signed) */ > case BPF_JMP32 | BPF_JSGT | BPF_X: > case BPF_JMP32 | BPF_JSGE | BPF_X: > case BPF_JMP32 | BPF_JSLT | BPF_X: > case BPF_JMP32 | BPF_JSLE | BPF_X: > jmp_offset = bpf2la_offset(i, off, ctx); > - move_reg(ctx, t1, dst); > - move_reg(ctx, t2, src); > - if (is_signed_bpf_cond(BPF_OP(code))) { > - emit_sext_32(ctx, t1, is32); > - emit_sext_32(ctx, t2, is32); > - } else { > - emit_zext_32(ctx, t1, is32); > - emit_zext_32(ctx, t2, is32); > - } > + emit_insn(ctx, addw, t1, dst, LOONGARCH_GPR_ZERO); > + emit_insn(ctx, addw, t2, src, LOONGARCH_GPR_ZERO); > if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > goto toofar; > break; > > - /* PC += off if dst cond imm */ > + /* PC += off if dst cond imm (64-bit) */ > case BPF_JMP | BPF_JEQ | BPF_K: > case BPF_JMP | BPF_JNE | BPF_K: > case BPF_JMP | BPF_JGT | BPF_K: > @@ -1094,12 +1112,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP | BPF_JSGE | BPF_K: > case BPF_JMP | BPF_JSLT | BPF_K: > case BPF_JMP | BPF_JSLE | BPF_K: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (imm) { > + move_imm(ctx, t1, imm); > + tm = t1; > + } else { > + tm = LOONGARCH_GPR_ZERO; > + } > + if (emit_cond_jmp(ctx, cond, dst, tm, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond imm (32-bit unsigned) */ > case BPF_JMP32 | BPF_JEQ | BPF_K: > case BPF_JMP32 | BPF_JNE | BPF_K: > case BPF_JMP32 | BPF_JGT | BPF_K: > case BPF_JMP32 | BPF_JGE | BPF_K: > case BPF_JMP32 | BPF_JLT | BPF_K: > case BPF_JMP32 | BPF_JLE | BPF_K: > + jmp_offset = bpf2la_offset(i, off, ctx); > + if (imm) { > + move_imm(ctx, t1, imm); > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > + tm = t1; > + } else { > + tm = LOONGARCH_GPR_ZERO; > + } > + emit_insn(ctx, bstrpickd, t2, dst, 31, 0); > + if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > + goto toofar; > + break; > + > + /* PC += off if dst cond imm (32-bit signed) */ > case BPF_JMP32 | BPF_JSGT | BPF_K: > case BPF_JMP32 | BPF_JSGE | BPF_K: > case BPF_JMP32 | BPF_JSLT | BPF_K: > @@ -1107,19 +1151,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > jmp_offset = bpf2la_offset(i, off, ctx); > if (imm) { > move_imm(ctx, t1, imm); > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > tm = t1; > } else { > - /* If imm is 0, simply use zero register. */ > tm = LOONGARCH_GPR_ZERO; > } > - move_reg(ctx, t2, dst); > - if (is_signed_bpf_cond(BPF_OP(code))) { > - emit_sext_32(ctx, tm, is32); > - emit_sext_32(ctx, t2, is32); > - } else { > - emit_zext_32(ctx, tm, is32); > - emit_zext_32(ctx, t2, is32); > - } > + emit_insn(ctx, addw, t2, dst, LOONGARCH_GPR_ZERO); > if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > goto toofar; > break; > @@ -1129,7 +1166,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP32 | BPF_JSET | BPF_X: > jmp_offset = bpf2la_offset(i, off, ctx); > emit_insn(ctx, and, t1, dst, src); > - emit_zext_32(ctx, t1, is32); > + if (BPF_CLASS(code) == BPF_JMP32) > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > goto toofar; > break; > @@ -1139,9 +1177,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > case BPF_JMP32 | BPF_JSET | BPF_K: > jmp_offset = bpf2la_offset(i, off, ctx); > move_imm(ctx, t1, imm); > - emit_zext_32(ctx, t1, is32); > emit_insn(ctx, and, t1, dst, t1); > - emit_zext_32(ctx, t1, is32); > + if (BPF_CLASS(code) == BPF_JMP32) > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > goto toofar; > break; > @@ -1468,6 +1506,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > return -EINVAL; > } > > + if (is32) > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > + > return 0; > > toofar: > diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h > index 57a98f8a055b..2bd0dc5b911c 100644 > --- a/arch/loongarch/net/bpf_jit.h > +++ b/arch/loongarch/net/bpf_jit.h > @@ -72,24 +72,6 @@ static inline int epilogue_offset(const struct jit_ctx *ctx) > return (to - from); > } > > -/* Zero-extend 32 bits into 64 bits */ > -static inline void emit_zext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > -{ > - if (!is32) > - return; > - > - emit_insn(ctx, lu32id, reg, 0); > -} > - > -/* Signed-extend 32 bits into 64 bits */ > -static inline void emit_sext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > -{ > - if (!is32) > - return; > - > - emit_insn(ctx, addiw, reg, reg, 0); > -} > - > /* Emit proper extension according to ABI requirements. > * Note that it requires a value of size `size` already resides in register `reg`. > */ > -- > 2.42.0 > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions 2026-10-28 4:19 ` Huacai Chen @ 2026-10-28 6:55 ` Hengqi Chen 0 siblings, 0 replies; 15+ messages in thread From: Hengqi Chen @ 2026-10-28 6:55 UTC (permalink / raw) To: Huacai Chen; +Cc: Tiezhu Yang, loongarch, bpf, linux-kernel On Wed, Oct 28, 2026 at 12:19 PM Huacai Chen <chenhuacai@kernel.org> wrote: > > Hi, Hengqi, > > On Fri, Oct 16, 2026 at 4:10 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > > > Currently, the LoongArch BPF JIT emits numerous redundant zero and sign > > extensions across 32-bit operations, bloating the final JIT image size > > and also leading to runtime overhead, particularly in the division and > > modulo operations. Thus, it is necessary to optimize 32-bit operations > > by emitting LoongArch native 32-bit word instructions. > > > > For general ALU32 operations, utilize native 32-bit word instructions > > for core calculations. Given that the BPF spec requires the upper 32 > > bits of the destination register to be zero-extended upon completion, > > use a unified 'bstrpickd' instruction to clear the upper 32 bits at > > the common exit of build_insn(), while immediate loads are optimized > > via (u32) casting to bypass the trailing extension. > > > > According to the LoongArch Reference Manual, native 32-bit division > > and modulo instructions (DIV.W[U] and MOD.W[U]) require the upper > > bits [63:31] of both input operands to be all 0s or all 1s (0x0 or > > 0x1ffffffff), that is to say, they need to be properly sign-extended > > by the software on CPU cores without the DIV32 feature. Otherwise, > > the 32-bit division and modulo instructions could trigger undefined > > hardware behaviors and return unexpected values. > > > > To ensure both performance and hardware compatibility, leverage the > > newly introduced cpu_has_div32 helper: > > (1) on newer CPUs with DIV32, emit native 32-bit div/mod instructions > > directly for optimal performance; > > (2) on older CPUs without DIV32, sign-extend the destination register > > in place and the source register into a temporary register via 'addw' > > before executing the native 32-bit div/mod instructions. Note that the > > JIT compiler must never alter the state of the src register because it > > remains read-only for the duration of the instruction. > > > > For JMP32 conditional branches, the source operands may originate from > > 64-bit contexts where the upper 32 bits contain arbitrary data, so use > > 6 independent case structures to prevent 64-bit comparison inversions. > > Furthermore, use 'bstrpickd' to mask out upper bits for unsigned JMP32, > > and 'addw' with GPR_ZERO to re-align sign bits for signed JMP32. > > > > With this patch, the emit_{zext, sext}_32() and is_signed_bpf_cond() > > helpers are completely wiped out from the header and source files, > > delivering a native execution stream with a reduced JIT image size, > > especially for division and mod operations. > > > > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > > and Loongson-3A5000 (without DIV32), here are the test results: > > > > $ sudo modprobe test_bpf > > $ dmesg -t | grep Summary > > test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] > > test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] > > test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED > > > > For the BPF selftests test_verifier and test_progs, there are no any > > regressions. > > > > Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> > > --- > > arch/loongarch/include/asm/inst.h | 6 + > > arch/loongarch/net/bpf_jit.c | 313 +++++++++++++++++------------- > > arch/loongarch/net/bpf_jit.h | 18 -- > > 3 files changed, 183 insertions(+), 154 deletions(-) > > > > diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h > > index 585667e361c2..b55aae325300 100644 > > --- a/arch/loongarch/include/asm/inst.h > > +++ b/arch/loongarch/include/asm/inst.h > > @@ -752,11 +752,17 @@ static inline void emit_##NAME(union loongarch_instruction *insn, \ > > > > DEF_EMIT_REG3_FORMAT(addw, addw_op) > > DEF_EMIT_REG3_FORMAT(addd, addd_op) > > +DEF_EMIT_REG3_FORMAT(subw, subw_op) > > DEF_EMIT_REG3_FORMAT(subd, subd_op) > > +DEF_EMIT_REG3_FORMAT(mulw, mulw_op) > > DEF_EMIT_REG3_FORMAT(muld, muld_op) > > +DEF_EMIT_REG3_FORMAT(divw, divw_op) > > DEF_EMIT_REG3_FORMAT(divd, divd_op) > > +DEF_EMIT_REG3_FORMAT(modw, modw_op) > > DEF_EMIT_REG3_FORMAT(modd, modd_op) > > +DEF_EMIT_REG3_FORMAT(divwu, divwu_op) > > DEF_EMIT_REG3_FORMAT(divdu, divdu_op) > > +DEF_EMIT_REG3_FORMAT(modwu, modwu_op) > > DEF_EMIT_REG3_FORMAT(moddu, moddu_op) > > DEF_EMIT_REG3_FORMAT(maskeqz, maskeqz_op) > > DEF_EMIT_REG3_FORMAT(masknez, masknez_op) > > diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c > > index 77cd96a6f84c..1a7c8effc37b 100644 > > --- a/arch/loongarch/net/bpf_jit.c > > +++ b/arch/loongarch/net/bpf_jit.c > > @@ -288,7 +288,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn) > > * if (index >= array->map.max_entries) > > * goto out; > > */ > > - emit_zext_32(ctx, a2, true); > > + emit_insn(ctx, bstrpickd, a2, a2, 31, 0); > > > > off = offsetof(struct bpf_array, map.max_entries); > > emit_insn(ctx, ldwu, t1, a1, off); > > @@ -412,7 +412,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > return -EINVAL; > > } > > emit_insn(ctx, amadddbb, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_H: > > if (!cpu_has_lam_bh) { > > @@ -420,23 +419,23 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > return -EINVAL; > > } > > emit_insn(ctx, amadddbh, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_W: > > emit_insn(ctx, amadddbw, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_DW: > > emit_insn(ctx, amadddbd, src, t1, t3); > > break; > > } > > + if (!isdw) > > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > > break; > > case BPF_AND | BPF_FETCH: > > if (isdw) { > > emit_insn(ctx, amanddbd, src, t1, t3); > > } else { > > emit_insn(ctx, amanddbw, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > > } > > break; > > case BPF_OR | BPF_FETCH: > > @@ -444,7 +443,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > emit_insn(ctx, amordbd, src, t1, t3); > > } else { > > emit_insn(ctx, amordbw, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > > } > > break; > > case BPF_XOR | BPF_FETCH: > > @@ -452,7 +451,7 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > emit_insn(ctx, amxordbd, src, t1, t3); > > } else { > > emit_insn(ctx, amxordbw, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > > } > > break; > > /* src = atomic_xchg(dst + off, src); */ > > @@ -464,7 +463,6 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > return -EINVAL; > > } > > emit_insn(ctx, amswapdbb, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_H: > > if (!cpu_has_lam_bh) { > > @@ -472,16 +470,16 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > return -EINVAL; > > } > > emit_insn(ctx, amswapdbh, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_W: > > emit_insn(ctx, amswapdbw, src, t1, t3); > > - emit_zext_32(ctx, src, true); > > break; > > case BPF_DW: > > emit_insn(ctx, amswapdbd, src, t1, t3); > > break; > > } > > + if (!isdw) > > + emit_insn(ctx, bstrpickd, src, src, 31, 0); > > break; > > /* r0 = atomic_cmpxchg(dst + off, r0, src); */ > > case BPF_CMPXCHG: > > @@ -494,13 +492,13 @@ static int emit_atomic_rmw(const struct bpf_insn *insn, struct jit_ctx *ctx) > > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -4); > > } else { > > emit_insn(ctx, llw, r0, t1, 0); > > - emit_zext_32(ctx, t2, true); > > - emit_zext_32(ctx, r0, true); > > + emit_insn(ctx, bstrpickd, t2, t2, 31, 0); > > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > > emit_insn(ctx, bne, t2, r0, 4); > > move_reg(ctx, t3, src); > > emit_insn(ctx, scw, t3, t1, 0); > > emit_insn(ctx, beq, t3, LOONGARCH_GPR_ZERO, -6); > > - emit_zext_32(ctx, r0, true); > > + emit_insn(ctx, bstrpickd, r0, r0, 31, 0); > > } > > emit_insn(ctx, dbar, DBAR_LLSC_MB); > > break; > > @@ -605,12 +603,6 @@ static int emit_atomic_ld_st(const struct bpf_insn *insn, struct jit_ctx *ctx) > > return 0; > > } > > > > -static bool is_signed_bpf_cond(u8 cond) > > -{ > > - return cond == BPF_JSGT || cond == BPF_JSLT || > > - cond == BPF_JSGE || cond == BPF_JSLE; > > -} > > - > > #define BPF_FIXUP_REG_MASK GENMASK(31, 27) > > #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) > > #define REG_DONT_CLEAR_MARKER 0 > > @@ -707,7 +699,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > u8 dst = regmap[insn->dst_reg]; > > const s16 off = insn->off; > > const s32 imm = insn->imm; > > - const bool is32 = BPF_CLASS(insn->code) == BPF_ALU || BPF_CLASS(insn->code) == BPF_JMP32; > > + const bool is32 = BPF_CLASS(insn->code) == BPF_ALU && BPF_OP(insn->code) != BPF_END; > > > > switch (code) { > > /* dst = src */ > > @@ -715,7 +707,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_ALU64 | BPF_MOV | BPF_X: > > if (insn_is_cast_user(insn)) { > > move_reg(ctx, t1, src); > > - emit_zext_32(ctx, t1, true); > > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > > move_imm(ctx, dst, (ctx->user_vm_start >> 32) << 32); > > emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 2); > > emit_insn(ctx, or, t1, dst, t1); > > @@ -734,15 +726,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > switch (off) { > > case 0: > > move_reg(ctx, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > case 8: > > emit_insn(ctx, extwb, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > case 16: > > emit_insn(ctx, extwh, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > case 32: > > emit_insn(ctx, addw, dst, src, LOONGARCH_GPR_ZERO); > > @@ -752,151 +741,188 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > > > /* dst = imm */ > > case BPF_ALU | BPF_MOV | BPF_K: > > + move_imm(ctx, dst, (u32)imm); > > + return 0; > > case BPF_ALU64 | BPF_MOV | BPF_K: > > move_imm(ctx, dst, imm); > > - emit_zext_32(ctx, dst, is32); > > break; > dst = imm is the simplest case, I think the 64bit version can also > "return 0" here. If you agree, I will modify it when apply. > I am fine with the change. Cheers. > Huacai > > > > > /* dst = dst + src */ > > case BPF_ALU | BPF_ADD | BPF_X: > > + emit_insn(ctx, addw, dst, dst, src); > > + break; > > case BPF_ALU64 | BPF_ADD | BPF_X: > > emit_insn(ctx, addd, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst + imm */ > > case BPF_ALU | BPF_ADD | BPF_K: > > + if (is_signed_imm12(imm)) { > > + emit_insn(ctx, addiw, dst, dst, imm); > > + } else { > > + move_imm(ctx, t1, (u32)imm); > > + emit_insn(ctx, addw, dst, dst, t1); > > + } > > + break; > > case BPF_ALU64 | BPF_ADD | BPF_K: > > if (is_signed_imm12(imm)) { > > emit_insn(ctx, addid, dst, dst, imm); > > } else { > > move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > emit_insn(ctx, addd, dst, dst, t1); > > } > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst - src */ > > case BPF_ALU | BPF_SUB | BPF_X: > > + emit_insn(ctx, subw, dst, dst, src); > > + break; > > case BPF_ALU64 | BPF_SUB | BPF_X: > > emit_insn(ctx, subd, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst - imm */ > > case BPF_ALU | BPF_SUB | BPF_K: > > + if (is_signed_imm12(-imm)) { > > + emit_insn(ctx, addiw, dst, dst, -imm); > > + } else { > > + move_imm(ctx, t1, (u32)imm); > > + emit_insn(ctx, subw, dst, dst, t1); > > + } > > + break; > > case BPF_ALU64 | BPF_SUB | BPF_K: > > if (is_signed_imm12(-imm)) { > > emit_insn(ctx, addid, dst, dst, -imm); > > } else { > > move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > emit_insn(ctx, subd, dst, dst, t1); > > } > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst * src */ > > case BPF_ALU | BPF_MUL | BPF_X: > > + emit_insn(ctx, mulw, dst, dst, src); > > + break; > > case BPF_ALU64 | BPF_MUL | BPF_X: > > emit_insn(ctx, muld, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst * imm */ > > case BPF_ALU | BPF_MUL | BPF_K: > > + move_imm(ctx, t1, (u32)imm); > > + emit_insn(ctx, mulw, dst, dst, t1); > > + break; > > case BPF_ALU64 | BPF_MUL | BPF_K: > > move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > emit_insn(ctx, muld, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst / src */ > > case BPF_ALU | BPF_DIV | BPF_X: > > - case BPF_ALU64 | BPF_DIV | BPF_X: > > - if (!off) { > > - emit_zext_32(ctx, dst, is32); > > - move_reg(ctx, t1, src); > > - emit_zext_32(ctx, t1, is32); > > - emit_insn(ctx, divdu, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } else { > > - emit_sext_32(ctx, dst, is32); > > - move_reg(ctx, t1, src); > > - emit_sext_32(ctx, t1, is32); > > - emit_insn(ctx, divd, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > + if (cpu_has_div32) { > > + if (!off) > > + emit_insn(ctx, divwu, dst, dst, src); > > + else > > + emit_insn(ctx, divw, dst, dst, src); > > + break; > > } > > + /* > > + * Legacy fallback for !div32: src must be extended into a > > + * temporary register to keep the original src register intact. > > + */ > > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > > + if (!off) > > + emit_insn(ctx, divwu, dst, dst, t1); > > + else > > + emit_insn(ctx, divw, dst, dst, t1); > > + break; > > + case BPF_ALU64 | BPF_DIV | BPF_X: > > + if (!off) > > + emit_insn(ctx, divdu, dst, dst, src); > > + else > > + emit_insn(ctx, divd, dst, dst, src); > > break; > > > > /* dst = dst / imm */ > > case BPF_ALU | BPF_DIV | BPF_K: > > + move_imm(ctx, t1, (u32)imm); > > + if (!cpu_has_div32) { > > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > > + } > > + if (!off) > > + emit_insn(ctx, divwu, dst, dst, t1); > > + else > > + emit_insn(ctx, divw, dst, dst, t1); > > + break; > > case BPF_ALU64 | BPF_DIV | BPF_K: > > - if (!off) { > > - move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > - emit_zext_32(ctx, dst, is32); > > + move_imm(ctx, t1, imm); > > + if (!off) > > emit_insn(ctx, divdu, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } else { > > - move_imm(ctx, t1, imm); > > - emit_sext_32(ctx, t1, is32); > > - emit_sext_32(ctx, dst, is32); > > + else > > emit_insn(ctx, divd, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } > > break; > > > > /* dst = dst % src */ > > case BPF_ALU | BPF_MOD | BPF_X: > > - case BPF_ALU64 | BPF_MOD | BPF_X: > > - if (!off) { > > - emit_zext_32(ctx, dst, is32); > > - move_reg(ctx, t1, src); > > - emit_zext_32(ctx, t1, is32); > > - emit_insn(ctx, moddu, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } else { > > - emit_sext_32(ctx, dst, is32); > > - move_reg(ctx, t1, src); > > - emit_sext_32(ctx, t1, is32); > > - emit_insn(ctx, modd, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > + if (cpu_has_div32) { > > + if (!off) > > + emit_insn(ctx, modwu, dst, dst, src); > > + else > > + emit_insn(ctx, modw, dst, dst, src); > > + break; > > } > > + /* > > + * Legacy fallback for !div32: src must be extended into a > > + * temporary register to keep the original src register intact. > > + */ > > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > > + emit_insn(ctx, addw, t1, src, LOONGARCH_GPR_ZERO); > > + if (!off) > > + emit_insn(ctx, modwu, dst, dst, t1); > > + else > > + emit_insn(ctx, modw, dst, dst, t1); > > + break; > > + case BPF_ALU64 | BPF_MOD | BPF_X: > > + if (!off) > > + emit_insn(ctx, moddu, dst, dst, src); > > + else > > + emit_insn(ctx, modd, dst, dst, src); > > break; > > > > /* dst = dst % imm */ > > case BPF_ALU | BPF_MOD | BPF_K: > > + move_imm(ctx, t1, (u32)imm); > > + if (!cpu_has_div32) { > > + emit_insn(ctx, addw, dst, dst, LOONGARCH_GPR_ZERO); > > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > > + } > > + if (!off) > > + emit_insn(ctx, modwu, dst, dst, t1); > > + else > > + emit_insn(ctx, modw, dst, dst, t1); > > + break; > > case BPF_ALU64 | BPF_MOD | BPF_K: > > - if (!off) { > > - move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > - emit_zext_32(ctx, dst, is32); > > + move_imm(ctx, t1, imm); > > + if (!off) > > emit_insn(ctx, moddu, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } else { > > - move_imm(ctx, t1, imm); > > - emit_sext_32(ctx, t1, is32); > > - emit_sext_32(ctx, dst, is32); > > + else > > emit_insn(ctx, modd, dst, dst, t1); > > - emit_zext_32(ctx, dst, is32); > > - } > > break; > > > > /* dst = -dst */ > > case BPF_ALU | BPF_NEG: > > + emit_insn(ctx, subw, dst, LOONGARCH_GPR_ZERO, dst); > > + break; > > case BPF_ALU64 | BPF_NEG: > > emit_insn(ctx, subd, dst, LOONGARCH_GPR_ZERO, dst); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst & src */ > > case BPF_ALU | BPF_AND | BPF_X: > > case BPF_ALU64 | BPF_AND | BPF_X: > > emit_insn(ctx, and, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst & imm */ > > @@ -905,18 +931,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > if (is_unsigned_imm12(imm)) { > > emit_insn(ctx, andi, dst, dst, imm); > > } else { > > - move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > + if (is32) > > + move_imm(ctx, t1, (u32)imm); > > + else > > + move_imm(ctx, t1, imm); > > emit_insn(ctx, and, dst, dst, t1); > > } > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst | src */ > > case BPF_ALU | BPF_OR | BPF_X: > > case BPF_ALU64 | BPF_OR | BPF_X: > > emit_insn(ctx, or, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst | imm */ > > @@ -925,18 +951,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > if (is_unsigned_imm12(imm)) { > > emit_insn(ctx, ori, dst, dst, imm); > > } else { > > - move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > + if (is32) > > + move_imm(ctx, t1, (u32)imm); > > + else > > + move_imm(ctx, t1, imm); > > emit_insn(ctx, or, dst, dst, t1); > > } > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst ^ src */ > > case BPF_ALU | BPF_XOR | BPF_X: > > case BPF_ALU64 | BPF_XOR | BPF_X: > > emit_insn(ctx, xor, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst ^ imm */ > > @@ -945,19 +971,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > if (is_unsigned_imm12(imm)) { > > emit_insn(ctx, xori, dst, dst, imm); > > } else { > > - move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > + if (is32) > > + move_imm(ctx, t1, (u32)imm); > > + else > > + move_imm(ctx, t1, imm); > > emit_insn(ctx, xor, dst, dst, t1); > > } > > - emit_zext_32(ctx, dst, is32); > > break; > > > > /* dst = dst << src (logical) */ > > case BPF_ALU | BPF_LSH | BPF_X: > > emit_insn(ctx, sllw, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_LSH | BPF_X: > > emit_insn(ctx, slld, dst, dst, src); > > break; > > @@ -965,9 +990,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = dst << imm (logical) */ > > case BPF_ALU | BPF_LSH | BPF_K: > > emit_insn(ctx, slliw, dst, dst, imm); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_LSH | BPF_K: > > emit_insn(ctx, sllid, dst, dst, imm); > > break; > > @@ -975,9 +998,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = dst >> src (logical) */ > > case BPF_ALU | BPF_RSH | BPF_X: > > emit_insn(ctx, srlw, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_RSH | BPF_X: > > emit_insn(ctx, srld, dst, dst, src); > > break; > > @@ -985,9 +1006,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = dst >> imm (logical) */ > > case BPF_ALU | BPF_RSH | BPF_K: > > emit_insn(ctx, srliw, dst, dst, imm); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_RSH | BPF_K: > > emit_insn(ctx, srlid, dst, dst, imm); > > break; > > @@ -995,9 +1014,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = dst >> src (arithmetic) */ > > case BPF_ALU | BPF_ARSH | BPF_X: > > emit_insn(ctx, sraw, dst, dst, src); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_ARSH | BPF_X: > > emit_insn(ctx, srad, dst, dst, src); > > break; > > @@ -1005,9 +1022,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > /* dst = dst >> imm (arithmetic) */ > > case BPF_ALU | BPF_ARSH | BPF_K: > > emit_insn(ctx, sraiw, dst, dst, imm); > > - emit_zext_32(ctx, dst, is32); > > break; > > - > > case BPF_ALU64 | BPF_ARSH | BPF_K: > > emit_insn(ctx, sraid, dst, dst, imm); > > break; > > @@ -1016,12 +1031,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_ALU | BPF_END | BPF_FROM_LE: > > switch (imm) { > > case 16: > > - /* zero-extend 16 bits into 64 bits */ > > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > > break; > > case 32: > > - /* zero-extend 32 bits into 64 bits */ > > - emit_zext_32(ctx, dst, is32); > > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > > break; > > case 64: > > /* do nothing */ > > @@ -1034,13 +1047,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > switch (imm) { > > case 16: > > emit_insn(ctx, revb2h, dst, dst); > > - /* zero-extend 16 bits into 64 bits */ > > emit_insn(ctx, bstrpickd, dst, dst, 15, 0); > > break; > > case 32: > > emit_insn(ctx, revb2w, dst, dst); > > - /* clear the upper 32 bits */ > > - emit_zext_32(ctx, dst, true); > > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > > break; > > case 64: > > emit_insn(ctx, revbd, dst, dst); > > @@ -1048,7 +1059,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > } > > break; > > > > - /* PC += off if dst cond src */ > > + /* PC += off if dst cond src (64-bit) */ > > case BPF_JMP | BPF_JEQ | BPF_X: > > case BPF_JMP | BPF_JNE | BPF_X: > > case BPF_JMP | BPF_JGT | BPF_X: > > @@ -1059,31 +1070,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_JMP | BPF_JSGE | BPF_X: > > case BPF_JMP | BPF_JSLT | BPF_X: > > case BPF_JMP | BPF_JSLE | BPF_X: > > + jmp_offset = bpf2la_offset(i, off, ctx); > > + if (emit_cond_jmp(ctx, cond, dst, src, jmp_offset) < 0) > > + goto toofar; > > + break; > > + > > + /* PC += off if dst cond src (32-bit unsigned) */ > > case BPF_JMP32 | BPF_JEQ | BPF_X: > > case BPF_JMP32 | BPF_JNE | BPF_X: > > case BPF_JMP32 | BPF_JGT | BPF_X: > > case BPF_JMP32 | BPF_JGE | BPF_X: > > case BPF_JMP32 | BPF_JLT | BPF_X: > > case BPF_JMP32 | BPF_JLE | BPF_X: > > + jmp_offset = bpf2la_offset(i, off, ctx); > > + emit_insn(ctx, bstrpickd, t1, dst, 31, 0); > > + emit_insn(ctx, bstrpickd, t2, src, 31, 0); > > + if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > > + goto toofar; > > + break; > > + > > + /* PC += off if dst cond src (32-bit signed) */ > > case BPF_JMP32 | BPF_JSGT | BPF_X: > > case BPF_JMP32 | BPF_JSGE | BPF_X: > > case BPF_JMP32 | BPF_JSLT | BPF_X: > > case BPF_JMP32 | BPF_JSLE | BPF_X: > > jmp_offset = bpf2la_offset(i, off, ctx); > > - move_reg(ctx, t1, dst); > > - move_reg(ctx, t2, src); > > - if (is_signed_bpf_cond(BPF_OP(code))) { > > - emit_sext_32(ctx, t1, is32); > > - emit_sext_32(ctx, t2, is32); > > - } else { > > - emit_zext_32(ctx, t1, is32); > > - emit_zext_32(ctx, t2, is32); > > - } > > + emit_insn(ctx, addw, t1, dst, LOONGARCH_GPR_ZERO); > > + emit_insn(ctx, addw, t2, src, LOONGARCH_GPR_ZERO); > > if (emit_cond_jmp(ctx, cond, t1, t2, jmp_offset) < 0) > > goto toofar; > > break; > > > > - /* PC += off if dst cond imm */ > > + /* PC += off if dst cond imm (64-bit) */ > > case BPF_JMP | BPF_JEQ | BPF_K: > > case BPF_JMP | BPF_JNE | BPF_K: > > case BPF_JMP | BPF_JGT | BPF_K: > > @@ -1094,12 +1112,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_JMP | BPF_JSGE | BPF_K: > > case BPF_JMP | BPF_JSLT | BPF_K: > > case BPF_JMP | BPF_JSLE | BPF_K: > > + jmp_offset = bpf2la_offset(i, off, ctx); > > + if (imm) { > > + move_imm(ctx, t1, imm); > > + tm = t1; > > + } else { > > + tm = LOONGARCH_GPR_ZERO; > > + } > > + if (emit_cond_jmp(ctx, cond, dst, tm, jmp_offset) < 0) > > + goto toofar; > > + break; > > + > > + /* PC += off if dst cond imm (32-bit unsigned) */ > > case BPF_JMP32 | BPF_JEQ | BPF_K: > > case BPF_JMP32 | BPF_JNE | BPF_K: > > case BPF_JMP32 | BPF_JGT | BPF_K: > > case BPF_JMP32 | BPF_JGE | BPF_K: > > case BPF_JMP32 | BPF_JLT | BPF_K: > > case BPF_JMP32 | BPF_JLE | BPF_K: > > + jmp_offset = bpf2la_offset(i, off, ctx); > > + if (imm) { > > + move_imm(ctx, t1, imm); > > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > > + tm = t1; > > + } else { > > + tm = LOONGARCH_GPR_ZERO; > > + } > > + emit_insn(ctx, bstrpickd, t2, dst, 31, 0); > > + if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > > + goto toofar; > > + break; > > + > > + /* PC += off if dst cond imm (32-bit signed) */ > > case BPF_JMP32 | BPF_JSGT | BPF_K: > > case BPF_JMP32 | BPF_JSGE | BPF_K: > > case BPF_JMP32 | BPF_JSLT | BPF_K: > > @@ -1107,19 +1151,12 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > jmp_offset = bpf2la_offset(i, off, ctx); > > if (imm) { > > move_imm(ctx, t1, imm); > > + emit_insn(ctx, addw, t1, t1, LOONGARCH_GPR_ZERO); > > tm = t1; > > } else { > > - /* If imm is 0, simply use zero register. */ > > tm = LOONGARCH_GPR_ZERO; > > } > > - move_reg(ctx, t2, dst); > > - if (is_signed_bpf_cond(BPF_OP(code))) { > > - emit_sext_32(ctx, tm, is32); > > - emit_sext_32(ctx, t2, is32); > > - } else { > > - emit_zext_32(ctx, tm, is32); > > - emit_zext_32(ctx, t2, is32); > > - } > > + emit_insn(ctx, addw, t2, dst, LOONGARCH_GPR_ZERO); > > if (emit_cond_jmp(ctx, cond, t2, tm, jmp_offset) < 0) > > goto toofar; > > break; > > @@ -1129,7 +1166,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_JMP32 | BPF_JSET | BPF_X: > > jmp_offset = bpf2la_offset(i, off, ctx); > > emit_insn(ctx, and, t1, dst, src); > > - emit_zext_32(ctx, t1, is32); > > + if (BPF_CLASS(code) == BPF_JMP32) > > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > > goto toofar; > > break; > > @@ -1139,9 +1177,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > case BPF_JMP32 | BPF_JSET | BPF_K: > > jmp_offset = bpf2la_offset(i, off, ctx); > > move_imm(ctx, t1, imm); > > - emit_zext_32(ctx, t1, is32); > > emit_insn(ctx, and, t1, dst, t1); > > - emit_zext_32(ctx, t1, is32); > > + if (BPF_CLASS(code) == BPF_JMP32) > > + emit_insn(ctx, bstrpickd, t1, t1, 31, 0); > > if (emit_cond_jmp(ctx, cond, t1, LOONGARCH_GPR_ZERO, jmp_offset) < 0) > > goto toofar; > > break; > > @@ -1468,6 +1506,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext > > return -EINVAL; > > } > > > > + if (is32) > > + emit_insn(ctx, bstrpickd, dst, dst, 31, 0); > > + > > return 0; > > > > toofar: > > diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h > > index 57a98f8a055b..2bd0dc5b911c 100644 > > --- a/arch/loongarch/net/bpf_jit.h > > +++ b/arch/loongarch/net/bpf_jit.h > > @@ -72,24 +72,6 @@ static inline int epilogue_offset(const struct jit_ctx *ctx) > > return (to - from); > > } > > > > -/* Zero-extend 32 bits into 64 bits */ > > -static inline void emit_zext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > > -{ > > - if (!is32) > > - return; > > - > > - emit_insn(ctx, lu32id, reg, 0); > > -} > > - > > -/* Signed-extend 32 bits into 64 bits */ > > -static inline void emit_sext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, bool is32) > > -{ > > - if (!is32) > > - return; > > - > > - emit_insn(ctx, addiw, reg, reg, 0); > > -} > > - > > /* Emit proper extension according to ABI requirements. > > * Note that it requires a value of size `size` already resides in register `reg`. > > */ > > -- > > 2.42.0 > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations 2026-10-16 8:10 [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang ` (2 preceding siblings ...) 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang @ 2026-10-22 7:43 ` Tiezhu Yang 2026-10-23 8:42 ` Huacai Chen 3 siblings, 1 reply; 15+ messages in thread From: Tiezhu Yang @ 2026-10-22 7:43 UTC (permalink / raw) To: Huacai Chen, Hengqi Chen; +Cc: loongarch, bpf, linux-kernel On 2026/10/16 下午4:10, Tiezhu Yang wrote: > v3: > -- Add a prerequisite patch to detect CPUCFG2_DIV32 feature. > -- Fix a hardware compatibility bug by checking cpu_has_div32 > before emitting native 32-bit div/mod instructions. > > This version is based on v7.3-rc7 and has been tested on Loongson-3A6000 > and Loongson-3A5000 physical machines. > > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > and Loongson-3A5000 (without DIV32), here are the test results: > > $ sudo modprobe test_bpf > $ dmesg -t | grep Summary > test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] > test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] > test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED > > For the BPF selftests test_verifier and test_progs, there are no any > regressions. > > Tiezhu Yang (3): > LoongArch: Add CPUCFG2 DIV32 feature detection > LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() > LoongArch: BPF: Optimize 32-bit operations using native instructions Hi Huacai, Hengqi, A gentle ping on the last 2 patches of this series since patch 1 has been applied. There are no any reports from bot+bpf-ci and sashiko-bot, and also "test_bpf" and "./test_progs -t jit_insns" tests passed. Just to clarify the testing status: the 3 failures mentioned in the previous test report are entirely pre-existing on the clean bpf-next tree and completely unrelated to this 32-bit optimization series. The core ALU32/JMP32 paths introduced here have been fully verified and passed without any regressions on both 3A6000 and 3A5000 physical machines. Do you have any further comments or objections to patches 2 and 3? If these look good to you, it would be great to have them merged to avoid rebasing and resending these patches after this merge window. Thanks, Tiezhu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations 2026-10-22 7:43 ` [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang @ 2026-10-23 8:42 ` Huacai Chen 0 siblings, 0 replies; 15+ messages in thread From: Huacai Chen @ 2026-10-23 8:42 UTC (permalink / raw) To: Tiezhu Yang; +Cc: Hengqi Chen, loongarch, bpf, linux-kernel On Thu, Oct 22, 2026 at 3:44 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote: > > On 2026/10/16 下午4:10, Tiezhu Yang wrote: > > v3: > > -- Add a prerequisite patch to detect CPUCFG2_DIV32 feature. > > -- Fix a hardware compatibility bug by checking cpu_has_div32 > > before emitting native 32-bit div/mod instructions. > > > > This version is based on v7.3-rc7 and has been tested on Loongson-3A6000 > > and Loongson-3A5000 physical machines. > > > > All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32) > > and Loongson-3A5000 (without DIV32), here are the test results: > > > > $ sudo modprobe test_bpf > > $ dmesg -t | grep Summary > > test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed] > > test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] > > test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED > > > > For the BPF selftests test_verifier and test_progs, there are no any > > regressions. > > > > Tiezhu Yang (3): > > LoongArch: Add CPUCFG2 DIV32 feature detection > > LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() > > LoongArch: BPF: Optimize 32-bit operations using native instructions > > Hi Huacai, Hengqi, > > A gentle ping on the last 2 patches of this series since patch 1 > has been applied. > > There are no any reports from bot+bpf-ci and sashiko-bot, > and also "test_bpf" and "./test_progs -t jit_insns" tests passed. > > Just to clarify the testing status: the 3 failures mentioned in > the previous test report are entirely pre-existing on the clean > bpf-next tree and completely unrelated to this 32-bit optimization > series. > > The core ALU32/JMP32 paths introduced here have been fully verified > and passed without any regressions on both 3A6000 and 3A5000 physical > machines. > > Do you have any further comments or objections to patches 2 and 3? > If these look good to you, it would be great to have them merged to > avoid rebasing and resending these patches after this merge window. I hope Hengqi can take a look before applying. Huacai > > Thanks, > Tiezhu > ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-10-28 6:55 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-10-16 8:10 [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 2026-10-16 8:10 ` [PATCH v3 1/3] LoongArch: Add CPUCFG2 DIV32 feature detection Tiezhu Yang 2026-10-20 10:48 ` Huacai Chen 2026-10-16 8:10 ` [PATCH v3 2/3] LoongArch: BPF: Clean up and decouple is32 parameter from move_imm() Tiezhu Yang 2026-10-27 1:36 ` Hengqi Chen 2026-10-16 8:10 ` [PATCH v3 3/3] LoongArch: BPF: Optimize 32-bit operations using native instructions Tiezhu Yang 2026-10-16 15:18 ` Alexei Starovoitov 2026-10-17 10:58 ` Tiezhu Yang 2026-10-27 2:02 ` Hengqi Chen 2026-10-27 4:10 ` Tiezhu Yang 2026-10-27 8:13 ` Hengqi Chen 2026-10-28 4:19 ` Huacai Chen 2026-10-28 6:55 ` Hengqi Chen 2026-10-22 7:43 ` [PATCH v3 0/3] LoongArch: BPF: Refine move_imm() and optimize 32-bit operations Tiezhu Yang 2026-10-23 8:42 ` Huacai Chen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®