mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Boqun Feng <boqun@kernel.org>
To: Thomas Gleixner <tglx@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	linux-kernel@vger.kernel.org, linux-tip-commits@vger.kernel.org,
	x86@kernel.org
Subject: Re: [PATCH] locking: Revert switching guards to _irq_{disable,enable}()
Date: Thu, 27 Aug 2026 09:52:54 -0700	[thread overview]
Message-ID: <apBrZuxU8bmCgG8h@tardis.local> (raw)
In-Reply-To: <87jypbfu1t.ffs@fw13>

On Thu, Aug 27, 2026 at 05:43:26PM +0200, Thomas Gleixner wrote:
> On Thu, Aug 27 2026 at 06:14, Boqun Feng wrote:
> > On Thu, Aug 27, 2026 at 10:30:50AM +0200, Thomas Gleixner wrote:
> >> The main problem is that you cover only half of it and there are
> >> completely correct cases where this simply blows up in your face:
> >> 
> >>   local_irq_disable();                          // does not affect CNT
> >>   ....
> >>   guard(raw_spinlock)(&l1);                     // does not affect CNT
> >>      foo()
> >>        guard(raw_spinlock_irqsave)(&l2);        // observes CNT = 0
> >> 
> >> so the unlocking of &l2 will enable interrupts prematurely.
> >> 
> >
> > If we are talking the switch in this patch, then no, the unlocking
> > of &l2 will NOT enable interrupts prematurely. The above code expands as
> > the following (using pseudo code to describe how
> > raw_spin_lock_irq_disable(), raw_spin_lock_irq_enable(),
> > local_interrupt_disable(), and local_interrupt_enable() work)
> >
> >    local_irq_disable();                          // does not affect CNT
> >    ....
> >    guard(raw_spinlock)(&l1);                     // does not affect CNT
> >       foo()
> >         guard(raw_spinlock_irqsave)(&l2): 
> >           raw_spin_lock_irq_disable():
> >             local_interrupt_disable():
> > 	      CNT++;
> >               this_cpu(state) = local_irq_save(); // record the current state
> >             raw_spin_lock(&l2);
> > 	  ...
> >           raw_spin_lock_irq_enable():
> >             raw_spin_unlock(&l2);
> >             local_interrupt_enable():
> >               CNT--;
> >               if (CNT == 0)
> >                 local_irq_restore(this_cpu(state)); // recover the previous state
> >
> > So local_interrupt_disable() and local_interrupt_enable() only recover
> > to the previous state, as a result it'll not enable interrupt
> > prematurely here. In other words, the following code works:
> >
> >     local_irq_disable();
> >     local_interrupt_disable();
> >     local_interrupt_enable();  // interrupt is not re-enabled here
> >                                // similar to how preempt_disable() does
> > 			       // in a nested preemption disable
> > 			       // critical section.
> >     local_irq_enable();
> 
> Fair enough. I misread that part.
> 
> But my main observation that the counter is inconsistent still stands
> and I think that's a fundamental flaw because there is no way that code
> can rely on that counter until everything has been converted over and
> the interrupt/exception/nmi/syscall entry/exit code has been fixed up.
> 
> Just let me look at local_interrupt_disable() and __irq_exit_rcu()
> again.
> 
> local_interrupt_disable()
>    new_count = hardirq_disable_enter();
> 
>    /* Interrupts can happen here, but it's OK, see __irq_exit_rcu(). */
> 
>    if ((new_count & HARDIRQ_DISABLE_MASK) == HARDIRQ_DISABLE_OFFSET)
>       _local_interrupt_disable();
> 
> This is absolutely not ok. Why?
> 
> The counter is incremented _before_ interrupts are actually
> disabled. Now in __irq_exit_rcu():
> 
>         if (!in_interrupt() && !hardirq_disable_count() &&
>             local_softirq_pending()) {
> 
> which prevents soft interrupt handling in a completely legitimate
> situation. As a consequence _nothing_ will handle the pending soft
> interrupt until:
> 
>      - an interrupt coming in which observes consistent state
> 
>      - a local_bh_enable() processes them
> 
> That explains that recently quite a few more spurious 'local soft irq
> pending' printk's have been observed by people as there is no guarantee
> that either one of those events happens _before_ a CPU reaches
> idle. Even in the non-idle case deferring this to the next 'by chance'
> handling is fundamentally broken. This needs to be removed ASAP.
> 
> But coming back to the problem underneath. The ordering in
> local_interrupt_disable() is simply wrong. You need to disable first and
> then update the counter. Reverse order for enable() obviously update
> counter and enable, which you got right.
> 

Noted, the reason that I used the current order is to optimize
local_interrupt_disable() from re-disabling interrupt every time:

	https://lore.kernel.org/rust-for-linux/87a5eu7gvw.ffs@tglx/

but looks like we cannot do it without the fixups you mention below.
For now I will reverse the order and remove the additional checking in
softirq to fix the softirq pending issue.

Regards,
Boqun

> And to make stuff work correctly you need the fixups I pointed out in my
> previous reply to the various entry/exit functions. It's exactly the
> same problem as we handle in interrupt/exception/nmi entry/exit code
> vs. RCU, lockdep, tracing etc.
> 
> So if disable() does:
> 
>    if (!count)
>    	arch_local_irq_disable();
>    count++;
> 
> then an interrupt hitting before arch_local_irq_disable() will always
> observe the correct state. After that it can't be delivered.
> 
[...]

  reply	other threads:[~2026-08-27 16:52 UTC|newest]

Thread overview: 110+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04 16:14 [PATCH v4 00/17] Refcounted interrupt disable and SpinLockIrq for Rust Boqun Feng
2026-08-04 16:14 ` [PATCH v4 01/17] preempt: Track NMI nesting to separate per-CPU counter Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Joel Fernandes
2026-08-04 16:14 ` [PATCH v4 02/17] preempt: Introduce HARDIRQ_DISABLE_BITS Boqun Feng
2026-08-05  6:31   ` Peter Zijlstra
2026-08-05  6:59     ` Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 03/17] preempt: Introduce __preempt_count_{sub,add}_return() Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 04/17] openrisc: Include <linux/cpumask.h> in smp.h Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Lyude Paul
2026-08-04 16:14 ` [PATCH v4 05/17] irq & spin_lock: Add counted interrupt disabling/enabling Boqun Feng
2026-08-04 18:20   ` Boqun Feng
2026-08-04 18:26   ` [PATCH v4.1 " Boqun Feng
2026-08-08 20:48     ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57     ` [tip: locking/core] irq,spin_lock: " tip-bot2 for Boqun Feng
2026-08-04 20:51   ` [PATCH v4 05/17] irq & spin_lock: " Shrikanth Hegde
2026-08-04 21:08     ` Boqun Feng
2026-08-05  6:36       ` Peter Zijlstra
2026-08-05  7:07         ` Boqun Feng
2026-08-05  7:09           ` Shrikanth Hegde
2026-08-05  7:19             ` Boqun Feng
2026-08-05 13:53               ` Boqun Feng
2026-08-05 14:10                 ` Shrikanth Hegde
2026-08-05 14:20                   ` Boqun Feng
2026-08-05 14:56                     ` Shrikanth Hegde
2026-08-05 15:11                       ` Boqun Feng
2026-08-05 16:53                         ` Shrikanth Hegde
2026-08-05 17:38                           ` Boqun Feng
2026-08-05 18:07                       ` Boqun Feng
2026-08-04 16:14 ` [PATCH v4 06/17] irq: Add KUnit test for refcounted interrupt enable/disable Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Lyude Paul
2026-08-10  8:57   ` tip-bot2 for Lyude Paul
2026-08-04 16:14 ` [PATCH v4 07/17] locking: Switch to _irq_{disable,enable}() variants in cleanup guards Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57   ` tip-bot2 for Boqun Feng
2026-08-24 10:47     ` Peter Zijlstra
2026-08-24 10:55       ` [PATCH] locking: Revert switching guards to _irq_{disable,enable}() Peter Zijlstra
2026-08-24 11:01         ` [tip: locking/urgent] " tip-bot2 for Peter Zijlstra
2026-08-25  1:33         ` [PATCH] " Boqun Feng
2026-08-25 22:59           ` Thomas Gleixner
2026-08-25 23:28             ` Boqun Feng
2026-08-25 23:48               ` Boqun Feng
2026-08-26  1:33                 ` Boqun Feng
2026-08-27  8:30               ` Thomas Gleixner
2026-08-27 13:14                 ` Boqun Feng
2026-08-27 15:43                   ` Thomas Gleixner
2026-08-27 16:52                     ` Boqun Feng [this message]
2026-08-27 18:15                       ` Thomas Gleixner
2026-08-27 19:41                         ` Boqun Feng
2026-08-27 22:52                           ` Thomas Gleixner
2026-08-28  1:56                             ` Boqun Feng
2026-08-28  6:42                             ` Peter Zijlstra
2026-08-28 23:11                             ` Thomas Gleixner
2026-08-29  0:45                               ` Boqun Feng
2026-08-29 20:44                                 ` Thomas Gleixner
2026-08-29 20:53                                   ` Boqun Feng
2026-08-29  8:05                               ` Peter Zijlstra
2026-08-29 19:52                                 ` Thomas Gleixner
2026-08-29 23:37                               ` Boqun Feng
2026-08-30 15:18                                 ` Boqun Feng
2026-08-30 19:57                                   ` Thomas Gleixner
2026-08-30 21:23                                     ` Boqun Feng
2026-08-31 10:02                                       ` Thomas Gleixner
2026-08-31 12:41                                         ` Boqun Feng
2026-09-01 13:43                                           ` Thomas Gleixner
2026-09-01 15:13                                             ` Boqun Feng
2026-09-04 13:19                                               ` Thomas Gleixner
2026-09-04 13:14                                             ` [PATCH] irq: Move local_irq_enable/disable() into Rust, Thomas Gleixner
2026-09-04 13:26                                               ` [PATCH] irq: Move local_irq_enable/disable() into Rust Thomas Gleixner
2026-09-04 15:25                                                 ` Boqun Feng
2026-09-04 21:20                                                   ` Thomas Gleixner
2026-08-30 21:42                                     ` [PATCH] locking: Revert switching guards to _irq_{disable,enable}() Boqun Feng
2026-08-31  9:56                                       ` Thomas Gleixner
2026-08-30 20:01                                 ` Thomas Gleixner
2026-08-27 20:29                 ` Thomas Gleixner
2026-08-27 21:33                   ` Boqun Feng
2026-08-28  6:55                     ` Peter Zijlstra
2026-08-28  8:22                     ` David Laight
2026-08-28 21:26                       ` Boqun Feng
2026-08-04 16:14 ` [PATCH v4 08/17] sched: Remove the unused preempt_offset parameter of __cant_sleep() Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57   ` tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 09/17] sched: Avoid signed comparison of preempt_count() in __cant_migrate() Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57   ` tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 10/17] preempt: Introduce HAS_SEPARATE_PREEMPT_RESCHED_BITS Boqun Feng
2026-08-04 20:11   ` Shrikanth Hegde
2026-08-05  6:54     ` Boqun Feng
2026-08-05  7:15       ` Shrikanth Hegde
2026-08-05  7:27         ` Boqun Feng
2026-08-06  0:58       ` Boqun Feng
2026-08-04 21:09   ` Shrikanth Hegde
2026-08-04 23:14     ` Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57   ` tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 11/17] arm64: sched/preempt: Enable HAS_SEPARATE_PREEMPT_RESCHED_BITS Boqun Feng
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Boqun Feng
2026-08-10  8:57   ` tip-bot2 for Boqun Feng
2026-08-04 16:14 ` [PATCH v4 12/17] s390/preempt: " Boqun Feng
2026-08-04 20:27   ` Shrikanth Hegde
2026-08-05  9:42     ` Peter Zijlstra
2026-08-05 12:37       ` Shrikanth Hegde
2026-08-08 20:48   ` [tip: locking/core] " tip-bot2 for Heiko Carstens
2026-08-10  8:57   ` tip-bot2 for Heiko Carstens
2026-08-04 16:14 ` [PATCH v4 13/17] rust: Introduce interrupt module Boqun Feng
2026-08-04 16:14 ` [PATCH v4 14/17] rust: helper: Add spin_{un,}lock_irq_{enable,disable}() helpers Boqun Feng
2026-08-04 16:14 ` [PATCH v4 15/17] rust: sync: Use super::* in spinlock.rs Boqun Feng
2026-08-04 16:14 ` [PATCH v4 16/17] rust: sync: Add SpinLockIrq Boqun Feng
2026-08-04 16:14 ` [PATCH v4 17/17] rust: sync: Introduce SpinLockIrq::lock_with() and friends Boqun Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=apBrZuxU8bmCgG8h@tardis.local \
    --to=boqun@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®