Re: [RFC PATCH] sched/fair: Introduce SIS_PAIR to wakeup task on local idle core first

[Mike Galbraith <efault@gmx.de>]

On Mon, 2023-05-22 at 12:10 +0800, Chen Yu wrote:
>
> Meanwhile, I looked back at Yicong's proposal on waking up task
> on local cluster first. It did show some improvement on Jacobsville,
> I guess that could also be a chance to reduce C2C latency.

Something else to consider is that communication data comes in many
size chunks and volumes, and cache footprints can easily be entirely
local constructs unrelated to any transferred data.

At one extreme of the huge spectrum of possibilities, a couple less
than brilliant tasks playing high speed ping-pong can bounce all over a
box with zero consequences, but for a pair more akin to say Einstein
and Bohr pondering chalkboards full of mind bending math and meeting
occasionally at the water cooler to exchange snarky remarks, needlessly
bouncing them about forces them to repopulate chalkboards, and C2C
traffic you try to avoid via bounce you generate via bounce.

Try as you may, methinks you're unlikely to succeed at avoiding C2C in
a box where roughly half of all paths are C2C.  What tasks have in
their pockets and what they'll do with a CPU at any point in time is
unknown and unknowable by the scheduler, dooming pinpoint placement
accuracy as a goal.

	-Mike