Re: [PATCH RFC 0/2] sched: Introduce idle SMT priority for asymmetric capacity systems

[Mete Durlu <meted@linux.ibm.com>]

[..snip..]

>> Andrea does this resemble what you were describing;
>>
>> 1-) (steal time == 0)
>> ====================================================================
>> cpu_preffered_mask = active_cpus
>> cpu_soft_affinity = 0
>>
>> All cpus are available for scheduler to pick and use.
>> Idle cores are preferred in case of asymmetric capacities.
>>
>> 2-) (steal time > 0) && (steal time <= initial_threshold*)
>> ====================================================================
>> cpu_preffered_mask = active_cpus
>> cpu_soft_affinity = entitled cpus
>>
>> In this case entitled cores get fully filled (SMT siblings first)
>> before spilling load onto the rest of the cpus on preferred mask.
>> *Initial threshold is a value I just devised. Ideally arch specific
>> and dynamically configurable, setting it to just 0 should also
>> be an option.
>>
>> 3-) (steal time > initial_threshold)
>> ====================================================================
>> cpu_preffered_mask = entitled cpus + steal governor's modifications
>> cpu_soft_affinity = entitled cpus
>>
>> Carrying the logic from case 2 but shrinking the preferred mask
>> according to the increased steal time value.
>>
>> Shrikanth/Andrea,
>> does what I describe above make sense and would it make a reasonable
>> addition to the vCPU backoff mechanism?
>>
> 
> I am confused. Are you talking about
> 1. picking CPU to run idle load balancing?
> Or
> 2. actual task placement decisions?

I was talking about actual task placement decisions. I didn't really
consider where idle load balancing should take place tbh. On s390
even if it takes place on a non-preferred CPU, there shouldn't be
too much of a penalty. Is there a specific reason why you are asking
for idle load balancing case? I might be missing some conceptual depth
here.

[..snip..]

> If it is 2, I am not sure. getting soft affinity is kind of tricky with 
> load balance.
> It puts maximum effort in spreading the tasks. Since you have steal 
> time, there is already
> enough tasks to create physical CPU contention.

I am not fully familiar with vCPU backoff yet, but couldn't the
soft affinity be implemented in form of another cpu mask. We can
traverse the new mask to make decisions there first and after its
resources are depleted there, we can then move onto cpu_preferred_mask?

> PS: Steal governor provides a tunable named low_threshold, which means 
> any steal value
> below is practically 0. So there is no soft affinity there.

Right, my initial instinct was to have an initial threshold like
the low_threshold steal governor already has. I am not sure if it
is a good idea to just use that with a higher default value but
I need to think about this a bit more.

What do you think, does it worth exploring?

Thanks!
-Mete