Re: [PATCH 1/1] liveupdate: kho: calculate per-node scratch sizes before allocation

[Sourabh Jain <sourabhjain@linux.ibm.com>]

On 18/09/26 15:03, George Guo wrote:
> Hi Sourabh, Pratyush,
>
>> But in practice, this problem is only on CONFIG_NUMA=n and I don't think
>> in practice KHO or LUO is being used in non-NUMA systems. So while I
>> think it is worth fixing, I think we should also have a test where we
>> enable CONFIG_NUMA.
> Confirmed. My vmtest kernel has CONFIG_NUMA unset, and your analysis
> matches the data.
>
> I reran the vmtest with Sourabh's debug prints on both configurations:
> same kernel, same QEMU command, and without my this fix patch, so the
> default percentage policy is exercised. The only difference is
> CONFIG_NUMA.
>
> Without CONFIG_NUMA:
>
>    KHO: Before low and global scratch allocations
>    KHO: low size = 330185 KB
>    KHO: global size = 322 MB
>    KHO: Per node 0 = 672 MB
>    KHO: After low and global scratch allocations
>    KHO: low size = 330185 KB
>    KHO: global size = 322 MB
>    KHO: Per node 0 = 672 MB
>    KHO: Failed to reserve nid 0 scratch buffer
>    KHO: Failed to reserve scratch area, disabling kexec handover
>
> With CONFIG_NUMA=y:
>
>    KHO: Before low and global scratch allocations
>    KHO: low size = 330197 KB
>    KHO: global size = 322 MB
>    KHO: Per node 0 = 96 MB
>    KHO: After low and global scratch allocations
>    KHO: low size = 330197 KB
>    KHO: global size = 322 MB
>    KHO: Per node 0 = 96 MB
>    KHO: After per node allocation
>    KHO: low size = 428501 KB
>    KHO: global size = 418 MB
>    KHO: Per node 0 = 288 MB

[...]

>
> In the run without CONFIG_NUMA, the reserved-kern sum the sizing sees
> is 330185 KB, which is the 98.45 MiB baseline plus the 224 MiB lowmem
> scratch area: with memblock_get_region_node() hardcoded to return 0,
> the NUMA_NO_NODE lowmem area is counted as node 0's kernel reservation.
> Node 0 therefore requests 200% of (98.45 MiB + 224 MiB), rounded up to
> 32 MiB alignment: 672 MiB. That no longer fits next to the other areas
> in the 1 GiB guest, and KHO disables itself.
>
> In the run with CONFIG_NUMA=y, the real node ID excludes the
> NUMA_NO_NODE regions, so node 0 requests 96 MiB. The allocation
> succeeds and is visible in the sums printed afterwards (330197 KB ->
> 428501 KB), and the KHO selftest passes end to end ("KHO: found kexec
> handover data", restore succeeds).
>
> Sourabh, this also answers your question. Your PowerPC system runs
> CONFIG_NUMA=y, so the node filter excludes the lowmem and global areas
> and your numbers stay flat. Your experiment and mine are the two halves
> of the same mechanism.
>
>> I think on NUMA systems the problem is the other way round. The
>> calculation for the global scratch also counts per-node allocations.
>> So I think the proper fix for scratch sizing is what this patch does
>> and then a fixup for the global scratch calculation as well.
> Agreed. For v2 I plan to:
>
>    - Compute all scratch sizes (lowmem, global, per-node) before any
>      scratch area is allocated, per Mike's comment. The sizes are a
>      function of the pre-allocation state, so this seals both feedback
>      directions at once.
>    - State the !CONFIG_NUMA condition in the commit message. The feedback
>      described there is not unconditional, which is what triggered the
>      question.
>    - Add the LLM attribution Mike asked for.
>    - Include the CONFIG_NUMA=y vmtest result as coverage.
>    - Look at the global scratch calculation on NUMA systems as a
>      follow-up.

I have sent a fix to handle global scratch calculation on NUMA systems.
https://lore.kernel.org/all/20260922131217.698809-1-sourabhjain@linux.ibm.com/

Please review the changes and share your comments.

- Sourabh Jain