mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/2] hv: vmbus: prevent control-plane starvation and order-7 ring deadlocks under memory pressure
@ 2026-09-18  1:40 Emerson Busson
  2026-09-18  1:40 ` [PATCH 1/2] hv: vmbus: prevent control-plane starvation and balloon thrash " Emerson Busson
  2026-09-18  1:40 ` [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations " Emerson Busson
  0 siblings, 2 replies; 4+ messages in thread
From: Emerson Busson @ 2026-09-18  1:40 UTC (permalink / raw)
  To: linux-hyperv; +Cc: kys, haiyangz, wei.liu, decui, linux-kernel, emersonbusson

Under Microsoft Hyper-V and WSL2 virtual machine environments, heavy
memory pressure and memory tiering workloads (e.g. swap-heavy tasks,
containerized compilation, in-memory databases) frequently expose
guest kernels to starvation across two critical VMBus mechanisms:

1. Control-plane and heartbeat starvation:
   During aggressive direct reclaim, available pages drop below safety
   watermarks. Atomic allocations (GFP_ATOMIC) required by synthetic
   VMBus packets, virtual network switches (netvsc), and balloon
   management fail. When the guest fails to service host heartbeats,
   the Windows Host Compute System (HCS) watchdog presumes a guest
   hard lock, triggering abrupt virtual switch teardowns
   (Hyper-V-VmSwitch Event 102/291) and ungracefully restarting the VM
   prior to any Linux OOM killer intervention. Concurrently, host
   balloon driver requests exacerbate exhaustion.

2. High-order contiguous allocation failures:
   Dynamically instantiated VMBus sub-channels (e.g., synthetic SCSI,
   network, and vsock) invoke vmbus_alloc_ring(), requiring order-7
   (512 KiB) contiguous physical pages via alloc_pages(GFP_KERNEL |
   __GFP_ZERO, order). Severe buddy allocator fragmentation causes
   alloc_pages() to fail with -ENOMEM even when ample virtual memory
   exists. On WSL2, this directly manifests as userspace communication
   failures (e.g. "accept4 failed 110: Connection timed out" on vsock
   control planes).

This series resolves both issues:

- Patch 1/2 dynamically establishes a memory headroom floor for
  vm.min_free_kbytes during late_initcall (clamped between 64 MiB and
  512 MiB based on 3.125% of total guest RAM), safeguarding atomic
  allocations and VMBus heartbeats. It also defers balloon driver
  inflation when available memory drops below totalram_pages() / 32 to
  avoid competing with direct reclaim.

- Patch 2/2 introduces a fallback to vzalloc_node() / vzalloc() in
  vmbus_alloc_ring() when high-order physical allocation fails, adapts
  hv_ringbuffer_init() using vmalloc_to_page() for double-mapped ring
  wraparounds, and preserves Confidential VM (CoCo) guest memory
  re-encryption guarantees in vmbus_free_ring().

Verification & Testing:
Tested and validated against Microsoft WSL2 rolling-lts 6.18 kernel
under multi-tier memory pressure (13.66 GB/s transfer bandwidth across
ZRAM, VRAM, and SSD tiers), verifying zero kernel panics, zero HCS
disconnects, and flawless vsock channel open resilience.

Related issue discussions:
- https://github.com/microsoft/WSL/issues/41634
- https://github.com/microsoft/WSL/issues/40795


Emerson Busson (2):
  hv: vmbus: prevent control-plane starvation and balloon thrash under
    memory pressure
  hv: vmbus: add virtual memory fallback for ring buffer allocations
    under memory pressure

 drivers/hv/channel.c      | 46 ++++++++++++++++++++++++++++++++++-----
 drivers/hv/hv_balloon.c   | 12 ++++++++++
 drivers/hv/hv_common.c    | 32 +++++++++++++++++++++++++++
 drivers/hv/hyperv_vmbus.h |  2 +-
 drivers/hv/ring_buffer.c  | 19 +++++++++++-----
 include/linux/hyperv.h    |  2 ++
 6 files changed, 101 insertions(+), 12 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/2] hv: vmbus: prevent control-plane starvation and balloon thrash under memory pressure
  2026-09-18  1:40 [PATCH 0/2] hv: vmbus: prevent control-plane starvation and order-7 ring deadlocks under memory pressure Emerson Busson
@ 2026-09-18  1:40 ` Emerson Busson
  2026-09-18  1:40 ` [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations " Emerson Busson
  1 sibling, 0 replies; 4+ messages in thread
From: Emerson Busson @ 2026-09-18  1:40 UTC (permalink / raw)
  To: linux-hyperv; +Cc: kys, haiyangz, wei.liu, decui, linux-kernel, emersonbusson

Under Microsoft WSL2 and Hyper-V guest environments, workloads
generating heavy memory allocation and rapid page dirtiness can drive
the guest kernel into aggressive direct reclaim. When guest available
memory drops below critical thresholds (~400-512 MB), atomic page
allocations (GFP_ATOMIC) for synthetic VMBus packet rings, virtual
networking (netvsc), and balloon management fail.

Because the guest is unable to allocate atomic buffers to service host
heartbeats, the Windows Host Compute System (HCS) and Hyper-V watchdog
infer that the guest kernel has hard-locked. This triggers an abrupt
virtual network switch teardown (Hyper-V-VmSwitch Event 102/291) and
forces a VM restart before the guest Linux OOM killer can intervene.

Furthermore, under such memory pressure, the host balloon driver may
simultaneously request balloon inflation, exacerbating memory
starvation.

This patch addresses the issue in two architecture-neutral ways:
1. Auto-calibrates `vm.min_free_kbytes` during `late_initcall` via
   `ms_hyperv_init_memory_headroom` in `drivers/hv/hv_common.c` to a
   dynamic floor scaled to total guest RAM (clamped between 64 MiB and
   512 MiB). Running as `late_initcall` ensures `totalram_pages()` is
   fully cataloged. This guarantees that physical pages remain
   permanently reserved for kernel atomic allocations and VMBus
   communication.
2. Introduces a memory pressure check in `drivers/hv/hv_balloon.c` to
   defer balloon inflation whenever available memory drops below the
   critical threshold (`totalram_pages() / 32`), avoiding competing
   with kswapd.

Signed-off-by: Emerson Busson <emersonbusson@gmail.com>
---
 drivers/hv/hv_balloon.c | 12 ++++++++++++
 drivers/hv/hv_common.c  | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
index 2b4080e51..41cf97329 100644
--- a/drivers/hv/hv_balloon.c
+++ b/drivers/hv/hv_balloon.c
@@ -26,6 +26,7 @@
 #include <linux/percpu_counter.h>
 #include <linux/page_reporting.h>
 #include <linux/sizes.h>
+#include <linux/mm.h>
 
 #include <linux/hyperv.h>
 #include <hyperv/hvhdk.h>
@@ -1205,6 +1206,17 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm,
 	unsigned int i, j;
 	struct page *pg;
 
+	/*
+	 * Defer balloon inflation if guest is in critical reclaim zone.
+	 * Competing with kswapd under pressure starves VMBus channels.
+	 * Evaluates against 3.125% of total system RAM to avoid forbidden
+	 * out-of-core extern mutations while maintaining memory headroom.
+	 */
+	if (si_mem_available() < (totalram_pages() / 32)) {
+		pr_warn_ratelimited("hv_balloon: balloon inflation deferred; guest memory constrained\n");
+		return 0;
+	}
+
 	for (i = 0; i < num_pages / alloc_unit; i++) {
 		if (bl_resp->hdr.size + sizeof(union dm_mem_page_range) >
 			HV_HYP_PAGE_SIZE)
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index 71fd3ea4f..8a7ee00b6 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -28,6 +28,8 @@
 #include <linux/slab.h>
 #include <linux/dma-map-ops.h>
 #include <linux/set_memory.h>
+#include <linux/mm.h>
+#include <linux/init.h>
 #include <hyperv/hvhdk.h>
 #include <asm/mshyperv.h>
 
@@ -397,6 +399,36 @@ int __init hv_common_init(void)
 	return 0;
 }
 
+extern int min_free_kbytes;
+extern void setup_per_zone_wmarks(void);
+
+/*
+ * Ensure sufficient atomic page headroom for Hyper-V synthetic devices
+ * (vmbus, netvsc, balloon) to prevent host watchdog timeouts under pressure.
+ */
+static int __init ms_hyperv_init_memory_headroom(void)
+{
+	unsigned long total_ram_kb;
+	unsigned long min_headroom_kb;
+
+	if (!hv_is_hyperv_initialized())
+		return 0;
+
+	total_ram_kb = totalram_pages() * (PAGE_SIZE / 1024);
+	/* Scale headroom: 3.125% of RAM, clamped between 64MB and 512MB */
+	min_headroom_kb = clamp_t(unsigned long, total_ram_kb / 32, 64 * 1024, 512 * 1024);
+
+	if (min_free_kbytes < min_headroom_kb) {
+		pr_info("Hyper-V: Calibrating min_free_kbytes from %d kB to %lu kB for VMBus resilience\n",
+			min_free_kbytes, min_headroom_kb);
+		min_free_kbytes = min_headroom_kb;
+		setup_per_zone_wmarks();
+	}
+
+	return 0;
+}
+late_initcall(ms_hyperv_init_memory_headroom);
+
 void __init ms_hyperv_late_init(void)
 {
 	struct acpi_table_header *header;
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations under memory pressure
  2026-09-18  1:40 [PATCH 0/2] hv: vmbus: prevent control-plane starvation and order-7 ring deadlocks under memory pressure Emerson Busson
  2026-09-18  1:40 ` [PATCH 1/2] hv: vmbus: prevent control-plane starvation and balloon thrash " Emerson Busson
@ 2026-09-18  1:40 ` Emerson Busson
  2026-09-22  4:25   ` Michael Kelley
  1 sibling, 1 reply; 4+ messages in thread
From: Emerson Busson @ 2026-09-18  1:40 UTC (permalink / raw)
  To: linux-hyperv; +Cc: kys, haiyangz, wei.liu, decui, linux-kernel, emersonbusson

When VMBus sub-channels (such as synthetic SCSI, network, or vsock
channels) are dynamically opened during periods of sustained memory
load or memory tier eviction, `vmbus_alloc_ring()` attempts to
allocate physically contiguous memory using
`alloc_pages(GFP_KERNEL | __GFP_ZERO, order)`.

For standard rings (order-7, 512 KiB contiguous memory), high buddy
allocator fragmentation under memory pressure frequently causes
`alloc_pages()` to fail with -ENOMEM even when ample total virtual
memory is available. This manifests in userspace as connection
timeouts (e.g. `accept4 failed 110: Connection timed out` on WSL2 vsock
control planes).

This patch introduces a resilient fallback mechanism:
1. When `alloc_pages()` fails due to external fragmentation,
   `vmbus_alloc_ring()` falls back to `vzalloc_node()` (or `vzalloc()`)
   to satisfy the buffer allocation from virtually contiguous pages.
2. In `hv_ringbuffer_init()`, detects `is_vmalloc_addr(virt_addr)` and
   populates the `pages_wraparound` mapping array using
   `vmalloc_to_page()`.
3. In `vmbus_free_ring()`, tracks `ringbuffer_is_vmalloc` and calls
   `vfree()` safely, preserving Confidential VM (CoCo) memory
   re-encryption isolation checks prior to release.

Signed-off-by: Emerson Busson <emersonbusson@gmail.com>
---
 drivers/hv/channel.c      | 46 ++++++++++++++++++++++++++++++++++-----
 drivers/hv/hyperv_vmbus.h |  2 +-
 drivers/hv/ring_buffer.c  | 19 +++++++++++-----
 include/linux/hyperv.h    |  2 ++
 4 files changed, 57 insertions(+), 12 deletions(-)

diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index 162d6aeec..f0fb3dd8f 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -12,6 +12,7 @@
 #include <linux/sched.h>
 #include <linux/wait.h>
 #include <linux/mm.h>
+#include <linux/vmalloc.h>
 #include <linux/slab.h>
 #include <linux/module.h>
 #include <linux/hyperv.h>
@@ -153,13 +154,20 @@ void vmbus_free_ring(struct vmbus_channel *channel)
 	hv_ringbuffer_cleanup(&channel->outbound);
 	hv_ringbuffer_cleanup(&channel->inbound);
 
-	if (channel->ringbuffer_page) {
+	if (channel->ringbuffer_is_vmalloc && channel->ringbuffer_page_virt) {
+		/* In a CoCo VM leak the memory if it didn't get re-encrypted */
+		if (!channel->ringbuffer_gpadlhandle.decrypted)
+			vfree(channel->ringbuffer_page_virt);
+		channel->ringbuffer_page_virt = NULL;
+		channel->ringbuffer_is_vmalloc = false;
+	} else if (channel->ringbuffer_page) {
 		/* In a CoCo VM leak the memory if it didn't get re-encrypted */
 		if (!channel->ringbuffer_gpadlhandle.decrypted)
 			__free_pages(channel->ringbuffer_page,
 			     get_order(channel->ringbuffer_pagecount
 				       << PAGE_SHIFT));
 		channel->ringbuffer_page = NULL;
+		channel->ringbuffer_page_virt = NULL;
 	}
 }
 EXPORT_SYMBOL_GPL(vmbus_free_ring);
@@ -182,10 +190,26 @@ int vmbus_alloc_ring(struct vmbus_channel *newchannel,
 	if (!page)
 		page = alloc_pages(GFP_KERNEL|__GFP_ZERO, order);
 
-	if (!page)
-		return -ENOMEM;
+	if (!page) {
+		/* Fallback to virtual memory allocation under buddy fragmentation */
+		void *virt_addr = vzalloc_node(send_size + recv_size,
+					       cpu_to_node(newchannel->target_cpu));
+
+		if (!virt_addr)
+			virt_addr = vzalloc(send_size + recv_size);
+
+		if (!virt_addr)
+			return -ENOMEM;
+
+		newchannel->ringbuffer_page = NULL;
+		newchannel->ringbuffer_page_virt = virt_addr;
+		newchannel->ringbuffer_is_vmalloc = true;
+	} else {
+		newchannel->ringbuffer_page = page;
+		newchannel->ringbuffer_page_virt = page_address(page);
+		newchannel->ringbuffer_is_vmalloc = false;
+	}
 
-	newchannel->ringbuffer_page = page;
 	newchannel->ringbuffer_pagecount = (send_size + recv_size) >> PAGE_SHIFT;
 	newchannel->ringbuffer_send_offset = send_size >> PAGE_SHIFT;
 
@@ -639,6 +663,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	struct vmbus_channel_open_channel *open_msg;
 	struct vmbus_channel_msginfo *open_info = NULL;
 	struct page *page = newchannel->ringbuffer_page;
+	void *inbound_virt = NULL;
 	u32 send_pages, recv_pages;
 	unsigned long flags;
 	int err;
@@ -669,6 +694,8 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0;
 
 	err = __vmbus_establish_gpadl(newchannel, HV_GPADL_RING,
+				      newchannel->ringbuffer_page_virt ?
+				      newchannel->ringbuffer_page_virt :
 				      page_address(newchannel->ringbuffer_page),
 				      (send_pages + recv_pages) << PAGE_SHIFT,
 				      newchannel->ringbuffer_send_offset << PAGE_SHIFT,
@@ -677,11 +704,18 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 		goto error_clean_ring;
 
 	err = hv_ringbuffer_init(&newchannel->outbound,
-				 page, send_pages, 0);
+				 page, newchannel->ringbuffer_page_virt,
+				 send_pages, 0);
 	if (err)
 		goto error_free_gpadl;
 
-	err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
+	if (newchannel->ringbuffer_page_virt)
+		inbound_virt = newchannel->ringbuffer_page_virt +
+			       (send_pages << PAGE_SHIFT);
+
+	err = hv_ringbuffer_init(&newchannel->inbound,
+				 page ? &page[send_pages] : NULL,
+				 inbound_virt,
 				 recv_pages, newchannel->max_pkt_size);
 	if (err)
 		goto error_free_gpadl;
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 34943de7d..ec06c30d2 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -182,7 +182,7 @@ extern int hv_synic_cleanup(unsigned int cpu);
 void hv_ringbuffer_pre_init(struct vmbus_channel *channel);
 
 int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
-		       struct page *pages, u32 pagecnt, u32 max_pkt_size);
+		       struct page *pages, void *virt_addr, u32 pagecnt, u32 max_pkt_size);
 
 void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info);
 
diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
index 23ce1fb70..e6d4cf185 100644
--- a/drivers/hv/ring_buffer.c
+++ b/drivers/hv/ring_buffer.c
@@ -184,7 +184,7 @@ void hv_ringbuffer_pre_init(struct vmbus_channel *channel)
 
 /* Initialize the ring buffer. */
 int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
-		       struct page *pages, u32 page_cnt, u32 max_pkt_size)
+		       struct page *pages, void *virt_addr, u32 page_cnt, u32 max_pkt_size)
 {
 	struct page **pages_wraparound;
 	int i;
@@ -201,10 +201,19 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
 	if (!pages_wraparound)
 		return -ENOMEM;
 
-	pages_wraparound[0] = pages;
-	for (i = 0; i < 2 * (page_cnt - 1); i++)
-		pages_wraparound[i + 1] =
-			&pages[i % (page_cnt - 1) + 1];
+	if (virt_addr && is_vmalloc_addr(virt_addr)) {
+		pages_wraparound[0] = vmalloc_to_page(virt_addr);
+		for (i = 0; i < 2 * (page_cnt - 1); i++) {
+			void *curr_virt = virt_addr + ((i % (page_cnt - 1) + 1) << PAGE_SHIFT);
+
+			pages_wraparound[i + 1] = vmalloc_to_page(curr_virt);
+		}
+	} else {
+		pages_wraparound[0] = pages;
+		for (i = 0; i < 2 * (page_cnt - 1); i++)
+			pages_wraparound[i + 1] =
+				&pages[i % (page_cnt - 1) + 1];
+	}
 
 	ring_info->ring_buffer = (struct hv_ring_buffer *)
 		vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP,
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index a76f556f5..63203df4d 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -807,6 +807,8 @@ struct vmbus_channel {
 
 	/* Allocated memory for ring buffer */
 	struct page *ringbuffer_page;
+	void *ringbuffer_page_virt;
+	bool ringbuffer_is_vmalloc;
 	u32 ringbuffer_pagecount;
 	u32 ringbuffer_send_offset;
 	struct hv_ring_buffer_info outbound;	/* send to parent */
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* RE: [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations under memory pressure
  2026-09-18  1:40 ` [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations " Emerson Busson
@ 2026-09-22  4:25   ` Michael Kelley
  0 siblings, 0 replies; 4+ messages in thread
From: Michael Kelley @ 2026-09-22  4:25 UTC (permalink / raw)
  To: Emerson Busson, linux-hyperv, Kameron Carr
  Cc: kys, haiyangz, wei.liu, decui, linux-kernel

From: Emerson Busson <emersonbusson@gmail.com> Sent: Thursday, September 17, 2026 6:40 PM
> 
> When VMBus sub-channels (such as synthetic SCSI, network, or vsock
> channels) are dynamically opened during periods of sustained memory
> load or memory tier eviction, `vmbus_alloc_ring()` attempts to
> allocate physically contiguous memory using
> `alloc_pages(GFP_KERNEL | __GFP_ZERO, order)`.
> 
> For standard rings (order-7, 512 KiB contiguous memory), high buddy
> allocator fragmentation under memory pressure frequently causes
> `alloc_pages()` to fail with -ENOMEM even when ample total virtual
> memory is available. This manifests in userspace as connection
> timeouts (e.g. `accept4 failed 110: Connection timed out` on WSL2 vsock
> control planes).
> 
> This patch introduces a resilient fallback mechanism:
> 1. When `alloc_pages()` fails due to external fragmentation,
>    `vmbus_alloc_ring()` falls back to `vzalloc_node()` (or `vzalloc()`)
>    to satisfy the buffer allocation from virtually contiguous pages.
> 2. In `hv_ringbuffer_init()`, detects `is_vmalloc_addr(virt_addr)` and
>    populates the `pages_wraparound` mapping array using
>    `vmalloc_to_page()`.
> 3. In `vmbus_free_ring()`, tracks `ringbuffer_is_vmalloc` and calls
>    `vfree()` safely, preserving Confidential VM (CoCo) memory
>    re-encryption isolation checks prior to release.

The problem you describe is real, and there have been code changes
in the past to remove places where high-order kernel memory
allocations were needed to load and initialize a VMBus device or
a PCI pass-thru devices (such as the Mellanox NIC VF). But as you
noticed, the ring buffer allocations were never fixed.

Unfortunately, your solution doesn't work in all CoCo VMs. In
arm64 CCA VMs and in TDX VMs running without a paravisor,
the set_memory_decrypted() call in __vmbus_establish_gpadl()
doesn't work for vmalloc'ed memory. It only works for physically
contiguous memory, and various proposals to make it work
for vmalloc'ed memory have been stymied. As a result, Kameron
Carr recently had a patch set [1] accepted to solve the problem.
See the new function vmbus_alloc_buffer() and related.

Solving this high-order ring buffer allocation problem is a prime
opportunity to do a bit of refactoring:

1) Ring buffer allocation should *always* use Kameron's new
mechanism, not just as a fallback if normal alloc_pages() fails.
This would make things simpler, and even though Kameron's
mechanism has a little more overhead, I think that's OK as
ring buffer allocation/free is not a fast path.

2) The three data fields needed by Kameron's mechanism should
be grouped into a structure that can be passed around. Currently
the fields are separately added in the netvsc driver because that's
the only place that needed a GPADL for a buffer larger than
alloc_pages() could provide. But with broader usage for all
ring buffers, some encapsulation into a struct such as
struct vmbus_buffer would make sense.

3) We also have struct vmbus_gpadl. There's some logical
duplication between the fields Kameron needed and what's
in struct vmbus_gpadl. The stuff in struct vmbus_gpadl could
be folded into struct vmbus_buffer and the duplication
eliminated.

4) I have separately been working on a fix to making sure
that memory is leaked if the GPADL can't be removed, or
if the buffer memory can't be re-encrypted in a CoCo VM.
I've run into messy issues that can't be cleanly solved with
the current data structures. But having a struct vmbus_buffer
with a "leak" flag field would be a clean solution.
 
5) Kameron's new code required introducing a new GPADL
type HV_GPADL_BUFFER_DECRYPTED because "normal"
buffers are decrypted in __vmbus_establish_gpadl(). But
if vmbus_alloc_buffer() is always used, and it does the
decryption in a CoCo VM instead of __vmbus_establish_gpadl(),
then some of the HV_GPADL_* types can probably be
eliminated.

That's a quick summary of what I'm thinking. I haven't
coded any of this, so maybe there are some flaws that
I haven't immediately spotted, but I'm pretty sure this
could all be worked out in a way that's better and
simpler than what we have now, and that would
avoid ring buffer allocation failures due to memory
fragmentation.

Michael

[1] https://lore.kernel.org/linux-hyperv/20260811160447.2529876-1-kameroncarr@linux.microsoft.com/

> 
> Signed-off-by: Emerson Busson <emersonbusson@gmail.com>
> ---
>  drivers/hv/channel.c      | 46 ++++++++++++++++++++++++++++++++++-----
>  drivers/hv/hyperv_vmbus.h |  2 +-
>  drivers/hv/ring_buffer.c  | 19 +++++++++++-----
>  include/linux/hyperv.h    |  2 ++
>  4 files changed, 57 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
> index 162d6aeec..f0fb3dd8f 100644
> --- a/drivers/hv/channel.c
> +++ b/drivers/hv/channel.c
> @@ -12,6 +12,7 @@
>  #include <linux/sched.h>
>  #include <linux/wait.h>
>  #include <linux/mm.h>
> +#include <linux/vmalloc.h>
>  #include <linux/slab.h>
>  #include <linux/module.h>
>  #include <linux/hyperv.h>
> @@ -153,13 +154,20 @@ void vmbus_free_ring(struct vmbus_channel *channel)
>  	hv_ringbuffer_cleanup(&channel->outbound);
>  	hv_ringbuffer_cleanup(&channel->inbound);
> 
> -	if (channel->ringbuffer_page) {
> +	if (channel->ringbuffer_is_vmalloc && channel->ringbuffer_page_virt) {
> +		/* In a CoCo VM leak the memory if it didn't get re-encrypted */
> +		if (!channel->ringbuffer_gpadlhandle.decrypted)
> +			vfree(channel->ringbuffer_page_virt);
> +		channel->ringbuffer_page_virt = NULL;
> +		channel->ringbuffer_is_vmalloc = false;
> +	} else if (channel->ringbuffer_page) {
>  		/* In a CoCo VM leak the memory if it didn't get re-encrypted */
>  		if (!channel->ringbuffer_gpadlhandle.decrypted)
>  			__free_pages(channel->ringbuffer_page,
>  			     get_order(channel->ringbuffer_pagecount
>  				       << PAGE_SHIFT));
>  		channel->ringbuffer_page = NULL;
> +		channel->ringbuffer_page_virt = NULL;
>  	}
>  }
>  EXPORT_SYMBOL_GPL(vmbus_free_ring);
> @@ -182,10 +190,26 @@ int vmbus_alloc_ring(struct vmbus_channel *newchannel,
>  	if (!page)
>  		page = alloc_pages(GFP_KERNEL|__GFP_ZERO, order);
> 
> -	if (!page)
> -		return -ENOMEM;
> +	if (!page) {
> +		/* Fallback to virtual memory allocation under buddy fragmentation */
> +		void *virt_addr = vzalloc_node(send_size + recv_size,
> +					       cpu_to_node(newchannel->target_cpu));
> +
> +		if (!virt_addr)
> +			virt_addr = vzalloc(send_size + recv_size);
> +
> +		if (!virt_addr)
> +			return -ENOMEM;
> +
> +		newchannel->ringbuffer_page = NULL;
> +		newchannel->ringbuffer_page_virt = virt_addr;
> +		newchannel->ringbuffer_is_vmalloc = true;
> +	} else {
> +		newchannel->ringbuffer_page = page;
> +		newchannel->ringbuffer_page_virt = page_address(page);
> +		newchannel->ringbuffer_is_vmalloc = false;
> +	}
> 
> -	newchannel->ringbuffer_page = page;
>  	newchannel->ringbuffer_pagecount = (send_size + recv_size) >> PAGE_SHIFT;
>  	newchannel->ringbuffer_send_offset = send_size >> PAGE_SHIFT;
> 
> @@ -639,6 +663,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
>  	struct vmbus_channel_open_channel *open_msg;
>  	struct vmbus_channel_msginfo *open_info = NULL;
>  	struct page *page = newchannel->ringbuffer_page;
> +	void *inbound_virt = NULL;
>  	u32 send_pages, recv_pages;
>  	unsigned long flags;
>  	int err;
> @@ -669,6 +694,8 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
>  	newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0;
> 
>  	err = __vmbus_establish_gpadl(newchannel, HV_GPADL_RING,
> +				      newchannel->ringbuffer_page_virt ?
> +				      newchannel->ringbuffer_page_virt :
>  				      page_address(newchannel->ringbuffer_page),
>  				      (send_pages + recv_pages) << PAGE_SHIFT,
>  				      newchannel->ringbuffer_send_offset <<
> PAGE_SHIFT,
> @@ -677,11 +704,18 @@ static int __vmbus_open(struct vmbus_channel
> *newchannel,
>  		goto error_clean_ring;
> 
>  	err = hv_ringbuffer_init(&newchannel->outbound,
> -				 page, send_pages, 0);
> +				 page, newchannel->ringbuffer_page_virt,
> +				 send_pages, 0);
>  	if (err)
>  		goto error_free_gpadl;
> 
> -	err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
> +	if (newchannel->ringbuffer_page_virt)
> +		inbound_virt = newchannel->ringbuffer_page_virt +
> +			       (send_pages << PAGE_SHIFT);
> +
> +	err = hv_ringbuffer_init(&newchannel->inbound,
> +				 page ? &page[send_pages] : NULL,
> +				 inbound_virt,
>  				 recv_pages, newchannel->max_pkt_size);
>  	if (err)
>  		goto error_free_gpadl;
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 34943de7d..ec06c30d2 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -182,7 +182,7 @@ extern int hv_synic_cleanup(unsigned int cpu);
>  void hv_ringbuffer_pre_init(struct vmbus_channel *channel);
> 
>  int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
> -		       struct page *pages, u32 pagecnt, u32 max_pkt_size);
> +		       struct page *pages, void *virt_addr, u32 pagecnt, u32
> max_pkt_size);
> 
>  void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info);
> 
> diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
> index 23ce1fb70..e6d4cf185 100644
> --- a/drivers/hv/ring_buffer.c
> +++ b/drivers/hv/ring_buffer.c
> @@ -184,7 +184,7 @@ void hv_ringbuffer_pre_init(struct vmbus_channel *channel)
> 
>  /* Initialize the ring buffer. */
>  int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
> -		       struct page *pages, u32 page_cnt, u32 max_pkt_size)
> +		       struct page *pages, void *virt_addr, u32 page_cnt, u32
> max_pkt_size)
>  {
>  	struct page **pages_wraparound;
>  	int i;
> @@ -201,10 +201,19 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
>  	if (!pages_wraparound)
>  		return -ENOMEM;
> 
> -	pages_wraparound[0] = pages;
> -	for (i = 0; i < 2 * (page_cnt - 1); i++)
> -		pages_wraparound[i + 1] =
> -			&pages[i % (page_cnt - 1) + 1];
> +	if (virt_addr && is_vmalloc_addr(virt_addr)) {
> +		pages_wraparound[0] = vmalloc_to_page(virt_addr);
> +		for (i = 0; i < 2 * (page_cnt - 1); i++) {
> +			void *curr_virt = virt_addr + ((i % (page_cnt - 1) + 1) <<
> PAGE_SHIFT);
> +
> +			pages_wraparound[i + 1] = vmalloc_to_page(curr_virt);
> +		}
> +	} else {
> +		pages_wraparound[0] = pages;
> +		for (i = 0; i < 2 * (page_cnt - 1); i++)
> +			pages_wraparound[i + 1] =
> +				&pages[i % (page_cnt - 1) + 1];
> +	}
> 
>  	ring_info->ring_buffer = (struct hv_ring_buffer *)
>  		vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP,
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index a76f556f5..63203df4d 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -807,6 +807,8 @@ struct vmbus_channel {
> 
>  	/* Allocated memory for ring buffer */
>  	struct page *ringbuffer_page;
> +	void *ringbuffer_page_virt;
> +	bool ringbuffer_is_vmalloc;
>  	u32 ringbuffer_pagecount;
>  	u32 ringbuffer_send_offset;
>  	struct hv_ring_buffer_info outbound;	/* send to parent */
> --
> 2.43.0
> 


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-09-22  4:25 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-18  1:40 [PATCH 0/2] hv: vmbus: prevent control-plane starvation and order-7 ring deadlocks under memory pressure Emerson Busson
2026-09-18  1:40 ` [PATCH 1/2] hv: vmbus: prevent control-plane starvation and balloon thrash " Emerson Busson
2026-09-18  1:40 ` [PATCH 2/2] hv: vmbus: add virtual memory fallback for ring buffer allocations " Emerson Busson
2026-09-22  4:25   ` Michael Kelley

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®