[PATCH 1/3] KVM: SVM: Re-queue events that were never injected

["Pratik R. Sampat" <prsampat@amd.com>]

When injecting an event into the guest via the event_inj field, a
non-zero event_inj value on #VMEXIT means that the hardware was not able
to inject the event into the guest.  This used to only occur for the
VMEXIT_INVALID intercept code, which was a fatal error and resulted in
the guest being torn down.

Enhanced SMT Protection (ESMTP) invalidates that assumption.  When ESMTP
is enabled, VMRUN doesn't enter guest mode immediately; it stalls at a
synchronization point until every sibling thread is either idle or has
executed VMRUN for a legal sibling vCPU.  If an ESMTP timeout / illegal
sibling exit / interrupt arrives while VMRUN is stalled, VMRUN can now
terminate with the corresponding #VMEXIT intercept code, without
entering guest mode, and thus without injecting the event.

For example, on a 2-way SMT core running vCPU0 on thread 0 and vCPU1 on
thread 1:
  Thread 0 (vCPU0)                         Thread 1 (vCPU1)
  ----------------                         ----------------
  Interrupt A injected to the guest             in host
  VMRUN                                           |
    |                                             |
    v                                             |
  Stall waiting for sibling                       |
    |                                             |
    |  host INTR arrives on thread 0              |
    |  while it waits                             |
    |                                             |
    v                                             |
  #VMEXIT                                         v
  Interrupt B injected to the guest              idle
  VMRUN

In this case injecting interrupt B clobbers the last event. Interrupt A
is never delivered and lost forvever.

Therefore, in preparation for ESMTP support, accommodate for potentially
lost interrupts by detecting an undelivered injected event and
re-queuing it so as not to lose the event.

svm_cancel_injection() already recovers a staged event this way and
becomes redundant, so fold it in.

Signed-off-by: Pratik R. Sampat <prsampat@amd.com>
---
 arch/x86/kvm/svm/svm.c | 33 +++++++++++++++++++++++++++++----
 1 file changed, 29 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 7d59d301e1e5..5d15c706e43b 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4340,6 +4340,7 @@ static void svm_complete_interrupts(struct kvm_vcpu *vcpu)
 	struct vcpu_svm *svm = to_svm(vcpu);
 	u8 vector;
 	int type;
+	struct vmcb_control_area *control = &svm->vmcb->control;
 	u32 exitintinfo = svm->vmcb->control.exit_int_info;
 	bool nmi_l1_to_l2 = svm->nmi_l1_to_l2;
 	bool soft_int_injected = svm->soft_int_injected;
@@ -4347,6 +4348,27 @@ static void svm_complete_interrupts(struct kvm_vcpu *vcpu)
 	svm->nmi_l1_to_l2 = false;
 	svm->soft_int_injected = false;
 
+	/*
+	 * Hardware clears EVENTINJ field when it injects an event.
+	 * A non-empty EVENTINJ on #VMEXIT means the vCPU never entered guest
+	 * mode, and thus that the event was never delivered.  Migrate the event
+	 * to EXITINTINFO so that it's requeued instead of being dropped.
+	 *
+	 * An undelivered event doesn't imply a fatal VMEXIT_INVALID. With
+	 * Enhanced SMT Protection, VMRUN may exit with an ordinary #VMEXIT
+	 * without having injected that event into the guest.
+	 *
+	 * Clobbering EXITINTINFO is safe precisely because the vCPU never
+	 * entered guest mode.
+	 */
+	if (control->event_inj) {
+		control->exit_int_info = control->event_inj;
+		control->exit_int_info_err = control->event_inj_err;
+		control->event_inj = 0;
+
+		exitintinfo = control->exit_int_info;
+	}
+
 	/*
 	 * If we've made progress since setting awaiting_iret_completion, we've
 	 * executed an IRET and can allow NMI injection.
@@ -4410,11 +4432,14 @@ static void svm_complete_interrupts(struct kvm_vcpu *vcpu)
 static void svm_cancel_injection(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
-	struct vmcb_control_area *control = &svm->vmcb->control;
 
-	control->exit_int_info = control->event_inj;
-	control->exit_int_info_err = control->event_inj_err;
-	control->event_inj = 0;
+	/*
+	 * EXITINTINFO is stale as it holds the event from the previous #VMEXIT
+	 * (or from the last time the current VMCB was run).  Invalidate it so
+	 * that svm_complete_interrupts() requeues if and only if KVM staged an
+	 * event in EVENTINJ.
+	 */
+	svm->vmcb->control.exit_int_info = 0;
 	svm_complete_interrupts(vcpu);
 }
 
-- 
2.43.0