[PATCH 0/2] crypto: caam - Fix rsa-caam unmap and DMA alignment

[PATCH 0/2] crypto: caam - Fix rsa-caam unmap and DMA alignment

[Changwei Zou]

This series fixes two issues in the rsa-caam driver.

The first patch fixes incorrect unmap operations in akcipher_do_one_req()
and akcipher_enqueue_req(), where rsa_pub_unmap() was called regardless
of the key type.

The second patch fixes an intermittent -EKEYREJECTED error observed on
i.MX8 when loading signed kernel modules. The root cause is that the
rsa-caam driver DMA-maps the destination buffer directly without checking
cacheline alignment. On non-coherent DMA systems this can corrupt
adjacent memory. A bounce buffer is introduced for destination buffers
that are not cacheline-aligned.

Changwei Zou (2):
  crypto: caam - Fix wrong unmap operations
  crypto: caam - Use bounce buffer for unaligned RSA destination buffers

 drivers/crypto/caam/caampkc.c | 120 ++++++++++++++++++++++++++++++----
 drivers/crypto/caam/caampkc.h |   6 ++
 2 files changed, 113 insertions(+), 13 deletions(-)

-- 
2.43.0

[PATCH 1/2] crypto: caam - Fix wrong unmap operations

[Changwei Zou]

Both akcipher_do_one_req() and akcipher_enqueue_req() call rsa_pub_unmap()
regardless of the key type. priv_form only takes values FORM1/FORM2/FORM3
with no distinct public key enumerator.

Use key->d to distinguish public from private key operations, then
dispatch to the correct unmap function based on key->priv_form.

caam_rsa_set_priv_key_form() implicitly relies on zero-initialization
for priv_form. Set priv_form = FORM1 explicitly at the head of the function
for clarity and robustness.

Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
Assisted-by: OpenCode:claude-sonnet-4.6
---
 drivers/crypto/caam/caampkc.c | 43 +++++++++++++++++++++++++----------
 1 file changed, 31 insertions(+), 12 deletions(-)

diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
index cb001aa1de66..840271840cce 100644
--- a/drivers/crypto/caam/caampkc.c
+++ b/drivers/crypto/caam/caampkc.c
@@ -379,6 +379,7 @@ static int akcipher_do_one_req(struct crypto_engine *engine, void *areq)
 	struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
 	struct caam_rsa_req_ctx *req_ctx = akcipher_request_ctx(req);
 	struct caam_rsa_ctx *ctx = akcipher_tfm_ctx_dma(tfm);
+	struct caam_rsa_key *key = &ctx->key;
 	struct device *jrdev = ctx->dev;
 	u32 *desc = req_ctx->edesc->hw_desc;
 	int ret;
@@ -391,7 +392,21 @@ static int akcipher_do_one_req(struct crypto_engine *engine, void *areq)
 		return ret;
 
 	if (ret != -EINPROGRESS) {
-		rsa_pub_unmap(jrdev, req_ctx->edesc, req);
+		if (key->d) {
+			switch (key->priv_form) {
+			case FORM1:
+				rsa_priv_f1_unmap(jrdev, req_ctx->edesc, req);
+				break;
+			case FORM2:
+				rsa_priv_f2_unmap(jrdev, req_ctx->edesc, req);
+				break;
+			case FORM3:
+				rsa_priv_f3_unmap(jrdev, req_ctx->edesc, req);
+				break;
+			}
+		} else {
+			rsa_pub_unmap(jrdev, req_ctx->edesc, req);
+		}
 		rsa_io_unmap(jrdev, req_ctx->edesc, req);
 		kfree(req_ctx->edesc);
 	} else {
@@ -691,17 +706,19 @@ static int akcipher_enqueue_req(struct device *jrdev,
 		ret = caam_jr_enqueue(jrdev, desc, cbk, req);
 
 	if ((ret != -EINPROGRESS) && (ret != -EBUSY)) {
-		switch (key->priv_form) {
-		case FORM1:
-			rsa_priv_f1_unmap(jrdev, edesc, req);
-			break;
-		case FORM2:
-			rsa_priv_f2_unmap(jrdev, edesc, req);
-			break;
-		case FORM3:
-			rsa_priv_f3_unmap(jrdev, edesc, req);
-			break;
-		default:
+		if (key->d) {
+			switch (key->priv_form) {
+			case FORM1:
+				rsa_priv_f1_unmap(jrdev, edesc, req);
+				break;
+			case FORM2:
+				rsa_priv_f2_unmap(jrdev, edesc, req);
+				break;
+			case FORM3:
+				rsa_priv_f3_unmap(jrdev, edesc, req);
+				break;
+			}
+		} else {
 			rsa_pub_unmap(jrdev, edesc, req);
 		}
 		rsa_io_unmap(jrdev, edesc, req);
@@ -992,6 +1009,8 @@ static int caam_rsa_set_priv_key_form(struct caam_rsa_ctx *ctx,
 	size_t q_sz = raw_key->q_sz;
 	unsigned aligned_size;
 
+	rsa_key->priv_form = FORM1;
+
 	rsa_key->p = caam_read_raw_data(raw_key->p, &p_sz);
 	if (!rsa_key->p)
 		return -ENOMEM;
-- 
2.43.0

RE: [EXT] [PATCH 1/2] crypto: caam - Fix wrong unmap operations

[Sahil Malhotra (OSS)]

Hi Changwei,

Honestly, I'd prefer an explicit request-type marker over inferring it from
key->d, but every cleaner option (e.g. adding FORM_PUBLIC=0 and switching on
priv_form everywhere) ripples across the driver and isn't worth it for a
targeted. So key->d check is ok from my side.
Please also add Fixes: bf53795025a2 ("crypto: caam - add crypto_engine support for RSA algorithms") in this commit.

Reviewed-by: Sahil Malhotra <sahil.malhotra@nxp.com>

Regards,
Sahil Malhotra


NXP Confidential
> -----Original Message-----
> From: Changwei Zou <changwei.zou@canonical.com>
> Sent: 21 August 2026 10:26
> To: Horia Geanta <horia.geanta@nxp.com>; Pankaj Gupta
> <pankaj.gupta@nxp.com>; gaurav.jain@nxp.com;
> herbert@gondor.apana.org.au; davem@davemloft.net
> Cc: linux-crypto@vger.kernel.org; linux-kernel@vger.kernel.org;
> lukas@wunner.de; changwei.zou@canonical.com
> Subject: [EXT] [PATCH 1/2] crypto: caam - Fix wrong unmap operations
>
> Caution: This is an external email. Please take care when clicking links or
> opening attachments. When in doubt, report the message using the 'Report
> this email' button
>
>
> Both akcipher_do_one_req() and akcipher_enqueue_req() call
> rsa_pub_unmap() regardless of the key type. priv_form only takes values
> FORM1/FORM2/FORM3 with no distinct public key enumerator.
>
> Use key->d to distinguish public from private key operations, then dispatch to
> the correct unmap function based on key->priv_form.
>
> caam_rsa_set_priv_key_form() implicitly relies on zero-initialization for
> priv_form. Set priv_form = FORM1 explicitly at the head of the function for
> clarity and robustness.
>
> Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
> Assisted-by: OpenCode:claude-sonnet-4.6
> ---
>  drivers/crypto/caam/caampkc.c | 43 +++++++++++++++++++++++++--------
> --
>  1 file changed, 31 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
> index cb001aa1de66..840271840cce 100644
> --- a/drivers/crypto/caam/caampkc.c
> +++ b/drivers/crypto/caam/caampkc.c
> @@ -379,6 +379,7 @@ static int akcipher_do_one_req(struct crypto_engine
> *engine, void *areq)
>         struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
>         struct caam_rsa_req_ctx *req_ctx = akcipher_request_ctx(req);
>         struct caam_rsa_ctx *ctx = akcipher_tfm_ctx_dma(tfm);
> +       struct caam_rsa_key *key = &ctx->key;
>         struct device *jrdev = ctx->dev;
>         u32 *desc = req_ctx->edesc->hw_desc;
>         int ret;
> @@ -391,7 +392,21 @@ static int akcipher_do_one_req(struct crypto_engine
> *engine, void *areq)
>                 return ret;
>
>         if (ret != -EINPROGRESS) {
> -               rsa_pub_unmap(jrdev, req_ctx->edesc, req);
> +               if (key->d) {
> +                       switch (key->priv_form) {
> +                       case FORM1:
> +                               rsa_priv_f1_unmap(jrdev, req_ctx->edesc, req);
> +                               break;
> +                       case FORM2:
> +                               rsa_priv_f2_unmap(jrdev, req_ctx->edesc, req);
> +                               break;
> +                       case FORM3:
> +                               rsa_priv_f3_unmap(jrdev, req_ctx->edesc, req);
> +                               break;
> +                       }
> +               } else {
> +                       rsa_pub_unmap(jrdev, req_ctx->edesc, req);
> +               }
>                 rsa_io_unmap(jrdev, req_ctx->edesc, req);
>                 kfree(req_ctx->edesc);
>         } else {
> @@ -691,17 +706,19 @@ static int akcipher_enqueue_req(struct device
> *jrdev,
>                 ret = caam_jr_enqueue(jrdev, desc, cbk, req);
>
>         if ((ret != -EINPROGRESS) && (ret != -EBUSY)) {
> -               switch (key->priv_form) {
> -               case FORM1:
> -                       rsa_priv_f1_unmap(jrdev, edesc, req);
> -                       break;
> -               case FORM2:
> -                       rsa_priv_f2_unmap(jrdev, edesc, req);
> -                       break;
> -               case FORM3:
> -                       rsa_priv_f3_unmap(jrdev, edesc, req);
> -                       break;
> -               default:
> +               if (key->d) {
> +                       switch (key->priv_form) {
> +                       case FORM1:
> +                               rsa_priv_f1_unmap(jrdev, edesc, req);
> +                               break;
> +                       case FORM2:
> +                               rsa_priv_f2_unmap(jrdev, edesc, req);
> +                               break;
> +                       case FORM3:
> +                               rsa_priv_f3_unmap(jrdev, edesc, req);
> +                               break;
> +                       }
> +               } else {
>                         rsa_pub_unmap(jrdev, edesc, req);
>                 }
>                 rsa_io_unmap(jrdev, edesc, req); @@ -992,6 +1009,8 @@ static int
> caam_rsa_set_priv_key_form(struct caam_rsa_ctx *ctx,
>         size_t q_sz = raw_key->q_sz;
>         unsigned aligned_size;
>
> +       rsa_key->priv_form = FORM1;
> +
>         rsa_key->p = caam_read_raw_data(raw_key->p, &p_sz);
>         if (!rsa_key->p)
>                 return -ENOMEM;
> --
> 2.43.0
>

Re: [EXT] [PATCH 1/2] crypto: caam - Fix wrong unmap operations

[Changwei Zou]

Hi Sahil and Martin,

Thank you very much.

A new version (PATCH v2) is available at the following link.

https://lore.kernel.org/linux-crypto/20261004210200.290119-1-changwei.zou@canonical.com/

Kind regards,
Changwei

[PATCH 2/2] crypto: caam - Use bounce buffer for unaligned RSA destination buffers

[Changwei Zou]

The rsa-caam driver directly DMA-maps the destination buffer supplied by
the caller via req->dst without checking whether it meets the cacheline
alignment requirements of DMA-incoherent hardware such as i.MX8.

On CPUs with non-coherent DMA caches, if the destination buffer shares a
cacheline with other data (i.e. it is not cacheline-aligned), cache
writeback/invalidation during DMA can corrupt adjacent memory or cause
stale data to be read back. This manifests as intermittent -EKEYREJECTED
errors when loading signed kernel modules.

When any segment of req->dst is not cacheline-aligned in either its
start offset or length, allocate a single contiguous aligned bounce
buffer covering the full dst_len rounded up to a cacheline multiple,
redirect the operation to it, and copy the result back to the original
destination once the hardware has completed successfully.

Introduce a helper sg_is_dma_aligned() that checks both sg->offset and
sg->length for cacheline alignment. Checking sg->offset suffices for
the start address since physical pages are always page-aligned and
PAGE_SIZE is a multiple of dma_get_cache_alignment(). sg->length is
also checked to ensure the buffer end does not share a cacheline with
adjacent memory.

The intermittent error 'Key was rejected by service' on i.MX8 with CAAM
can be triggered when loading signed kernel modules:

    for i in $(seq 1 100); do
        sudo modprobe xfs 2>&1 && echo "SUCCESS on attempt $i" \
        && sudo rmmod xfs || echo "FAILED on attempt $i"
    done

Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
Assisted-by: OpenCode:claude-sonnet-4.6
---
 drivers/crypto/caam/caampkc.c | 77 ++++++++++++++++++++++++++++++++++-
 drivers/crypto/caam/caampkc.h |  6 +++
 2 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
index 840271840cce..11c6b07f5dad 100644
--- a/drivers/crypto/caam/caampkc.c
+++ b/drivers/crypto/caam/caampkc.c
@@ -59,6 +59,40 @@ static void rsa_io_unmap(struct device *dev, struct rsa_edesc *edesc,
 				 DMA_TO_DEVICE);
 }
 
+static int do_rsa_bounce_buf(struct akcipher_request *req, int req_err)
+{
+	struct caam_rsa_req_ctx *req_ctx = akcipher_request_ctx(req);
+	int nents, err = 0;
+
+	if (!req_ctx->bounce_buf)
+		return 0;
+
+	/* Only copy back to the original destination on success */
+	if (!req_err) {
+		nents = sg_nents_for_len(req_ctx->orig_dst, req->dst_len);
+		if (nents < 0)
+			err = nents;
+		else if (sg_copy_from_buffer(req_ctx->orig_dst, nents,
+					     req_ctx->bounce_buf,
+					     req->dst_len) != req->dst_len)
+			err = -EFAULT;
+	}
+
+	kfree(req_ctx->bounce_buf);
+	req_ctx->bounce_buf = NULL;
+	req->dst = req_ctx->orig_dst;
+
+	return err;
+}
+
+static inline void rsa_bounce_buf_done(struct akcipher_request *req, int *err)
+{
+	int cperr = do_rsa_bounce_buf(req, *err);
+
+	if (!*err)
+		*err = cperr;
+}
+
 static void rsa_pub_unmap(struct device *dev, struct rsa_edesc *edesc,
 			  struct akcipher_request *req)
 {
@@ -138,6 +172,7 @@ static void rsa_pub_done(struct device *dev, u32 *desc, u32 err, void *context)
 	rsa_pub_unmap(dev, edesc, req);
 	rsa_io_unmap(dev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ecode);
 
 	/*
 	 * If no backlog flag, the completion of the request is done
@@ -181,6 +216,7 @@ static void rsa_priv_f_done(struct device *dev, u32 *desc, u32 err,
 
 	rsa_io_unmap(dev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ecode);
 
 	/*
 	 * If no backlog flag, the completion of the request is done
@@ -246,6 +282,12 @@ static int caam_rsa_count_leading_zeros(struct scatterlist *sgl,
 	return tbytes - nbytes;
 }
 
+static inline bool sg_is_dma_aligned(struct scatterlist *sg)
+{
+	return IS_ALIGNED(sg->offset, dma_get_cache_alignment()) &&
+	       IS_ALIGNED(sg->length, dma_get_cache_alignment());
+}
+
 static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
 					 size_t desclen)
 {
@@ -291,11 +333,34 @@ static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
 				     req_ctx->fixup_src_len);
 	dst_nents = sg_nents_for_len(req->dst, req->dst_len);
 
+	req_ctx->bounce_buf = NULL;
+	req_ctx->orig_dst = req->dst;
+	if (req->dst_len > 0) {
+		struct scatterlist *sg;
+		int i;
+
+		for_each_sg(req->dst, sg, dst_nents, i) {
+			if (!sg_is_dma_aligned(sg)) {
+				req_ctx->bounce_buf =
+					kzalloc(ALIGN(req->dst_len,
+						      dma_get_cache_alignment()),
+						flags);
+				if (!req_ctx->bounce_buf)
+					return ERR_PTR(-ENOMEM);
+				sg_init_one(&req_ctx->dst, req_ctx->bounce_buf,
+					    req->dst_len);
+				req->dst = &req_ctx->dst;
+				dst_nents = 1;
+				break;
+			}
+		}
+	}
+
 	mapped_src_nents = dma_map_sg(dev, req_ctx->fixup_src, src_nents,
 				      DMA_TO_DEVICE);
 	if (unlikely(!mapped_src_nents)) {
 		dev_err(dev, "unable to map source\n");
-		return ERR_PTR(-ENOMEM);
+		goto bounce_fail;
 	}
 	mapped_dst_nents = dma_map_sg(dev, req->dst, dst_nents,
 				      DMA_FROM_DEVICE);
@@ -368,6 +433,10 @@ static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
 	dma_unmap_sg(dev, req->dst, dst_nents, DMA_FROM_DEVICE);
 src_fail:
 	dma_unmap_sg(dev, req_ctx->fixup_src, src_nents, DMA_TO_DEVICE);
+bounce_fail:
+	kfree(req_ctx->bounce_buf);
+	req_ctx->bounce_buf = NULL;
+	req->dst = req_ctx->orig_dst;
 	return ERR_PTR(-ENOMEM);
 }
 
@@ -409,6 +478,7 @@ static int akcipher_do_one_req(struct crypto_engine *engine, void *areq)
 		}
 		rsa_io_unmap(jrdev, req_ctx->edesc, req);
 		kfree(req_ctx->edesc);
+		rsa_bounce_buf_done(req, &ret);
 	} else {
 		ret = 0;
 	}
@@ -723,6 +793,7 @@ static int akcipher_enqueue_req(struct device *jrdev,
 		}
 		rsa_io_unmap(jrdev, edesc, req);
 		kfree(edesc);
+		rsa_bounce_buf_done(req, &ret);
 	}
 
 	return ret;
@@ -764,6 +835,7 @@ static int caam_rsa_enc(struct akcipher_request *req)
 init_fail:
 	rsa_io_unmap(jrdev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ret);
 	return ret;
 }
 
@@ -793,6 +865,7 @@ static int caam_rsa_dec_priv_f1(struct akcipher_request *req)
 init_fail:
 	rsa_io_unmap(jrdev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ret);
 	return ret;
 }
 
@@ -822,6 +895,7 @@ static int caam_rsa_dec_priv_f2(struct akcipher_request *req)
 init_fail:
 	rsa_io_unmap(jrdev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ret);
 	return ret;
 }
 
@@ -851,6 +925,7 @@ static int caam_rsa_dec_priv_f3(struct akcipher_request *req)
 init_fail:
 	rsa_io_unmap(jrdev, edesc, req);
 	kfree(edesc);
+	rsa_bounce_buf_done(req, &ret);
 	return ret;
 }
 
diff --git a/drivers/crypto/caam/caampkc.h b/drivers/crypto/caam/caampkc.h
index 96d03704c9be..efad91d6058f 100644
--- a/drivers/crypto/caam/caampkc.h
+++ b/drivers/crypto/caam/caampkc.h
@@ -103,6 +103,9 @@ struct caam_rsa_ctx {
  * @src           : input scatterlist (stripped of leading zeros)
  * @fixup_src     : input scatterlist (that might be stripped of leading zeros)
  * @fixup_src_len : length of the fixup_src input scatterlist
+ * @dst           : destination scatterlist backed by bounce buffer (if needed)
+ * @bounce_buf    : DMA-aligned bounce buffer for destination (or NULL)
+ * @orig_dst      : original destination scatterlist (before bounce substitution)
  * @edesc         : s/w-extended rsa descriptor
  * @akcipher_op_done : callback used when operation is done
  */
@@ -110,6 +113,9 @@ struct caam_rsa_req_ctx {
 	struct scatterlist src[2];
 	struct scatterlist *fixup_src;
 	unsigned int fixup_src_len;
+	struct scatterlist dst;
+	u8 *bounce_buf;
+	struct scatterlist *orig_dst;
 	struct rsa_edesc *edesc;
 	void (*akcipher_op_done)(struct device *jrdev, u32 *desc, u32 err,
 				 void *context);
-- 
2.43.0

RE: [EXT] [PATCH 2/2] crypto: caam - Use bounce buffer for unaligned RSA destination buffers

[Sahil Malhotra (OSS)]

Hi Changwei,

Please find my comment inline.


NXP Confidential
> -----Original Message-----
> From: Changwei Zou <changwei.zou@canonical.com>
> Sent: 21 August 2026 10:26
> To: Horia Geanta <horia.geanta@nxp.com>; Pankaj Gupta
> <pankaj.gupta@nxp.com>; gaurav.jain@nxp.com;
> herbert@gondor.apana.org.au; davem@davemloft.net
> Cc: linux-crypto@vger.kernel.org; linux-kernel@vger.kernel.org;
> lukas@wunner.de; changwei.zou@canonical.com
> Subject: [EXT] [PATCH 2/2] crypto: caam - Use bounce buffer for unaligned
> RSA destination buffers
>
> Caution: This is an external email. Please take care when clicking links or
> opening attachments. When in doubt, report the message using the 'Report
> this email' button
>
>
> The rsa-caam driver directly DMA-maps the destination buffer supplied by the
> caller via req->dst without checking whether it meets the cacheline alignment
> requirements of DMA-incoherent hardware such as i.MX8.
>
> On CPUs with non-coherent DMA caches, if the destination buffer shares a
> cacheline with other data (i.e. it is not cacheline-aligned), cache
> writeback/invalidation during DMA can corrupt adjacent memory or cause
> stale data to be read back. This manifests as intermittent -EKEYREJECTED errors
> when loading signed kernel modules.
>
> When any segment of req->dst is not cacheline-aligned in either its start offset
> or length, allocate a single contiguous aligned bounce buffer covering the full
> dst_len rounded up to a cacheline multiple, redirect the operation to it, and
> copy the result back to the original destination once the hardware has
> completed successfully.
>
> Introduce a helper sg_is_dma_aligned() that checks both sg->offset and
> sg->length for cacheline alignment. Checking sg->offset suffices for
> the start address since physical pages are always page-aligned and PAGE_SIZE
> is a multiple of dma_get_cache_alignment(). sg->length is also checked to
> ensure the buffer end does not share a cacheline with adjacent memory.
>
> The intermittent error 'Key was rejected by service' on i.MX8 with CAAM can
> be triggered when loading signed kernel modules:
>
>     for i in $(seq 1 100); do
>         sudo modprobe xfs 2>&1 && echo "SUCCESS on attempt $i" \
>         && sudo rmmod xfs || echo "FAILED on attempt $i"
>     done
>
> Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
> Assisted-by: OpenCode:claude-sonnet-4.6
> ---
>  drivers/crypto/caam/caampkc.c | 77
> ++++++++++++++++++++++++++++++++++-
>  drivers/crypto/caam/caampkc.h |  6 +++
>  2 files changed, 82 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
> index 840271840cce..11c6b07f5dad 100644
> --- a/drivers/crypto/caam/caampkc.c
> +++ b/drivers/crypto/caam/caampkc.c
> @@ -59,6 +59,40 @@ static void rsa_io_unmap(struct device *dev, struct
> rsa_edesc *edesc,
>                                  DMA_TO_DEVICE);  }
>
> +static int do_rsa_bounce_buf(struct akcipher_request *req, int req_err)
> +{
> +       struct caam_rsa_req_ctx *req_ctx = akcipher_request_ctx(req);
> +       int nents, err = 0;
> +
> +       if (!req_ctx->bounce_buf)
> +               return 0;
Please add this check in rsa_bounce_buf_done() and return from there only.
We need not to come here to return.


> +
> +       /* Only copy back to the original destination on success */
> +       if (!req_err) {
> +               nents = sg_nents_for_len(req_ctx->orig_dst, req->dst_len);
> +               if (nents < 0)
> +                       err = nents;
> +               else if (sg_copy_from_buffer(req_ctx->orig_dst, nents,
> +                                            req_ctx->bounce_buf,
> +                                            req->dst_len) != req->dst_len)
> +                       err = -EFAULT;
> +       }
> +

Re: [PATCH 2/2] crypto: caam - Use bounce buffer for unaligned RSA destination buffers

[Kepplinger-Novakovic Martin]

Am Freitag, dem 21.08.2026 um 14:55 +1000 schrieb Changwei Zou:
> The rsa-caam driver directly DMA-maps the destination buffer supplied by
> the caller via req->dst without checking whether it meets the cacheline
> alignment requirements of DMA-incoherent hardware such as i.MX8.
> 
> On CPUs with non-coherent DMA caches, if the destination buffer shares a
> cacheline with other data (i.e. it is not cacheline-aligned), cache
> writeback/invalidation during DMA can corrupt adjacent memory or cause
> stale data to be read back. This manifests as intermittent -EKEYREJECTED
> errors when loading signed kernel modules.
> 
> When any segment of req->dst is not cacheline-aligned in either its
> start offset or length, allocate a single contiguous aligned bounce
> buffer covering the full dst_len rounded up to a cacheline multiple,
> redirect the operation to it, and copy the result back to the original
> destination once the hardware has completed successfully.
> 
> Introduce a helper sg_is_dma_aligned() that checks both sg->offset and
> sg->length for cacheline alignment. Checking sg->offset suffices for
> the start address since physical pages are always page-aligned and
> PAGE_SIZE is a multiple of dma_get_cache_alignment(). sg->length is
> also checked to ensure the buffer end does not share a cacheline with
> adjacent memory.
> 
> The intermittent error 'Key was rejected by service' on i.MX8 with CAAM
> can be triggered when loading signed kernel modules:
> 
>     for i in $(seq 1 100); do
>         sudo modprobe xfs 2>&1 && echo "SUCCESS on attempt $i" \
>         && sudo rmmod xfs || echo "FAILED on attempt $i"
>     done
> 
> Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
> Assisted-by: OpenCode:claude-sonnet-4.6
> ---
>  drivers/crypto/caam/caampkc.c | 77 ++++++++++++++++++++++++++++++++++-
>  drivers/crypto/caam/caampkc.h |  6 +++
>  2 files changed, 82 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
> index 840271840cce..11c6b07f5dad 100644
> --- a/drivers/crypto/caam/caampkc.c
> +++ b/drivers/crypto/caam/caampkc.c
> @@ -59,6 +59,40 @@ static void rsa_io_unmap(struct device *dev, struct rsa_edesc *edesc,
>  				 DMA_TO_DEVICE);
>  }
>  
> +static int do_rsa_bounce_buf(struct akcipher_request *req, int req_err)
> +{
> +	struct caam_rsa_req_ctx *req_ctx = akcipher_request_ctx(req);
> +	int nents, err = 0;
> +
> +	if (!req_ctx->bounce_buf)
> +		return 0;
> +
> +	/* Only copy back to the original destination on success */
> +	if (!req_err) {
> +		nents = sg_nents_for_len(req_ctx->orig_dst, req->dst_len);
> +		if (nents < 0)
> +			err = nents;
> +		else if (sg_copy_from_buffer(req_ctx->orig_dst, nents,
> +					     req_ctx->bounce_buf,
> +					     req->dst_len) != req->dst_len)
> +			err = -EFAULT;
> +	}
> +
> +	kfree(req_ctx->bounce_buf);
> +	req_ctx->bounce_buf = NULL;
> +	req->dst = req_ctx->orig_dst;
> +
> +	return err;
> +}
> +
> +static inline void rsa_bounce_buf_done(struct akcipher_request *req, int *err)
> +{
> +	int cperr = do_rsa_bounce_buf(req, *err);
> +
> +	if (!*err)
> +		*err = cperr;
> +}
> +
>  static void rsa_pub_unmap(struct device *dev, struct rsa_edesc *edesc,
>  			  struct akcipher_request *req)
>  {
> @@ -138,6 +172,7 @@ static void rsa_pub_done(struct device *dev, u32 *desc, u32 err, void *context)
>  	rsa_pub_unmap(dev, edesc, req);
>  	rsa_io_unmap(dev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ecode);
>  
>  	/*
>  	 * If no backlog flag, the completion of the request is done
> @@ -181,6 +216,7 @@ static void rsa_priv_f_done(struct device *dev, u32 *desc, u32 err,
>  
>  	rsa_io_unmap(dev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ecode);
>  
>  	/*
>  	 * If no backlog flag, the completion of the request is done
> @@ -246,6 +282,12 @@ static int caam_rsa_count_leading_zeros(struct scatterlist *sgl,
>  	return tbytes - nbytes;
>  }
>  
> +static inline bool sg_is_dma_aligned(struct scatterlist *sg)
> +{
> +	return IS_ALIGNED(sg->offset, dma_get_cache_alignment()) &&
> +	       IS_ALIGNED(sg->length, dma_get_cache_alignment());
> +}
> +
>  static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
>  					 size_t desclen)
>  {
> @@ -291,11 +333,34 @@ static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
>  				     req_ctx->fixup_src_len);
>  	dst_nents = sg_nents_for_len(req->dst, req->dst_len);
>  
> +	req_ctx->bounce_buf = NULL;
> +	req_ctx->orig_dst = req->dst;
> +	if (req->dst_len > 0) {
> +		struct scatterlist *sg;
> +		int i;
> +
> +		for_each_sg(req->dst, sg, dst_nents, i) {
> +			if (!sg_is_dma_aligned(sg)) {
> +				req_ctx->bounce_buf =
> +					kzalloc(ALIGN(req->dst_len,
> +						      dma_get_cache_alignment()),
> +						flags);
> +				if (!req_ctx->bounce_buf)
> +					return ERR_PTR(-ENOMEM);
> +				sg_init_one(&req_ctx->dst, req_ctx->bounce_buf,
> +					    req->dst_len);
> +				req->dst = &req_ctx->dst;
> +				dst_nents = 1;
> +				break;
> +			}
> +		}
> +	}
> +
>  	mapped_src_nents = dma_map_sg(dev, req_ctx->fixup_src, src_nents,
>  				      DMA_TO_DEVICE);
>  	if (unlikely(!mapped_src_nents)) {
>  		dev_err(dev, "unable to map source\n");
> -		return ERR_PTR(-ENOMEM);
> +		goto bounce_fail;
>  	}
>  	mapped_dst_nents = dma_map_sg(dev, req->dst, dst_nents,
>  				      DMA_FROM_DEVICE);
> @@ -368,6 +433,10 @@ static struct rsa_edesc *rsa_edesc_alloc(struct akcipher_request *req,
>  	dma_unmap_sg(dev, req->dst, dst_nents, DMA_FROM_DEVICE);
>  src_fail:
>  	dma_unmap_sg(dev, req_ctx->fixup_src, src_nents, DMA_TO_DEVICE);
> +bounce_fail:
> +	kfree(req_ctx->bounce_buf);
> +	req_ctx->bounce_buf = NULL;
> +	req->dst = req_ctx->orig_dst;
>  	return ERR_PTR(-ENOMEM);
>  }
>  
> @@ -409,6 +478,7 @@ static int akcipher_do_one_req(struct crypto_engine *engine, void *areq)
>  		}
>  		rsa_io_unmap(jrdev, req_ctx->edesc, req);
>  		kfree(req_ctx->edesc);
> +		rsa_bounce_buf_done(req, &ret);
>  	} else {
>  		ret = 0;
>  	}
> @@ -723,6 +793,7 @@ static int akcipher_enqueue_req(struct device *jrdev,
>  		}
>  		rsa_io_unmap(jrdev, edesc, req);
>  		kfree(edesc);
> +		rsa_bounce_buf_done(req, &ret);
>  	}
>  
>  	return ret;
> @@ -764,6 +835,7 @@ static int caam_rsa_enc(struct akcipher_request *req)
>  init_fail:
>  	rsa_io_unmap(jrdev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ret);
>  	return ret;
>  }
>  
> @@ -793,6 +865,7 @@ static int caam_rsa_dec_priv_f1(struct akcipher_request *req)
>  init_fail:
>  	rsa_io_unmap(jrdev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ret);
>  	return ret;
>  }
>  
> @@ -822,6 +895,7 @@ static int caam_rsa_dec_priv_f2(struct akcipher_request *req)
>  init_fail:
>  	rsa_io_unmap(jrdev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ret);
>  	return ret;
>  }
>  
> @@ -851,6 +925,7 @@ static int caam_rsa_dec_priv_f3(struct akcipher_request *req)
>  init_fail:
>  	rsa_io_unmap(jrdev, edesc, req);
>  	kfree(edesc);
> +	rsa_bounce_buf_done(req, &ret);
>  	return ret;
>  }
>  
> diff --git a/drivers/crypto/caam/caampkc.h b/drivers/crypto/caam/caampkc.h
> index 96d03704c9be..efad91d6058f 100644
> --- a/drivers/crypto/caam/caampkc.h
> +++ b/drivers/crypto/caam/caampkc.h
> @@ -103,6 +103,9 @@ struct caam_rsa_ctx {
>   * @src           : input scatterlist (stripped of leading zeros)
>   * @fixup_src     : input scatterlist (that might be stripped of leading zeros)
>   * @fixup_src_len : length of the fixup_src input scatterlist
> + * @dst           : destination scatterlist backed by bounce buffer (if needed)
> + * @bounce_buf    : DMA-aligned bounce buffer for destination (or NULL)
> + * @orig_dst      : original destination scatterlist (before bounce substitution)
>   * @edesc         : s/w-extended rsa descriptor
>   * @akcipher_op_done : callback used when operation is done
>   */
> @@ -110,6 +113,9 @@ struct caam_rsa_req_ctx {
>  	struct scatterlist src[2];
>  	struct scatterlist *fixup_src;
>  	unsigned int fixup_src_len;
> +	struct scatterlist dst;
> +	u8 *bounce_buf;
> +	struct scatterlist *orig_dst;
>  	struct rsa_edesc *edesc;
>  	void (*akcipher_op_done)(struct device *jrdev, u32 *desc, u32 err,
>  				 void *context);


hi Changwei,

I now test booting a signed squashfs rootfs using DM_VERITY on imx8mp and basically get similar issues like I did
during my tests on imx6ul earlier: https://lore.kernel.org/linux-crypto/6029acc0f0ddfe25e2537c2866d54fd7f54bc182.camel@ginzinger.com/T/#u

Your 2 patches applied fix this for me.

It's new to me, but quite simple if you know secureboot better than I do:
I create a keypair. I create a hashtree and append it to the squashfs. And after adding cert.pem to
CONFIG_SYSTEM_TRUSTED_KEYS it should verify and mount successfully.

the image is ok, and with crypto-debug enabled, I saw errors like this:

PKCS7: Sig 1: Issuing X.509 cert not found (#0da1e6a5e8314eb32932bc88de47509476e4ec85 ... "Ginzinger GESB rootfs signing key")
device-mapper: table: 254:0: verity: Root hash verification failed (-ENOKEY)

But also the regdb cert failed to load during boot:

[    0.283510] Loading compiled-in X.509 certificates
[    0.288607] Problem loading in-kernel X.509 certificate (-22)

which works now as well.


Tested-by: Martin Kepplinger-Novakovic <martin.kepplinger-novakovic@ginzinger.com>

Do you think this can be merged or should I disabled CONFIG_CRYPTO_DEV_FSL_CAAM_PKC_API for now?

thank you!

                                         martin