diff mbox series

[v2,01/13] KVM: s390: pv: avoid stall notifications for some UVCs

Message ID 20210728142631.41860-2-imbrenda@linux.ibm.com (mailing list archive)
State New, archived
Headers show
Series KVM: s390: pv: implement lazy destroy | expand

Commit Message

Claudio Imbrenda July 28, 2021, 2:26 p.m. UTC 
Improve make_secure_pte to avoid stalls when the system is heavily
overcommitted. This was especially problematic in kvm_s390_pv_unpack,
because of the loop over all pages that needed unpacking.

Also fix kvm_s390_pv_init_vm to avoid stalls when the system is heavily
overcommitted.

Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
 arch/s390/kernel/uv.c | 11 ++++++++---
 arch/s390/kvm/pv.c    |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)

Comments

Janosch Frank July 29, 2021, 9:58 a.m. UTC | #1 
On 7/28/21 4:26 PM, Claudio Imbrenda wrote:
> Improve make_secure_pte to avoid stalls when the system is heavily
> overcommitted. This was especially problematic in kvm_s390_pv_unpack,
> because of the loop over all pages that needed unpacking.
> 
> Also fix kvm_s390_pv_init_vm to avoid stalls when the system is heavily
> overcommitted.

Fixes tag?

> 
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
>  arch/s390/kernel/uv.c | 11 ++++++++---
>  arch/s390/kvm/pv.c    |  2 +-
>  2 files changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
> index aeb0a15bcbb7..fd0faa51c1bb 100644
> --- a/arch/s390/kernel/uv.c
> +++ b/arch/s390/kernel/uv.c
> @@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,
>  	if (!page_ref_freeze(page, expected))
>  		return -EBUSY;
>  	set_bit(PG_arch_1, &page->flags);
> -	rc = uv_call(0, (u64)uvcb);
> +	rc = __uv_call(0, (u64)uvcb);

We should exchange rc with cc since that's what we get back from
__uv_call(). Technically we always get a cc but for the other functions
it's only ever 0/1 which translates to success/error so rc is ok.

>  	page_ref_unfreeze(page, expected);
> -	/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
> -	if (rc)
> +	/*
> +	 * Return -ENXIO if the page was not mapped, -EINVAL for other errors.
> +	 * If busy or partially completed, return -EAGAIN.
> +	 */
> +	if (rc == 1)
>  		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
> +	else if (rc > 1)
> +		rc = -EAGAIN;
>  	return rc;

Could you define the CCs in uv.h and check against the constants here so
it's easier to understand that the rc > 1 checks against a "UV was busy
please re-issue the call again" cc?

Maybe also make it explicit for cc 2 and 3 instead of cc > 1

>  }
>  
> diff --git a/arch/s390/kvm/pv.c b/arch/s390/kvm/pv.c
> index c8841f476e91..e007df11a2fe 100644
> --- a/arch/s390/kvm/pv.c
> +++ b/arch/s390/kvm/pv.c
> @@ -196,7 +196,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)
>  	uvcb.conf_base_stor_origin = (u64)kvm->arch.pv.stor_base;
>  	uvcb.conf_virt_stor_origin = (u64)kvm->arch.pv.stor_var;
>  
> -	cc = uv_call(0, (u64)&uvcb);
> +	cc = uv_call_sched(0, (u64)&uvcb);
>  	*rc = uvcb.header.rc;
>  	*rrc = uvcb.header.rrc;
>  	KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len %llx rc %x rrc %x",
>
Cornelia Huck July 29, 2021, 10:49 a.m. UTC | #2 
On Wed, Jul 28 2021, Claudio Imbrenda <imbrenda@linux.ibm.com> wrote:

> Improve make_secure_pte to avoid stalls when the system is heavily
> overcommitted. This was especially problematic in kvm_s390_pv_unpack,
> because of the loop over all pages that needed unpacking.
>
> Also fix kvm_s390_pv_init_vm to avoid stalls when the system is heavily
> overcommitted.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
>  arch/s390/kernel/uv.c | 11 ++++++++---
>  arch/s390/kvm/pv.c    |  2 +-
>  2 files changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
> index aeb0a15bcbb7..fd0faa51c1bb 100644
> --- a/arch/s390/kernel/uv.c
> +++ b/arch/s390/kernel/uv.c
> @@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,
>  	if (!page_ref_freeze(page, expected))
>  		return -EBUSY;
>  	set_bit(PG_arch_1, &page->flags);
> -	rc = uv_call(0, (u64)uvcb);
> +	rc = __uv_call(0, (u64)uvcb);
>  	page_ref_unfreeze(page, expected);
> -	/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
> -	if (rc)
> +	/*
> +	 * Return -ENXIO if the page was not mapped, -EINVAL for other errors.
> +	 * If busy or partially completed, return -EAGAIN.
> +	 */
> +	if (rc == 1)
>  		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
> +	else if (rc > 1)
> +		rc = -EAGAIN;
>  	return rc;
>  }

Possibly dumb question: when does the call return > 1?
gmap_make_secure() will do a wait_on_page_writeback() for -EAGAIN, is
that always the right thing to do?
Claudio Imbrenda July 29, 2021, 12:52 p.m. UTC | #3 
On Thu, 29 Jul 2021 11:58:39 +0200
Janosch Frank <frankja@linux.ibm.com> wrote:

> On 7/28/21 4:26 PM, Claudio Imbrenda wrote:
> > Improve make_secure_pte to avoid stalls when the system is heavily
> > overcommitted. This was especially problematic in
> > kvm_s390_pv_unpack, because of the loop over all pages that needed
> > unpacking.
> > 
> > Also fix kvm_s390_pv_init_vm to avoid stalls when the system is
> > heavily overcommitted.  
> 
> Fixes tag?

will be in the next version

> > 
> > Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> > ---
> >  arch/s390/kernel/uv.c | 11 ++++++++---
> >  arch/s390/kvm/pv.c    |  2 +-
> >  2 files changed, 9 insertions(+), 4 deletions(-)
> > 
> > diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
> > index aeb0a15bcbb7..fd0faa51c1bb 100644
> > --- a/arch/s390/kernel/uv.c
> > +++ b/arch/s390/kernel/uv.c
> > @@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep,
> > unsigned long addr, if (!page_ref_freeze(page, expected))
> >  		return -EBUSY;
> >  	set_bit(PG_arch_1, &page->flags);
> > -	rc = uv_call(0, (u64)uvcb);
> > +	rc = __uv_call(0, (u64)uvcb);  
> 
> We should exchange rc with cc since that's what we get back from
> __uv_call(). Technically we always get a cc but for the other
> functions it's only ever 0/1 which translates to success/error so rc
> is ok.

will be in the next version

> >  	page_ref_unfreeze(page, expected);
> > -	/* Return -ENXIO if the page was not mapped, -EINVAL
> > otherwise */
> > -	if (rc)
> > +	/*
> > +	 * Return -ENXIO if the page was not mapped, -EINVAL for
> > other errors.
> > +	 * If busy or partially completed, return -EAGAIN.
> > +	 */
> > +	if (rc == 1)
> >  		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
> > +	else if (rc > 1)
> > +		rc = -EAGAIN;
> >  	return rc;  
> 
> Could you define the CCs in uv.h and check against the constants here
> so it's easier to understand that the rc > 1 checks against a "UV was
> busy please re-issue the call again" cc?
>
> Maybe also make it explicit for cc 2 and 3 instead of cc > 1

will be in the next version

> >  }
> >  
> > diff --git a/arch/s390/kvm/pv.c b/arch/s390/kvm/pv.c
> > index c8841f476e91..e007df11a2fe 100644
> > --- a/arch/s390/kvm/pv.c
> > +++ b/arch/s390/kvm/pv.c
> > @@ -196,7 +196,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16
> > *rc, u16 *rrc) uvcb.conf_base_stor_origin =
> > (u64)kvm->arch.pv.stor_base; uvcb.conf_virt_stor_origin =
> > (u64)kvm->arch.pv.stor_var; 
> > -	cc = uv_call(0, (u64)&uvcb);
> > +	cc = uv_call_sched(0, (u64)&uvcb);
> >  	*rc = uvcb.header.rc;
> >  	*rrc = uvcb.header.rrc;
> >  	KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len
> > %llx rc %x rrc %x", 
>
Claudio Imbrenda July 29, 2021, 1:22 p.m. UTC | #4 
On Thu, 29 Jul 2021 12:49:03 +0200
Cornelia Huck <cohuck@redhat.com> wrote:

> On Wed, Jul 28 2021, Claudio Imbrenda <imbrenda@linux.ibm.com> wrote:
> 
> > Improve make_secure_pte to avoid stalls when the system is heavily
> > overcommitted. This was especially problematic in
> > kvm_s390_pv_unpack, because of the loop over all pages that needed
> > unpacking.
> >
> > Also fix kvm_s390_pv_init_vm to avoid stalls when the system is
> > heavily overcommitted.
> >
> > Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> > ---
> >  arch/s390/kernel/uv.c | 11 ++++++++---
> >  arch/s390/kvm/pv.c    |  2 +-
> >  2 files changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
> > index aeb0a15bcbb7..fd0faa51c1bb 100644
> > --- a/arch/s390/kernel/uv.c
> > +++ b/arch/s390/kernel/uv.c
> > @@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep,
> > unsigned long addr, if (!page_ref_freeze(page, expected))
> >  		return -EBUSY;
> >  	set_bit(PG_arch_1, &page->flags);
> > -	rc = uv_call(0, (u64)uvcb);
> > +	rc = __uv_call(0, (u64)uvcb);
> >  	page_ref_unfreeze(page, expected);
> > -	/* Return -ENXIO if the page was not mapped, -EINVAL
> > otherwise */
> > -	if (rc)
> > +	/*
> > +	 * Return -ENXIO if the page was not mapped, -EINVAL for
> > other errors.
> > +	 * If busy or partially completed, return -EAGAIN.
> > +	 */
> > +	if (rc == 1)
> >  		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
> > +	else if (rc > 1)
> > +		rc = -EAGAIN;
> >  	return rc;
> >  }  
> 
> Possibly dumb question: when does the call return > 1?

this is exactly what Janosch meant :)

the next version will have #defines for the 4 possible CC values.

in short:
0 OK
1 error
2 busy (nothing done, try again)
3 partial (something done but not all, try again)

> gmap_make_secure() will do a wait_on_page_writeback() for -EAGAIN, is
> that always the right thing to do?

it's the easiest way to get to a place where we will be able to
reschedule if needed.

wait_on_page_writeback will probably do nothing in that case because
the page is not in writeback.

(a few minutes later)

actually I have checked, it seems that the -EAGAIN gets eventually
propagated to places where it's not checked properly!

this will need some more fixing
diff mbox series

Patch

diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index aeb0a15bcbb7..fd0faa51c1bb 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -196,11 +196,16 @@  static int make_secure_pte(pte_t *ptep, unsigned long addr,
 	if (!page_ref_freeze(page, expected))
 		return -EBUSY;
 	set_bit(PG_arch_1, &page->flags);
-	rc = uv_call(0, (u64)uvcb);
+	rc = __uv_call(0, (u64)uvcb);
 	page_ref_unfreeze(page, expected);
-	/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
-	if (rc)
+	/*
+	 * Return -ENXIO if the page was not mapped, -EINVAL for other errors.
+	 * If busy or partially completed, return -EAGAIN.
+	 */
+	if (rc == 1)
 		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
+	else if (rc > 1)
+		rc = -EAGAIN;
 	return rc;
 }
 
diff --git a/arch/s390/kvm/pv.c b/arch/s390/kvm/pv.c
index c8841f476e91..e007df11a2fe 100644
--- a/arch/s390/kvm/pv.c
+++ b/arch/s390/kvm/pv.c
@@ -196,7 +196,7 @@  int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)
 	uvcb.conf_base_stor_origin = (u64)kvm->arch.pv.stor_base;
 	uvcb.conf_virt_stor_origin = (u64)kvm->arch.pv.stor_var;
 
-	cc = uv_call(0, (u64)&uvcb);
+	cc = uv_call_sched(0, (u64)&uvcb);
 	*rc = uvcb.header.rc;
 	*rrc = uvcb.header.rrc;
 	KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len %llx rc %x rrc %x",