diff mbox series

[05/26] cxl/core: Simplify cxl_dpa_set_mode()

Message ID 20240324-dcd-type2-upstream-v1-5-b7b00d623625@intel.com
State Changes Requested
Headers show
Series DCD: Add support for Dynamic Capacity Devices (DCD) | expand

Commit Message

Ira Weiny March 24, 2024, 11:18 p.m. UTC
cxl_dpa_set_mode() checks the mode for validity two times, once outside
of the DPA RW semaphore and again within.  The function is not in a
critical path.  Prior to Dynamic Capacity the extra check was not much
of an issue.  The addition of DC modes increases the complexity of
the check.

Simplify the mode check before adding the more complex DC modes.

Signed-off-by: Ira Weiny <ira.weiny@intel.com>

---
Changes for v1:
[iweiny: new patch]
[Jonathan: based on getting rid of the loop in cxl_dpa_set_mode]
[Jonathan: standardize on resource_size() == 0]
---
 drivers/cxl/core/hdm.c | 45 ++++++++++++++++++---------------------------
 1 file changed, 18 insertions(+), 27 deletions(-)

Comments

Jonathan Cameron March 25, 2024, 5:46 p.m. UTC | #1
On Sun, 24 Mar 2024 16:18:08 -0700
Ira Weiny <ira.weiny@intel.com> wrote:

> cxl_dpa_set_mode() checks the mode for validity two times, once outside
> of the DPA RW semaphore and again within.  The function is not in a
> critical path.  Prior to Dynamic Capacity the extra check was not much
> of an issue.  The addition of DC modes increases the complexity of
> the check.
> 
> Simplify the mode check before adding the more complex DC modes.
> 
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>

Nice. Maybe drag this earlier in series so it could potentially be
picked up as a cleanup?  Same with patch 2 potentially.
If Dave is fine with doing that sort of precursor patches going
earlier, it will save carrying quite so many in this series for
future versions (and make it look less terrifying :)

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Davidlohr Bueso March 25, 2024, 9:38 p.m. UTC | #2
On Sun, 24 Mar 2024, Ira Weiny wrote:

>cxl_dpa_set_mode() checks the mode for validity two times, once outside
>of the DPA RW semaphore and again within.  The function is not in a
>critical path.  Prior to Dynamic Capacity the extra check was not much
>of an issue.  The addition of DC modes increases the complexity of
>the check.

I agree (also to pick this up regardless of dcd work).

>
>Simplify the mode check before adding the more complex DC modes.

Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>

>Signed-off-by: Ira Weiny <ira.weiny@intel.com>
>
>---
>Changes for v1:
>[iweiny: new patch]
>[Jonathan: based on getting rid of the loop in cxl_dpa_set_mode]
>[Jonathan: standardize on resource_size() == 0]
>---
> drivers/cxl/core/hdm.c | 45 ++++++++++++++++++---------------------------
> 1 file changed, 18 insertions(+), 27 deletions(-)
>
>diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
>index 7d97790b893d..66b8419fd0c3 100644
>--- a/drivers/cxl/core/hdm.c
>+++ b/drivers/cxl/core/hdm.c
>@@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
>	struct cxl_dev_state *cxlds = cxlmd->cxlds;
>	struct device *dev = &cxled->cxld.dev;
>-	int rc;
>
>+	guard(rwsem_write)(&cxl_dpa_rwsem);
>+	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
>+		return -EBUSY;
>+
>+	/*
>+	 * Check that the mode is supported by the current partition
>+	 * configuration
>+	 */
>	switch (mode) {
>	case CXL_DECODER_RAM:
>+		if (!resource_size(&cxlds->ram_res)) {
>+			dev_dbg(dev, "no available ram capacity\n");
>+			return -ENXIO;
>+		}
>+		break;
>	case CXL_DECODER_PMEM:
>+		if (!resource_size(&cxlds->pmem_res)) {
>+			dev_dbg(dev, "no available pmem capacity\n");
>+			return -ENXIO;
>+		}
>		break;
>	default:
>		dev_dbg(dev, "unsupported mode: %d\n", mode);
>		return -EINVAL;
>	}
>
>-	down_write(&cxl_dpa_rwsem);
>-	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) {
>-		rc = -EBUSY;
>-		goto out;
>-	}
>-
>-	/*
>-	 * Only allow modes that are supported by the current partition
>-	 * configuration
>-	 */
>-	if (mode == CXL_DECODER_PMEM && !resource_size(&cxlds->pmem_res)) {
>-		dev_dbg(dev, "no available pmem capacity\n");
>-		rc = -ENXIO;
>-		goto out;
>-	}
>-	if (mode == CXL_DECODER_RAM && !resource_size(&cxlds->ram_res)) {
>-		dev_dbg(dev, "no available ram capacity\n");
>-		rc = -ENXIO;
>-		goto out;
>-	}
>-
>	cxled->mode = mode;
>-	rc = 0;
>-out:
>-	up_write(&cxl_dpa_rwsem);
>-
>-	return rc;
>+	return 0;
> }
>
> int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size)
>
>--
>2.44.0
>
Fan Ni March 26, 2024, 4:25 p.m. UTC | #3
On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> cxl_dpa_set_mode() checks the mode for validity two times, once outside
> of the DPA RW semaphore and again within.  The function is not in a
> critical path.  Prior to Dynamic Capacity the extra check was not much
> of an issue.  The addition of DC modes increases the complexity of
> the check.
> 
> Simplify the mode check before adding the more complex DC modes.
> 
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> 

Reviewed-by: Fan Ni <fan.ni@samsung.com>

> ---
> Changes for v1:
> [iweiny: new patch]
> [Jonathan: based on getting rid of the loop in cxl_dpa_set_mode]
> [Jonathan: standardize on resource_size() == 0]
> ---
>  drivers/cxl/core/hdm.c | 45 ++++++++++++++++++---------------------------
>  1 file changed, 18 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> index 7d97790b893d..66b8419fd0c3 100644
> --- a/drivers/cxl/core/hdm.c
> +++ b/drivers/cxl/core/hdm.c
> @@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
>  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
>  	struct device *dev = &cxled->cxld.dev;
> -	int rc;
>  
> +	guard(rwsem_write)(&cxl_dpa_rwsem);
> +	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> +		return -EBUSY;
> +
> +	/*
> +	 * Check that the mode is supported by the current partition
> +	 * configuration
> +	 */
>  	switch (mode) {
>  	case CXL_DECODER_RAM:
> +		if (!resource_size(&cxlds->ram_res)) {
> +			dev_dbg(dev, "no available ram capacity\n");
> +			return -ENXIO;
> +		}
> +		break;
>  	case CXL_DECODER_PMEM:
> +		if (!resource_size(&cxlds->pmem_res)) {
> +			dev_dbg(dev, "no available pmem capacity\n");
> +			return -ENXIO;
> +		}
>  		break;
>  	default:
>  		dev_dbg(dev, "unsupported mode: %d\n", mode);
>  		return -EINVAL;
>  	}
>  
> -	down_write(&cxl_dpa_rwsem);
> -	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) {
> -		rc = -EBUSY;
> -		goto out;
> -	}
> -
> -	/*
> -	 * Only allow modes that are supported by the current partition
> -	 * configuration
> -	 */
> -	if (mode == CXL_DECODER_PMEM && !resource_size(&cxlds->pmem_res)) {
> -		dev_dbg(dev, "no available pmem capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -	if (mode == CXL_DECODER_RAM && !resource_size(&cxlds->ram_res)) {
> -		dev_dbg(dev, "no available ram capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -
>  	cxled->mode = mode;
> -	rc = 0;
> -out:
> -	up_write(&cxl_dpa_rwsem);
> -
> -	return rc;
> +	return 0;
>  }
>  
>  int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size)
> 
> -- 
> 2.44.0
>
Dave Jiang March 26, 2024, 5:46 p.m. UTC | #4
On 3/24/24 4:18 PM, Ira Weiny wrote:
> cxl_dpa_set_mode() checks the mode for validity two times, once outside
> of the DPA RW semaphore and again within.  The function is not in a
> critical path.  Prior to Dynamic Capacity the extra check was not much
> of an issue.  The addition of DC modes increases the complexity of
> the check.
> 
> Simplify the mode check before adding the more complex DC modes.

I would augment this by saying simplify "by using scope-based resource menagement".
> 
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>

Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> 
> ---
> Changes for v1:
> [iweiny: new patch]
> [Jonathan: based on getting rid of the loop in cxl_dpa_set_mode]
> [Jonathan: standardize on resource_size() == 0]
> ---
>  drivers/cxl/core/hdm.c | 45 ++++++++++++++++++---------------------------
>  1 file changed, 18 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> index 7d97790b893d..66b8419fd0c3 100644
> --- a/drivers/cxl/core/hdm.c
> +++ b/drivers/cxl/core/hdm.c
> @@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
>  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
>  	struct device *dev = &cxled->cxld.dev;
> -	int rc;
>  
> +	guard(rwsem_write)(&cxl_dpa_rwsem);
> +	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> +		return -EBUSY;
> +
> +	/*
> +	 * Check that the mode is supported by the current partition
> +	 * configuration
> +	 */
>  	switch (mode) {
>  	case CXL_DECODER_RAM:
> +		if (!resource_size(&cxlds->ram_res)) {
> +			dev_dbg(dev, "no available ram capacity\n");
> +			return -ENXIO;
> +		}
> +		break;
>  	case CXL_DECODER_PMEM:
> +		if (!resource_size(&cxlds->pmem_res)) {
> +			dev_dbg(dev, "no available pmem capacity\n");
> +			return -ENXIO;
> +		}
>  		break;
>  	default:
>  		dev_dbg(dev, "unsupported mode: %d\n", mode);
>  		return -EINVAL;
>  	}
>  
> -	down_write(&cxl_dpa_rwsem);
> -	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) {
> -		rc = -EBUSY;
> -		goto out;
> -	}
> -
> -	/*
> -	 * Only allow modes that are supported by the current partition
> -	 * configuration
> -	 */
> -	if (mode == CXL_DECODER_PMEM && !resource_size(&cxlds->pmem_res)) {
> -		dev_dbg(dev, "no available pmem capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -	if (mode == CXL_DECODER_RAM && !resource_size(&cxlds->ram_res)) {
> -		dev_dbg(dev, "no available ram capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -
>  	cxled->mode = mode;
> -	rc = 0;
> -out:
> -	up_write(&cxl_dpa_rwsem);
> -
> -	return rc;
> +	return 0;
>  }
>  
>  int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size)
>
Ira Weiny April 5, 2024, 7:21 p.m. UTC | #5
Dave Jiang wrote:
> 
> 
> On 3/24/24 4:18 PM, Ira Weiny wrote:
> > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > of the DPA RW semaphore and again within.  The function is not in a
> > critical path.  Prior to Dynamic Capacity the extra check was not much
> > of an issue.  The addition of DC modes increases the complexity of
> > the check.
> > 
> > Simplify the mode check before adding the more complex DC modes.
> 
> I would augment this by saying simplify "by using scope-based resource menagement".

However, using the guard cleanup is not really the simplification here.  It is
more about checking the mode a single time.

That said I will change this to:

Simplify the mode check and convert to use of a cleanup guard.

Ira
Dave Jiang April 6, 2024, 12:02 a.m. UTC | #6
On 4/5/24 12:21 PM, Ira Weiny wrote:
> Dave Jiang wrote:
>>
>>
>> On 3/24/24 4:18 PM, Ira Weiny wrote:
>>> cxl_dpa_set_mode() checks the mode for validity two times, once outside
>>> of the DPA RW semaphore and again within.  The function is not in a
>>> critical path.  Prior to Dynamic Capacity the extra check was not much
>>> of an issue.  The addition of DC modes increases the complexity of
>>> the check.
>>>
>>> Simplify the mode check before adding the more complex DC modes.
>>
>> I would augment this by saying simplify "by using scope-based resource menagement".
> 
> However, using the guard cleanup is not really the simplification here.  It is
> more about checking the mode a single time.
> 
> That said I will change this to:
> 
> Simplify the mode check and convert to use of a cleanup guard.

Ok

> 
> Ira
Alison Schofield April 9, 2024, 12:43 a.m. UTC | #7
On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> cxl_dpa_set_mode() checks the mode for validity two times, once outside
> of the DPA RW semaphore and again within.

Not true. It only checks mode once before the lock. It checks for
capacity after the lock. If it didn't check mode before the lock,
then unsupported modes would fall through.

> The function is not in a critical path.

Implying what here?  OK to check twice (even though it wasn't)
or OK to expand scope of locking.

> Prior to Dynamic Capacity the extra check was not much
> of an issue.  The addition of DC modes increases the complexity of
> the check.
> 
> Simplify the mode check before adding the more complex DC modes.
> 

The addition of the DC mode check doesn't seem complex.

Pardon my picking at the words, but if you'd like to refactor the
function, just say so. The final result is a bit more readable, but
also adding the DC mode checks without refactoring would read fine
also.

and...a bit spacing nit below -

> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> 
> ---
> Changes for v1:
> [iweiny: new patch]
> [Jonathan: based on getting rid of the loop in cxl_dpa_set_mode]
> [Jonathan: standardize on resource_size() == 0]
> ---
>  drivers/cxl/core/hdm.c | 45 ++++++++++++++++++---------------------------
>  1 file changed, 18 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> index 7d97790b893d..66b8419fd0c3 100644
> --- a/drivers/cxl/core/hdm.c
> +++ b/drivers/cxl/core/hdm.c
> @@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
>  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
>  	struct device *dev = &cxled->cxld.dev;
> -	int rc;
>  
> +	guard(rwsem_write)(&cxl_dpa_rwsem);
> +	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> +		return -EBUSY;
> +
> +	/*
> +	 * Check that the mode is supported by the current partition
> +	 * configuration
> +	 */
>  	switch (mode) {
>  	case CXL_DECODER_RAM:
> +		if (!resource_size(&cxlds->ram_res)) {
> +			dev_dbg(dev, "no available ram capacity\n");
> +			return -ENXIO;
> +		}
> +		break;
>  	case CXL_DECODER_PMEM:
> +		if (!resource_size(&cxlds->pmem_res)) {
> +			dev_dbg(dev, "no available pmem capacity\n");
> +			return -ENXIO;
> +		}
>  		break;
>  	default:
>  		dev_dbg(dev, "unsupported mode: %d\n", mode);
>  		return -EINVAL;
>  	}
>  

delete extra line

> -	down_write(&cxl_dpa_rwsem);
> -	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) {
> -		rc = -EBUSY;
> -		goto out;
> -	}
> -
> -	/*
> -	 * Only allow modes that are supported by the current partition
> -	 * configuration
> -	 */
> -	if (mode == CXL_DECODER_PMEM && !resource_size(&cxlds->pmem_res)) {
> -		dev_dbg(dev, "no available pmem capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -	if (mode == CXL_DECODER_RAM && !resource_size(&cxlds->ram_res)) {
> -		dev_dbg(dev, "no available ram capacity\n");
> -		rc = -ENXIO;
> -		goto out;
> -	}
> -
>  	cxled->mode = mode;
> -	rc = 0;
> -out:
> -	up_write(&cxl_dpa_rwsem);
> -
> -	return rc;
insert blank line
> +	return 0;
>  }
>  
>  int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size)
> 
> -- 
> 2.44.0
>
Ira Weiny May 3, 2024, 7:09 p.m. UTC | #8
Alison Schofield wrote:
> On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > of the DPA RW semaphore and again within.
> 
> Not true.

Sorry for not being clear.  It does check the mode 2x but not for
validity.  I'll clarify.

> It only checks mode once before the lock. It checks for
> capacity after the lock. If it didn't check mode before the lock,
> then unsupported modes would fall through.

But we can check the mode 1 time and either check the size or fail.

> 
> > The function is not in a critical path.
> 
> Implying what here?  OK to check twice (even though it wasn't)
> or OK to expand scope of locking.

Implying that checking the mode outside the lock is not required.

> 
> > Prior to Dynamic Capacity the extra check was not much
> > of an issue.  The addition of DC modes increases the complexity of
> > the check.
> > 
> > Simplify the mode check before adding the more complex DC modes.
> > 
> 
> The addition of the DC mode check doesn't seem complex.

It is if you have to check it 2 times.

> 
> Pardon my picking at the words, but if you'd like to refactor the
> function, just say so. The final result is a bit more readable, but
> also adding the DC mode checks without refactoring would read fine
> also.

When I added the DC mode to this function without this refactoring it was
quite a bit more code and ugly IMO.  So this cleanup helped.  If I were
not adding the DC code there would be much less reason to change this
function.


[snip]

> > diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> > index 7d97790b893d..66b8419fd0c3 100644
> > --- a/drivers/cxl/core/hdm.c
> > +++ b/drivers/cxl/core/hdm.c
> > @@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
> >  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> >  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
> >  	struct device *dev = &cxled->cxld.dev;
> > -	int rc;
> >  
> > +	guard(rwsem_write)(&cxl_dpa_rwsem);
> > +	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> > +		return -EBUSY;
> > +
> > +	/*
> > +	 * Check that the mode is supported by the current partition
> > +	 * configuration
> > +	 */
> >  	switch (mode) {
> >  	case CXL_DECODER_RAM:
> > +		if (!resource_size(&cxlds->ram_res)) {
> > +			dev_dbg(dev, "no available ram capacity\n");
> > +			return -ENXIO;
> > +		}
> > +		break;
> >  	case CXL_DECODER_PMEM:
> > +		if (!resource_size(&cxlds->pmem_res)) {
> > +			dev_dbg(dev, "no available pmem capacity\n");
> > +			return -ENXIO;
> > +		}
> >  		break;
> >  	default:
> >  		dev_dbg(dev, "unsupported mode: %d\n", mode);
> >  		return -EINVAL;
> >  	}
> >  
> 
> delete extra line

You don't like the space following the switch?

Ira
Alison Schofield May 3, 2024, 8:33 p.m. UTC | #9
On Fri, May 03, 2024 at 12:09:27PM -0700, Ira Weiny wrote:

snip

> 
> > > diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> > > index 7d97790b893d..66b8419fd0c3 100644
> > > --- a/drivers/cxl/core/hdm.c
> > > +++ b/drivers/cxl/core/hdm.c
> > > @@ -411,44 +411,35 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
> > >  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> > >  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
> > >  	struct device *dev = &cxled->cxld.dev;
> > > -	int rc;
> > >  
> > > +	guard(rwsem_write)(&cxl_dpa_rwsem);
> > > +	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> > > +		return -EBUSY;
> > > +
> > > +	/*
> > > +	 * Check that the mode is supported by the current partition
> > > +	 * configuration
> > > +	 */
> > >  	switch (mode) {
> > >  	case CXL_DECODER_RAM:
> > > +		if (!resource_size(&cxlds->ram_res)) {
> > > +			dev_dbg(dev, "no available ram capacity\n");
> > > +			return -ENXIO;
> > > +		}
> > > +		break;
> > >  	case CXL_DECODER_PMEM:
> > > +		if (!resource_size(&cxlds->pmem_res)) {
> > > +			dev_dbg(dev, "no available pmem capacity\n");
> > > +			return -ENXIO;
> > > +		}
> > >  		break;
> > >  	default:
> > >  		dev_dbg(dev, "unsupported mode: %d\n", mode);
> > >  		return -EINVAL;
> > >  	}
> > >  
> > 
> > delete extra line
> 
> You don't like the space following the switch?
> 
> Ira

Sorry - looks fine. Ignore my comment.
Dan Williams May 4, 2024, 1:19 a.m. UTC | #10
Ira Weiny wrote:
> Alison Schofield wrote:
> > On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> > > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > > of the DPA RW semaphore and again within.
> > 
> > Not true.
> 
> Sorry for not being clear.  It does check the mode 2x but not for
> validity.  I'll clarify.
> 
> > It only checks mode once before the lock. It checks for
> > capacity after the lock. If it didn't check mode before the lock,
> > then unsupported modes would fall through.
> 
> But we can check the mode 1 time and either check the size or fail.
> 
> > 
> > > The function is not in a critical path.
> > 
> > Implying what here?  OK to check twice (even though it wasn't)
> > or OK to expand scope of locking.
> 
> Implying that checking the mode outside the lock is not required.

The @mode check outside the lock is there to taking the lock when not
necessary because the passed in mode is already bogus.

The lock is about making sure the write of cxled->mode relative to the
state of the dpa partitions is an atomic check-and-set.

So this makes the function unconditionally take the lock when it might
be bogus to do so. The value of reorganizing this is questionable.
Dan Williams May 4, 2024, 4:13 a.m. UTC | #11
Ira Weiny wrote:
> Alison Schofield wrote:
> > On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> > > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > > of the DPA RW semaphore and again within.
> > 
> > Not true.
> 
> Sorry for not being clear.  It does check the mode 2x but not for
> validity.  I'll clarify.
> 
> > It only checks mode once before the lock. It checks for
> > capacity after the lock. If it didn't check mode before the lock,
> > then unsupported modes would fall through.
> 
> But we can check the mode 1 time and either check the size or fail.
> 
> > 
> > > The function is not in a critical path.
> > 
> > Implying what here?  OK to check twice (even though it wasn't)
> > or OK to expand scope of locking.
> 
> Implying that checking the mode outside the lock is not required.
> 
> > 
> > > Prior to Dynamic Capacity the extra check was not much
> > > of an issue.  The addition of DC modes increases the complexity of
> > > the check.
> > > 
> > > Simplify the mode check before adding the more complex DC modes.
> > > 
> > 
> > The addition of the DC mode check doesn't seem complex.
> 
> It is if you have to check it 2 times.
> 
> > 
> > Pardon my picking at the words, but if you'd like to refactor the
> > function, just say so. The final result is a bit more readable, but
> > also adding the DC mode checks without refactoring would read fine
> > also.
> 
> When I added the DC mode to this function without this refactoring it was
> quite a bit more code and ugly IMO.  So this cleanup helped.  If I were
> not adding the DC code there would be much less reason to change this
> function.

Where did the "quite a bit more code" come from? A change that moves
unnecessary code under a lock and is larger than just incrementally
extending the status quo does not feel like a cleanup.

diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
index 7d97790b893d..0dc886bc22c6 100644
--- a/drivers/cxl/core/hdm.c
+++ b/drivers/cxl/core/hdm.c
@@ -411,11 +411,12 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
        struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
        struct cxl_dev_state *cxlds = cxlmd->cxlds;
        struct device *dev = &cxled->cxld.dev;
-       int rc;
+       int rc, dcd;
 
        switch (mode) {
        case CXL_DECODER_RAM:
        case CXL_DECODER_PMEM:
+       case CXL_DECODER_DC0 ... CXL_DECODER_DC7:
                break;
        default:
                dev_dbg(dev, "unsupported mode: %d\n", mode);
@@ -442,6 +443,11 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
                rc = -ENXIO;
                goto out;
        }
+       dcd = dc_mode_to_region_index(mode);
+       if (resource_size(&cxlds->dc_res[dcd]) == 0) {
+               dev_dbg(dev, "no available dynamic capacity\n");
+               goto out;
+       }
 
        cxled->mode = mode;
        rc = 0;
Ira Weiny May 6, 2024, 3:46 a.m. UTC | #12
Dan Williams wrote:
> Ira Weiny wrote:
> > Alison Schofield wrote:
> > > On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> > > > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > > > of the DPA RW semaphore and again within.
> > > 
> > > Not true.
> > 
> > Sorry for not being clear.  It does check the mode 2x but not for
> > validity.  I'll clarify.
> > 
> > > It only checks mode once before the lock. It checks for
> > > capacity after the lock. If it didn't check mode before the lock,
> > > then unsupported modes would fall through.
> > 
> > But we can check the mode 1 time and either check the size or fail.
> > 
> > > 
> > > > The function is not in a critical path.
> > > 
> > > Implying what here?  OK to check twice (even though it wasn't)
> > > or OK to expand scope of locking.
> > 
> > Implying that checking the mode outside the lock is not required.
> > 
> > > 
> > > > Prior to Dynamic Capacity the extra check was not much
> > > > of an issue.  The addition of DC modes increases the complexity of
> > > > the check.
> > > > 
> > > > Simplify the mode check before adding the more complex DC modes.
> > > > 
> > > 
> > > The addition of the DC mode check doesn't seem complex.
> > 
> > It is if you have to check it 2 times.
> > 
> > > 
> > > Pardon my picking at the words, but if you'd like to refactor the
> > > function, just say so. The final result is a bit more readable, but
> > > also adding the DC mode checks without refactoring would read fine
> > > also.
> > 
> > When I added the DC mode to this function without this refactoring it was
> > quite a bit more code and ugly IMO.  So this cleanup helped.  If I were
> > not adding the DC code there would be much less reason to change this
> > function.
> 
> Where did the "quite a bit more code" come from? A change that moves
> unnecessary code under a lock and is larger than just incrementally
> extending the status quo does not feel like a cleanup.

I'll drop the patch.

Ira

> 
> diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> index 7d97790b893d..0dc886bc22c6 100644
> --- a/drivers/cxl/core/hdm.c
> +++ b/drivers/cxl/core/hdm.c
> @@ -411,11 +411,12 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>         struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
>         struct cxl_dev_state *cxlds = cxlmd->cxlds;
>         struct device *dev = &cxled->cxld.dev;
> -       int rc;
> +       int rc, dcd;
>  
>         switch (mode) {
>         case CXL_DECODER_RAM:
>         case CXL_DECODER_PMEM:
> +       case CXL_DECODER_DC0 ... CXL_DECODER_DC7:
>                 break;
>         default:
>                 dev_dbg(dev, "unsupported mode: %d\n", mode);
> @@ -442,6 +443,11 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
>                 rc = -ENXIO;
>                 goto out;
>         }
> +       dcd = dc_mode_to_region_index(mode);
> +       if (resource_size(&cxlds->dc_res[dcd]) == 0) {
> +               dev_dbg(dev, "no available dynamic capacity\n");
> +               goto out;
> +       }
>  
>         cxled->mode = mode;
>         rc = 0;
Ira Weiny May 6, 2024, 4:06 a.m. UTC | #13
Dan Williams wrote:
> Ira Weiny wrote:
> > Alison Schofield wrote:
> > > On Sun, Mar 24, 2024 at 04:18:08PM -0700, Ira Weiny wrote:
> > > > cxl_dpa_set_mode() checks the mode for validity two times, once outside
> > > > of the DPA RW semaphore and again within.
> > > 
> > > Not true.
> > 
> > Sorry for not being clear.  It does check the mode 2x but not for
> > validity.  I'll clarify.
> > 
> > > It only checks mode once before the lock. It checks for
> > > capacity after the lock. If it didn't check mode before the lock,
> > > then unsupported modes would fall through.
> > 
> > But we can check the mode 1 time and either check the size or fail.
> > 
> > > 
> > > > The function is not in a critical path.
> > > 
> > > Implying what here?  OK to check twice (even though it wasn't)
> > > or OK to expand scope of locking.
> > 
> > Implying that checking the mode outside the lock is not required.
> 
> The @mode check outside the lock is there to taking the lock when not
> necessary because the passed in mode is already bogus.

Sorry I meant to say 'is required'.

> 
> The lock is about making sure the write of cxled->mode relative to the
> state of the dpa partitions is an atomic check-and-set.
> 
> So this makes the function unconditionally take the lock when it might
> be bogus to do so. The value of reorganizing this is questionable.

Why would it be bogus?  I don't see that.  Regardless I dropped the patch as it
is not worth spending more time on.  There are bigger issues to resolve with
this series.

Ira
diff mbox series

Patch

diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
index 7d97790b893d..66b8419fd0c3 100644
--- a/drivers/cxl/core/hdm.c
+++ b/drivers/cxl/core/hdm.c
@@ -411,44 +411,35 @@  int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
 	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
 	struct device *dev = &cxled->cxld.dev;
-	int rc;
 
+	guard(rwsem_write)(&cxl_dpa_rwsem);
+	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
+		return -EBUSY;
+
+	/*
+	 * Check that the mode is supported by the current partition
+	 * configuration
+	 */
 	switch (mode) {
 	case CXL_DECODER_RAM:
+		if (!resource_size(&cxlds->ram_res)) {
+			dev_dbg(dev, "no available ram capacity\n");
+			return -ENXIO;
+		}
+		break;
 	case CXL_DECODER_PMEM:
+		if (!resource_size(&cxlds->pmem_res)) {
+			dev_dbg(dev, "no available pmem capacity\n");
+			return -ENXIO;
+		}
 		break;
 	default:
 		dev_dbg(dev, "unsupported mode: %d\n", mode);
 		return -EINVAL;
 	}
 
-	down_write(&cxl_dpa_rwsem);
-	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) {
-		rc = -EBUSY;
-		goto out;
-	}
-
-	/*
-	 * Only allow modes that are supported by the current partition
-	 * configuration
-	 */
-	if (mode == CXL_DECODER_PMEM && !resource_size(&cxlds->pmem_res)) {
-		dev_dbg(dev, "no available pmem capacity\n");
-		rc = -ENXIO;
-		goto out;
-	}
-	if (mode == CXL_DECODER_RAM && !resource_size(&cxlds->ram_res)) {
-		dev_dbg(dev, "no available ram capacity\n");
-		rc = -ENXIO;
-		goto out;
-	}
-
 	cxled->mode = mode;
-	rc = 0;
-out:
-	up_write(&cxl_dpa_rwsem);
-
-	return rc;
+	return 0;
 }
 
 int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size)