diff mbox series

[v3] libnvdimm/region: Update nvdimm_has_flush() to handle explicit 'flush' callbacks

Message ID 20210408104622.943843-1-vaibhav@linux.ibm.com (mailing list archive)
State New
Headers show
Series [v3] libnvdimm/region: Update nvdimm_has_flush() to handle explicit 'flush' callbacks | expand

Commit Message

Vaibhav Jain April 8, 2021, 10:46 a.m. UTC
In case a platform doesn't provide explicit flush-hints but provides an
explicit flush callback, then nvdimm_has_flush() still returns '0'
indicating that writes do not require flushing. This happens on PPC64
with patch at [1] applied, where 'deep_flush' of a region was denied
even though an explicit flush function was provided.

Similar problem is also seen with virtio-pmem where the 'deep_flush'
sysfs attribute is not visible as in absence of any registered nvdimm,
'nd_region->ndr_mappings == 0'.

Fix this by updating nvdimm_has_flush() adding a condition to
nvdimm_has_flush() to test if a 'region->flush' callback is
assigned. Also remove explicit test for 'nd_region->ndr_mapping' since
regions may need 'flush' without any explicit mappings as in case of
virtio-pmem.

References:
[1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall"
https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87

Cc: <stable@vger.kernel.org>
Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support")
Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com>
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
Changelog:

v3:
* Removed the test for ND_REGION_SYNC to handle case where a
  synchronous region still wants to expose a deep-flush function.
  [ Aneesh ]
* Updated patch title and description from previous patch
  https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com

v2:
* Added the fixes tag and addressed the patch to stable tree [ Aneesh ]
* Updated patch description to address the virtio-pmem case.
* Removed test for 'nd_region->ndr_mappings' from beginning of
  nvdimm_has_flush() to handle the virtio-pmem case.
---
 drivers/nvdimm/region_devs.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

Comments

Aneesh Kumar K.V April 12, 2021, 6:52 a.m. UTC | #1
Vaibhav Jain <vaibhav@linux.ibm.com> writes:

> In case a platform doesn't provide explicit flush-hints but provides an
> explicit flush callback, then nvdimm_has_flush() still returns '0'
> indicating that writes do not require flushing. This happens on PPC64
> with patch at [1] applied, where 'deep_flush' of a region was denied
> even though an explicit flush function was provided.
>
> Similar problem is also seen with virtio-pmem where the 'deep_flush'
> sysfs attribute is not visible as in absence of any registered nvdimm,
> 'nd_region->ndr_mappings == 0'.
>
> Fix this by updating nvdimm_has_flush() adding a condition to
> nvdimm_has_flush() to test if a 'region->flush' callback is
> assigned. Also remove explicit test for 'nd_region->ndr_mapping' since
> regions may need 'flush' without any explicit mappings as in case of
> virtio-pmem.
>
> References:
> [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall"
> https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87
>

Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>

> Cc: <stable@vger.kernel.org>
> Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support")
> Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> Changelog:
>
> v3:
> * Removed the test for ND_REGION_SYNC to handle case where a
>   synchronous region still wants to expose a deep-flush function.
>   [ Aneesh ]
> * Updated patch title and description from previous patch
>   https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com
>
> v2:
> * Added the fixes tag and addressed the patch to stable tree [ Aneesh ]
> * Updated patch description to address the virtio-pmem case.
> * Removed test for 'nd_region->ndr_mappings' from beginning of
>   nvdimm_has_flush() to handle the virtio-pmem case.
> ---
>  drivers/nvdimm/region_devs.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
> index ef23119db574..c4b17bdd527f 100644
> --- a/drivers/nvdimm/region_devs.c
> +++ b/drivers/nvdimm/region_devs.c
> @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region)
>  {
>  	int i;
>  
> -	/* no nvdimm or pmem api == flushing capability unknown */
> -	if (nd_region->ndr_mappings == 0
> -			|| !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
> +	/* no pmem api == flushing capability unknown */
> +	if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
>  		return -ENXIO;
>  
> +	/* Test if an explicit flush function is defined */
> +	if (nd_region->flush)
> +		return 1;
> +
> +	/* Test if any flush hints for the region are available */
>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>  		struct nvdimm *nvdimm = nd_mapping->nvdimm;
> @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region)
>  	}
>  
>  	/*
> -	 * The platform defines dimm devices without hints, assume
> -	 * platform persistence mechanism like ADR
> +	 * The platform defines dimm devices without hints nor explicit flush,
> +	 * assume platform persistence mechanism like ADR
>  	 */
>  	return 0;
>  }
> -- 
> 2.30.2
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Pankaj Gupta April 12, 2021, 7:52 a.m. UTC | #2
Hi Vaibhav,

> In case a platform doesn't provide explicit flush-hints but provides an
> explicit flush callback, then nvdimm_has_flush() still returns '0'
> indicating that writes do not require flushing. This happens on PPC64
> with patch at [1] applied, where 'deep_flush' of a region was denied
> even though an explicit flush function was provided.
>
> Similar problem is also seen with virtio-pmem where the 'deep_flush'
> sysfs attribute is not visible as in absence of any registered nvdimm,
> 'nd_region->ndr_mappings == 0'.

In case of async flush callback, do we still need "deep_flush" ?

Thanks,
Pankaj
>
> Fix this by updating nvdimm_has_flush() adding a condition to
> nvdimm_has_flush() to test if a 'region->flush' callback is
> assigned. Also remove explicit test for 'nd_region->ndr_mapping' since
> regions may need 'flush' without any explicit mappings as in case of
> virtio-pmem.
>
> References:
> [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall"
> https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87
>
> Cc: <stable@vger.kernel.org>
> Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support")
> Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> Changelog:
>
> v3:
> * Removed the test for ND_REGION_SYNC to handle case where a
>   synchronous region still wants to expose a deep-flush function.
>   [ Aneesh ]
> * Updated patch title and description from previous patch
>   https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com
>
> v2:
> * Added the fixes tag and addressed the patch to stable tree [ Aneesh ]
> * Updated patch description to address the virtio-pmem case.
> * Removed test for 'nd_region->ndr_mappings' from beginning of
>   nvdimm_has_flush() to handle the virtio-pmem case.
> ---
>  drivers/nvdimm/region_devs.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
> index ef23119db574..c4b17bdd527f 100644
> --- a/drivers/nvdimm/region_devs.c
> +++ b/drivers/nvdimm/region_devs.c
> @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region)
>  {
>         int i;
>
> -       /* no nvdimm or pmem api == flushing capability unknown */
> -       if (nd_region->ndr_mappings == 0
> -                       || !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
> +       /* no pmem api == flushing capability unknown */
> +       if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
>                 return -ENXIO;
>
> +       /* Test if an explicit flush function is defined */
> +       if (nd_region->flush)
> +               return 1;
> +
> +       /* Test if any flush hints for the region are available */
>         for (i = 0; i < nd_region->ndr_mappings; i++) {
>                 struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>                 struct nvdimm *nvdimm = nd_mapping->nvdimm;
> @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region)
>         }
>
>         /*
> -        * The platform defines dimm devices without hints, assume
> -        * platform persistence mechanism like ADR
> +        * The platform defines dimm devices without hints nor explicit flush,
> +        * assume platform persistence mechanism like ADR
>          */
>         return 0;
>  }
> --
> 2.30.2
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Vaibhav Jain April 13, 2021, 2:53 p.m. UTC | #3
Thanks Pankaj for looking into this patch,

Pankaj Gupta <pankaj.gupta.linux@gmail.com> writes:

> Hi Vaibhav,
>
>> In case a platform doesn't provide explicit flush-hints but provides an
>> explicit flush callback, then nvdimm_has_flush() still returns '0'
>> indicating that writes do not require flushing. This happens on PPC64
>> with patch at [1] applied, where 'deep_flush' of a region was denied
>> even though an explicit flush function was provided.
>>
>> Similar problem is also seen with virtio-pmem where the 'deep_flush'
>> sysfs attribute is not visible as in absence of any registered nvdimm,
>> 'nd_region->ndr_mappings == 0'.
>
> In case of async flush callback, do we still need "deep_flush" ?

'deep_flush' in libnvdimm (specifically 'deep_flush_store()')
anyways resorts to calling 'async_flush' callback if its defined. Which
makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would
ensure that writes to pmem are now durable even if there is a sudden
power loss before cpu caches are flushed.

On non-nfit architectures the 'async_flush' callback should provide such
a guarantee, which can be triggered by user-space writing to the
'deep_flush' sysfs attr.

In absence of 'deep_flush' sysfs attr not sure how else can user-space
forcibly trigger async_flush callback for dev-dax char devices.

<snip>
Pankaj Gupta April 14, 2021, 10:22 a.m. UTC | #4
> >> In case a platform doesn't provide explicit flush-hints but provides an
> >> explicit flush callback, then nvdimm_has_flush() still returns '0'
> >> indicating that writes do not require flushing. This happens on PPC64
> >> with patch at [1] applied, where 'deep_flush' of a region was denied
> >> even though an explicit flush function was provided.
> >>
> >> Similar problem is also seen with virtio-pmem where the 'deep_flush'
> >> sysfs attribute is not visible as in absence of any registered nvdimm,
> >> 'nd_region->ndr_mappings == 0'.
> >
> > In case of async flush callback, do we still need "deep_flush" ?
>
> 'deep_flush' in libnvdimm (specifically 'deep_flush_store()')
> anyways resorts to calling 'async_flush' callback if its defined. Which
> makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would
> ensure that writes to pmem are now durable even if there is a sudden
> power loss before cpu caches are flushed.
>
> On non-nfit architectures the 'async_flush' callback should provide such
> a guarantee, which can be triggered by user-space writing to the
> 'deep_flush' sysfs attr.
>
> In absence of 'deep_flush' sysfs attr not sure how else can user-space
> forcibly trigger async_flush callback for dev-dax char devices.

O.k. that means for filesystem DAX deep_flush is alternative to
fsync/msync call.

I still have to dig deeper to understand more about "QUEUE_FLAG_FUA" flag &
why I was seeing REQ_FUA with virtio-pmem when doing fsync if its not enabled
in function "blk_queue_write_cache". But this is for my understanding.

Overall patch looks good to me and it looks to solve (not tested
though) the warning for
virtio-pmem as well.

Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>



Thanks,
Pankaj
diff mbox series

Patch

diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index ef23119db574..c4b17bdd527f 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -1234,11 +1234,15 @@  int nvdimm_has_flush(struct nd_region *nd_region)
 {
 	int i;
 
-	/* no nvdimm or pmem api == flushing capability unknown */
-	if (nd_region->ndr_mappings == 0
-			|| !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
+	/* no pmem api == flushing capability unknown */
+	if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
 		return -ENXIO;
 
+	/* Test if an explicit flush function is defined */
+	if (nd_region->flush)
+		return 1;
+
+	/* Test if any flush hints for the region are available */
 	for (i = 0; i < nd_region->ndr_mappings; i++) {
 		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
 		struct nvdimm *nvdimm = nd_mapping->nvdimm;
@@ -1249,8 +1253,8 @@  int nvdimm_has_flush(struct nd_region *nd_region)
 	}
 
 	/*
-	 * The platform defines dimm devices without hints, assume
-	 * platform persistence mechanism like ADR
+	 * The platform defines dimm devices without hints nor explicit flush,
+	 * assume platform persistence mechanism like ADR
 	 */
 	return 0;
 }