| Message ID | 20210408104622.943843-1-vaibhav@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v3] libnvdimm/region: Update nvdimm_has_flush() to handle explicit 'flush' callbacks | expand |
Vaibhav Jain <vaibhav@linux.ibm.com> writes: > In case a platform doesn't provide explicit flush-hints but provides an > explicit flush callback, then nvdimm_has_flush() still returns '0' > indicating that writes do not require flushing. This happens on PPC64 > with patch at [1] applied, where 'deep_flush' of a region was denied > even though an explicit flush function was provided. > > Similar problem is also seen with virtio-pmem where the 'deep_flush' > sysfs attribute is not visible as in absence of any registered nvdimm, > 'nd_region->ndr_mappings == 0'. > > Fix this by updating nvdimm_has_flush() adding a condition to > nvdimm_has_flush() to test if a 'region->flush' callback is > assigned. Also remove explicit test for 'nd_region->ndr_mapping' since > regions may need 'flush' without any explicit mappings as in case of > virtio-pmem. > > References: > [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" > https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87 > Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> > Cc: <stable@vger.kernel.org> > Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") > Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com> > Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> > --- > Changelog: > > v3: > * Removed the test for ND_REGION_SYNC to handle case where a > synchronous region still wants to expose a deep-flush function. > [ Aneesh ] > * Updated patch title and description from previous patch > https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com > > v2: > * Added the fixes tag and addressed the patch to stable tree [ Aneesh ] > * Updated patch description to address the virtio-pmem case. > * Removed test for 'nd_region->ndr_mappings' from beginning of > nvdimm_has_flush() to handle the virtio-pmem case. > --- > drivers/nvdimm/region_devs.c | 14 +++++++++----- > 1 file changed, 9 insertions(+), 5 deletions(-) > > diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c > index ef23119db574..c4b17bdd527f 100644 > --- a/drivers/nvdimm/region_devs.c > +++ b/drivers/nvdimm/region_devs.c > @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) > { > int i; > > - /* no nvdimm or pmem api == flushing capability unknown */ > - if (nd_region->ndr_mappings == 0 > - || !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) > + /* no pmem api == flushing capability unknown */ > + if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) > return -ENXIO; > > + /* Test if an explicit flush function is defined */ > + if (nd_region->flush) > + return 1; > + > + /* Test if any flush hints for the region are available */ > for (i = 0; i < nd_region->ndr_mappings; i++) { > struct nd_mapping *nd_mapping = &nd_region->mapping[i]; > struct nvdimm *nvdimm = nd_mapping->nvdimm; > @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) > } > > /* > - * The platform defines dimm devices without hints, assume > - * platform persistence mechanism like ADR > + * The platform defines dimm devices without hints nor explicit flush, > + * assume platform persistence mechanism like ADR > */ > return 0; > } > -- > 2.30.2 > _______________________________________________ > Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org > To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Hi Vaibhav, > In case a platform doesn't provide explicit flush-hints but provides an > explicit flush callback, then nvdimm_has_flush() still returns '0' > indicating that writes do not require flushing. This happens on PPC64 > with patch at [1] applied, where 'deep_flush' of a region was denied > even though an explicit flush function was provided. > > Similar problem is also seen with virtio-pmem where the 'deep_flush' > sysfs attribute is not visible as in absence of any registered nvdimm, > 'nd_region->ndr_mappings == 0'. In case of async flush callback, do we still need "deep_flush" ? Thanks, Pankaj > > Fix this by updating nvdimm_has_flush() adding a condition to > nvdimm_has_flush() to test if a 'region->flush' callback is > assigned. Also remove explicit test for 'nd_region->ndr_mapping' since > regions may need 'flush' without any explicit mappings as in case of > virtio-pmem. > > References: > [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" > https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87 > > Cc: <stable@vger.kernel.org> > Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") > Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com> > Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> > --- > Changelog: > > v3: > * Removed the test for ND_REGION_SYNC to handle case where a > synchronous region still wants to expose a deep-flush function. > [ Aneesh ] > * Updated patch title and description from previous patch > https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com > > v2: > * Added the fixes tag and addressed the patch to stable tree [ Aneesh ] > * Updated patch description to address the virtio-pmem case. > * Removed test for 'nd_region->ndr_mappings' from beginning of > nvdimm_has_flush() to handle the virtio-pmem case. > --- > drivers/nvdimm/region_devs.c | 14 +++++++++----- > 1 file changed, 9 insertions(+), 5 deletions(-) > > diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c > index ef23119db574..c4b17bdd527f 100644 > --- a/drivers/nvdimm/region_devs.c > +++ b/drivers/nvdimm/region_devs.c > @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) > { > int i; > > - /* no nvdimm or pmem api == flushing capability unknown */ > - if (nd_region->ndr_mappings == 0 > - || !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) > + /* no pmem api == flushing capability unknown */ > + if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) > return -ENXIO; > > + /* Test if an explicit flush function is defined */ > + if (nd_region->flush) > + return 1; > + > + /* Test if any flush hints for the region are available */ > for (i = 0; i < nd_region->ndr_mappings; i++) { > struct nd_mapping *nd_mapping = &nd_region->mapping[i]; > struct nvdimm *nvdimm = nd_mapping->nvdimm; > @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) > } > > /* > - * The platform defines dimm devices without hints, assume > - * platform persistence mechanism like ADR > + * The platform defines dimm devices without hints nor explicit flush, > + * assume platform persistence mechanism like ADR > */ > return 0; > } > -- > 2.30.2 > _______________________________________________ > Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org > To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Thanks Pankaj for looking into this patch, Pankaj Gupta <pankaj.gupta.linux@gmail.com> writes: > Hi Vaibhav, > >> In case a platform doesn't provide explicit flush-hints but provides an >> explicit flush callback, then nvdimm_has_flush() still returns '0' >> indicating that writes do not require flushing. This happens on PPC64 >> with patch at [1] applied, where 'deep_flush' of a region was denied >> even though an explicit flush function was provided. >> >> Similar problem is also seen with virtio-pmem where the 'deep_flush' >> sysfs attribute is not visible as in absence of any registered nvdimm, >> 'nd_region->ndr_mappings == 0'. > > In case of async flush callback, do we still need "deep_flush" ? 'deep_flush' in libnvdimm (specifically 'deep_flush_store()') anyways resorts to calling 'async_flush' callback if its defined. Which makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would ensure that writes to pmem are now durable even if there is a sudden power loss before cpu caches are flushed. On non-nfit architectures the 'async_flush' callback should provide such a guarantee, which can be triggered by user-space writing to the 'deep_flush' sysfs attr. In absence of 'deep_flush' sysfs attr not sure how else can user-space forcibly trigger async_flush callback for dev-dax char devices. <snip>
> >> In case a platform doesn't provide explicit flush-hints but provides an > >> explicit flush callback, then nvdimm_has_flush() still returns '0' > >> indicating that writes do not require flushing. This happens on PPC64 > >> with patch at [1] applied, where 'deep_flush' of a region was denied > >> even though an explicit flush function was provided. > >> > >> Similar problem is also seen with virtio-pmem where the 'deep_flush' > >> sysfs attribute is not visible as in absence of any registered nvdimm, > >> 'nd_region->ndr_mappings == 0'. > > > > In case of async flush callback, do we still need "deep_flush" ? > > 'deep_flush' in libnvdimm (specifically 'deep_flush_store()') > anyways resorts to calling 'async_flush' callback if its defined. Which > makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would > ensure that writes to pmem are now durable even if there is a sudden > power loss before cpu caches are flushed. > > On non-nfit architectures the 'async_flush' callback should provide such > a guarantee, which can be triggered by user-space writing to the > 'deep_flush' sysfs attr. > > In absence of 'deep_flush' sysfs attr not sure how else can user-space > forcibly trigger async_flush callback for dev-dax char devices. O.k. that means for filesystem DAX deep_flush is alternative to fsync/msync call. I still have to dig deeper to understand more about "QUEUE_FLAG_FUA" flag & why I was seeing REQ_FUA with virtio-pmem when doing fsync if its not enabled in function "blk_queue_write_cache". But this is for my understanding. Overall patch looks good to me and it looks to solve (not tested though) the warning for virtio-pmem as well. Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com> Thanks, Pankaj
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index ef23119db574..c4b17bdd527f 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) { int i; - /* no nvdimm or pmem api == flushing capability unknown */ - if (nd_region->ndr_mappings == 0 - || !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) + /* no pmem api == flushing capability unknown */ + if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) return -ENXIO; + /* Test if an explicit flush function is defined */ + if (nd_region->flush) + return 1; + + /* Test if any flush hints for the region are available */ for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; struct nvdimm *nvdimm = nd_mapping->nvdimm; @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) } /* - * The platform defines dimm devices without hints, assume - * platform persistence mechanism like ADR + * The platform defines dimm devices without hints nor explicit flush, + * assume platform persistence mechanism like ADR */ return 0; }
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided. Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'. Fix this by updating nvdimm_has_flush() adding a condition to nvdimm_has_flush() to test if a 'region->flush' callback is assigned. Also remove explicit test for 'nd_region->ndr_mapping' since regions may need 'flush' without any explicit mappings as in case of virtio-pmem. References: [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stgit@e1fbed493c87 Cc: <stable@vger.kernel.org> Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") Reported-by: Shivaprasad G Bhat <sbhat@linux.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> --- Changelog: v3: * Removed the test for ND_REGION_SYNC to handle case where a synchronous region still wants to expose a deep-flush function. [ Aneesh ] * Updated patch title and description from previous patch https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@linux.ibm.com v2: * Added the fixes tag and addressed the patch to stable tree [ Aneesh ] * Updated patch description to address the virtio-pmem case. * Removed test for 'nd_region->ndr_mappings' from beginning of nvdimm_has_flush() to handle the virtio-pmem case. --- drivers/nvdimm/region_devs.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)