| Message ID | 20210811170224.42837-1-colyli@suse.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | bcache: support NVDIMM for journaling | expand |
Hi Jens, Could you please consider take the v12 series for Linux v5.15 merge window? In this version the full pointer in the on-media data structures are modified to per-namespace offset, and all previous review comments are fixed. There is no more comments for 4 hours, and this series survives in my smoking test for 24+ hours, as an EXPERIMENTAL code the current status is fine IMHO. Thanks in advance. Coly Li On 8/12/21 1:02 AM, Coly Li wrote: > This is the v12 effort for supporting NVDIMM for bcache journal (some > versions may not posted with version numbers). > > The major change of this version is the full pointer of on-media data > structure is replaced by per-namespace offset. Now a pointer address is > calculated by namespace base mapping address + per-namespace offset. > The code logic is same as previous version, all changes are only related > to the base+offset style pointer replacement. > > The nvm-pages allocator is a buddy-like allocator, which allocates size > in power-of-2 pages from the NVDIMM namespace. User space tool 'bcache' > has a new added '-M' option to format a NVDIMM namespace and register it > via sysfs interface as a bcache meta device. The nvm-pages kernel code > does a DAX mapping to map the whole namespace into system's memory > address range, and allocating the pages to requestion like typical buddy > allocator does. The major difference is nvm-pages allocator maintains > the pages allocated to each requester by an allocation list which stored > on NVDIMM too. Allocation list of different requester is tracked by a > pre-defined UUID, all the pages tracked in all allocation lists are > treated as allocated busy pages and won't be initialized into buddy > system after the system reboot. > > The bcache journal code may request a block of power-of-2 size pages > from the nvm-pages allocator, normally it is a range of 256MB or 512MB > continuous pages range. During meta data journaling, the in-memory jsets > go into the calculated nvdimm pages location by kernel memcpy routine. > So the journaling I/Os won't go into block device (e.g. SSD) anymore, > the write and read for journal jsets happen on NVDIMM. > > Intel developers Jianpeng Ma and Qiaowei Ren compose the initial code of > nvm-pages, the related patches are, > - bcache: initialize the nvm pages allocator > - bcache: initialization of the buddy > - bcache: bch_nvm_alloc_pages() of the buddy > - bcache: bch_nvm_free_pages() of the buddy > - bcache: get recs list head for allocated pages by specific uuid > All the code depends on Linux libnvdimm and dax drivers, the bcache nvm- > pages allocator can be treated as user of these two drivers. > > I modify the bcache code to recognize the nvm meta device feature, > initialize journal on NVDIMM, and do journal I/Os on NVDIMM in the > following patches, > - bcache: add initial data structures for nvm pages > - bcache: use bucket index to set GC_MARK_METADATA for journal buckets > in bch_btree_gc_finish() > - bcache: add BCH_FEATURE_INCOMPAT_NVDIMM_META into incompat feature set > - bcache: initialize bcache journal for NVDIMM meta device > - bcache: support storing bcache journal into NVDIMM meta device > - bcache: read jset from NVDIMM pages for journal replay > - bcache: add sysfs interface register_nvdimm_meta to register NVDIMM > meta device > > In this series, all previously addressed issue via code reviews are all > fixed. And all known issue during testing are fixed. The code survives > from 24+ hours smoking and I/O pressure testing among many reboots, it > works well as expected. > > All the code is EXPERIMENTAL, they won't be enabled by default until we > feel the NVDIMM support is completed and stable. > > Although there are some experts helped to review the code logic, but we > do appreciate if more people may help to review the code. It is quite > common that bcache patches don't have enough code reviewer, but this > time I do need help for more review or comments on this series. > > Thanks in advance. > > Coly Li > --- > > Coly Li (7): > bcache: add initial data structures for nvm pages > bcache: use bucket index to set GC_MARK_METADATA for journal buckets > in bch_btree_gc_finish() > bcache: add BCH_FEATURE_INCOMPAT_NVDIMM_META into incompat feature set > bcache: initialize bcache journal for NVDIMM meta device > bcache: support storing bcache journal into NVDIMM meta device > bcache: read jset from NVDIMM pages for journal replay > bcache: add sysfs interface register_nvdimm_meta to register NVDIMM > meta device > > Jianpeng Ma (5): > bcache: initialize the nvm pages allocator > bcache: initialization of the buddy > bcache: bch_nvmpg_alloc_pages() of the buddy > bcache: bch_nvmpg_free_pages() of the buddy allocator > bcache: get recs list head for allocated pages by specific uuid > > drivers/md/bcache/Kconfig | 10 + > drivers/md/bcache/Makefile | 1 + > drivers/md/bcache/btree.c | 6 +- > drivers/md/bcache/features.h | 9 + > drivers/md/bcache/journal.c | 325 +++++++++-- > drivers/md/bcache/journal.h | 2 +- > drivers/md/bcache/nvm-pages.c | 931 ++++++++++++++++++++++++++++++++ > drivers/md/bcache/nvm-pages.h | 127 +++++ > drivers/md/bcache/super.c | 53 +- > include/uapi/linux/bcache-nvm.h | 253 +++++++++ > 10 files changed, 1649 insertions(+), 68 deletions(-) > create mode 100644 drivers/md/bcache/nvm-pages.c > create mode 100644 drivers/md/bcache/nvm-pages.h > create mode 100644 include/uapi/linux/bcache-nvm.h >
This is the v12 effort for supporting NVDIMM for bcache journal (some versions may not posted with version numbers). The major change of this version is the full pointer of on-media data structure is replaced by per-namespace offset. Now a pointer address is calculated by namespace base mapping address + per-namespace offset. The code logic is same as previous version, all changes are only related to the base+offset style pointer replacement. The nvm-pages allocator is a buddy-like allocator, which allocates size in power-of-2 pages from the NVDIMM namespace. User space tool 'bcache' has a new added '-M' option to format a NVDIMM namespace and register it via sysfs interface as a bcache meta device. The nvm-pages kernel code does a DAX mapping to map the whole namespace into system's memory address range, and allocating the pages to requestion like typical buddy allocator does. The major difference is nvm-pages allocator maintains the pages allocated to each requester by an allocation list which stored on NVDIMM too. Allocation list of different requester is tracked by a pre-defined UUID, all the pages tracked in all allocation lists are treated as allocated busy pages and won't be initialized into buddy system after the system reboot. The bcache journal code may request a block of power-of-2 size pages from the nvm-pages allocator, normally it is a range of 256MB or 512MB continuous pages range. During meta data journaling, the in-memory jsets go into the calculated nvdimm pages location by kernel memcpy routine. So the journaling I/Os won't go into block device (e.g. SSD) anymore, the write and read for journal jsets happen on NVDIMM. Intel developers Jianpeng Ma and Qiaowei Ren compose the initial code of nvm-pages, the related patches are, - bcache: initialize the nvm pages allocator - bcache: initialization of the buddy - bcache: bch_nvm_alloc_pages() of the buddy - bcache: bch_nvm_free_pages() of the buddy - bcache: get recs list head for allocated pages by specific uuid All the code depends on Linux libnvdimm and dax drivers, the bcache nvm- pages allocator can be treated as user of these two drivers. I modify the bcache code to recognize the nvm meta device feature, initialize journal on NVDIMM, and do journal I/Os on NVDIMM in the following patches, - bcache: add initial data structures for nvm pages - bcache: use bucket index to set GC_MARK_METADATA for journal buckets in bch_btree_gc_finish() - bcache: add BCH_FEATURE_INCOMPAT_NVDIMM_META into incompat feature set - bcache: initialize bcache journal for NVDIMM meta device - bcache: support storing bcache journal into NVDIMM meta device - bcache: read jset from NVDIMM pages for journal replay - bcache: add sysfs interface register_nvdimm_meta to register NVDIMM meta device In this series, all previously addressed issue via code reviews are all fixed. And all known issue during testing are fixed. The code survives from 24+ hours smoking and I/O pressure testing among many reboots, it works well as expected. All the code is EXPERIMENTAL, they won't be enabled by default until we feel the NVDIMM support is completed and stable. Although there are some experts helped to review the code logic, but we do appreciate if more people may help to review the code. It is quite common that bcache patches don't have enough code reviewer, but this time I do need help for more review or comments on this series. Thanks in advance. Coly Li --- Coly Li (7): bcache: add initial data structures for nvm pages bcache: use bucket index to set GC_MARK_METADATA for journal buckets in bch_btree_gc_finish() bcache: add BCH_FEATURE_INCOMPAT_NVDIMM_META into incompat feature set bcache: initialize bcache journal for NVDIMM meta device bcache: support storing bcache journal into NVDIMM meta device bcache: read jset from NVDIMM pages for journal replay bcache: add sysfs interface register_nvdimm_meta to register NVDIMM meta device Jianpeng Ma (5): bcache: initialize the nvm pages allocator bcache: initialization of the buddy bcache: bch_nvmpg_alloc_pages() of the buddy bcache: bch_nvmpg_free_pages() of the buddy allocator bcache: get recs list head for allocated pages by specific uuid drivers/md/bcache/Kconfig | 10 + drivers/md/bcache/Makefile | 1 + drivers/md/bcache/btree.c | 6 +- drivers/md/bcache/features.h | 9 + drivers/md/bcache/journal.c | 325 +++++++++-- drivers/md/bcache/journal.h | 2 +- drivers/md/bcache/nvm-pages.c | 931 ++++++++++++++++++++++++++++++++ drivers/md/bcache/nvm-pages.h | 127 +++++ drivers/md/bcache/super.c | 53 +- include/uapi/linux/bcache-nvm.h | 253 +++++++++ 10 files changed, 1649 insertions(+), 68 deletions(-) create mode 100644 drivers/md/bcache/nvm-pages.c create mode 100644 drivers/md/bcache/nvm-pages.h create mode 100644 include/uapi/linux/bcache-nvm.h