| Message ID | BCD9738E-472D-4AA7-B4F9-CCF36B5DA0E1@fb.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL] md-next 20230622 | expand |
On 6/22/23 11:48?PM, Song Liu wrote: > Hi Jens, > > Please consider pulling the following changes for md-next on top of your > for-6.5/block branch. The major changes are: > > 1. Deprecate bitmap file support, by Christoph Hellwig; > 2. Fix deadlock with md sync thread, by Yu Kuai; > 3. Refactor md io accounting, by Yu Kuai. This is quite a lot on the day that I prepare pull requests for the merge window... I've said this many times before, but just to state this in completeness, maybe it'll benefit others too: 1) Major changes for the next release should be sent to me _at least_ 1 week prior to the merge window opening. That way it gets some decent soak time in linux-next before heading upstream. 2) Minor fixes, either for major pulls that already went into my next branch or just fixes in general, can be sent anytime and I'll shove them into the appropriate branch. When bigger stuff gets sent this late, then I have two choices: reject them and tell you to send it in for the next version, or setup a new branch just for this so I can send it to Linus in a later pull in the merge window. Neither of those two options are great - the first one delays you by a release, the second one creates more churn and hassle for me.
Hi Jens, I am so sorry for this problem. > On Jun 23, 2023, at 7:12 AM, Jens Axboe <axboe@kernel.dk> wrote: > > On 6/22/23 11:48?PM, Song Liu wrote: >> Hi Jens, >> >> Please consider pulling the following changes for md-next on top of your >> for-6.5/block branch. The major changes are: >> >> 1. Deprecate bitmap file support, by Christoph Hellwig; >> 2. Fix deadlock with md sync thread, by Yu Kuai; >> 3. Refactor md io accounting, by Yu Kuai. > > This is quite a lot on the day that I prepare pull requests for the > merge window... I've said this many times before, but just to state this > in completeness, maybe it'll benefit others too: > > 1) Major changes for the next release should be sent to me _at least_ 1 > week prior to the merge window opening. That way it gets some decent > soak time in linux-next before heading upstream. I am aware of the rule. A couple reasons caused a late PR this time: 1. Set #1 and set #3 are relatively new, especially set #3, which was first sent earlier this week. Set #2 is older, but there was more discussions on it until recently. (It is still my fault not pushing on set #2 sooner). 2. I wasn't very sure whether there will be a rc8. The announcement for rc7 didn't state it clearly. (Shall I assume there is no rc8 unless Linus states it clearly?) 3. I was hoping to group more patches into one PR. I guess this was the biggest mistake, especially when it is close to the merge window. > 2) Minor fixes, either for major pulls that already went into my next > branch or just fixes in general, can be sent anytime and I'll shove > them into the appropriate branch. > > When bigger stuff gets sent this late, then I have two choices: reject > them and tell you to send it in for the next version, or setup a new > branch just for this so I can send it to Linus in a later pull in the > merge window. Neither of those two options are great - the first one > delays you by a release, the second one creates more churn and hassle > for me. I will prepare another PR with just fixes. Christoph, Please let me know if you need set #1 (deprecate file bitmap) to unblock other work. Otherwise, we will delay it until 6.6. Thanks, Song
On 6/23/23 9:08?AM, Song Liu wrote: > Hi Jens, > > I am so sorry for this problem. > >> On Jun 23, 2023, at 7:12 AM, Jens Axboe <axboe@kernel.dk> wrote: >> >> On 6/22/23 11:48?PM, Song Liu wrote: >>> Hi Jens, >>> >>> Please consider pulling the following changes for md-next on top of your >>> for-6.5/block branch. The major changes are: >>> >>> 1. Deprecate bitmap file support, by Christoph Hellwig; >>> 2. Fix deadlock with md sync thread, by Yu Kuai; >>> 3. Refactor md io accounting, by Yu Kuai. >> >> This is quite a lot on the day that I prepare pull requests for the >> merge window... I've said this many times before, but just to state this >> in completeness, maybe it'll benefit others too: >> >> 1) Major changes for the next release should be sent to me _at least_ 1 >> week prior to the merge window opening. That way it gets some decent >> soak time in linux-next before heading upstream. > > I am aware of the rule. A couple reasons caused a late PR this time: Then please be explicit when sending out a pull request like this on why it's late. Otherwise it just looks like a normal pull request, which it is not. If your original pull request had any kind of explanation on why so much is being sent so late, then we would not be having this followup at all... > 1. Set #1 and set #3 are relatively new, especially set #3, which was > first sent earlier this week. Set #2 is older, but there was more > discussions on it until recently. (It is still my fault not pushing > on set #2 sooner). > > 2. I wasn't very sure whether there will be a rc8. The announcement for > rc7 didn't state it clearly. (Shall I assume there is no rc8 unless > Linus states it clearly?) > > 3. I was hoping to group more patches into one PR. I guess this was the > biggest mistake, especially when it is close to the merge window. Most of the time Linus will be explicit about _maybe_ doing an -rc8, but it doesn't really change the rule that I should have bigger stuff in the week between rc6 and rc7. When -rc7 is cut, my for-next branches should be basically baked and done at that point, and then post -rc7 just fixes for existing code. If people stick to that rc6-7 timing, then it'll always work, regardless of an -rc8 happening or not. Even when Linus has -rc8 musings in his later rc posts, it's not a given that they will happen. >> 2) Minor fixes, either for major pulls that already went into my next >> branch or just fixes in general, can be sent anytime and I'll shove >> them into the appropriate branch. >> >> When bigger stuff gets sent this late, then I have two choices: reject >> them and tell you to send it in for the next version, or setup a new >> branch just for this so I can send it to Linus in a later pull in the >> merge window. Neither of those two options are great - the first one >> delays you by a release, the second one creates more churn and hassle >> for me. > > I will prepare another PR with just fixes. > > Christoph, > > Please let me know if you need set #1 (deprecate file bitmap) to > unblock other work. Otherwise, we will delay it until 6.6. I've done a for-6.5/block-late and put your stuff there, but it can be dropped very easily. It doesn't really matter if Christoph's stuff can get pushed, it's still a lot of commits late. So regardless of that, the only real difference with a new PR would be that we'd drop some bits. It'd still go into a late branch, as it is indeed late.
On 6/23/23 9:16?AM, Jens Axboe wrote: >> I will prepare another PR with just fixes. >> >> Christoph, >> >> Please let me know if you need set #1 (deprecate file bitmap) to >> unblock other work. Otherwise, we will delay it until 6.6. > > I've done a for-6.5/block-late and put your stuff there, but it can be > dropped very easily. It doesn't really matter if Christoph's stuff can > get pushed, it's still a lot of commits late. So regardless of that, the > only real difference with a new PR would be that we'd drop some bits. > It'd still go into a late branch, as it is indeed late. On second thought, yes let's go your suggested route. Make a branch with JUST the fixes, and send them my way. Anything else will have to wait for 6.6 at this point. I'll drop my late branch for now.
> On Jun 23, 2023, at 8:16 AM, Jens Axboe <axboe@kernel.dk> wrote: > > On 6/23/23 9:08?AM, Song Liu wrote: >> Hi Jens, >> >> I am so sorry for this problem. >> >>> On Jun 23, 2023, at 7:12 AM, Jens Axboe <axboe@kernel.dk> wrote: >>> >>> On 6/22/23 11:48?PM, Song Liu wrote: >>>> Hi Jens, >>>> >>>> Please consider pulling the following changes for md-next on top of your >>>> for-6.5/block branch. The major changes are: >>>> >>>> 1. Deprecate bitmap file support, by Christoph Hellwig; >>>> 2. Fix deadlock with md sync thread, by Yu Kuai; >>>> 3. Refactor md io accounting, by Yu Kuai. >>> >>> This is quite a lot on the day that I prepare pull requests for the >>> merge window... I've said this many times before, but just to state this >>> in completeness, maybe it'll benefit others too: >>> >>> 1) Major changes for the next release should be sent to me _at least_ 1 >>> week prior to the merge window opening. That way it gets some decent >>> soak time in linux-next before heading upstream. >> >> I am aware of the rule. A couple reasons caused a late PR this time: > > Then please be explicit when sending out a pull request like this on why > it's late. Otherwise it just looks like a normal pull request, which it > is not. If your original pull request had any kind of explanation on why > so much is being sent so late, then we would not be having this followup > at all... Indeed! I should have explained the case in the first place. > >> 1. Set #1 and set #3 are relatively new, especially set #3, which was >> first sent earlier this week. Set #2 is older, but there was more >> discussions on it until recently. (It is still my fault not pushing >> on set #2 sooner). >> >> 2. I wasn't very sure whether there will be a rc8. The announcement for >> rc7 didn't state it clearly. (Shall I assume there is no rc8 unless >> Linus states it clearly?) >> >> 3. I was hoping to group more patches into one PR. I guess this was the >> biggest mistake, especially when it is close to the merge window. > > Most of the time Linus will be explicit about _maybe_ doing an -rc8, but > it doesn't really change the rule that I should have bigger stuff in the > week between rc6 and rc7. When -rc7 is cut, my for-next branches should > be basically baked and done at that point, and then post -rc7 just fixes > for existing code. If people stick to that rc6-7 timing, then it'll > always work, regardless of an -rc8 happening or not. > > Even when Linus has -rc8 musings in his later rc posts, it's not a given > that they will happen. Thanks for the explanation. I will get big things ready before rc7 in the future. > >>> 2) Minor fixes, either for major pulls that already went into my next >>> branch or just fixes in general, can be sent anytime and I'll shove >>> them into the appropriate branch. >>> >>> When bigger stuff gets sent this late, then I have two choices: reject >>> them and tell you to send it in for the next version, or setup a new >>> branch just for this so I can send it to Linus in a later pull in the >>> merge window. Neither of those two options are great - the first one >>> delays you by a release, the second one creates more churn and hassle >>> for me. >> >> I will prepare another PR with just fixes. >> >> Christoph, >> >> Please let me know if you need set #1 (deprecate file bitmap) to >> unblock other work. Otherwise, we will delay it until 6.6. > > I've done a for-6.5/block-late and put your stuff there, but it can be > dropped very easily. It doesn't really matter if Christoph's stuff can > get pushed, it's still a lot of commits late. So regardless of that, the > only real difference with a new PR would be that we'd drop some bits. > It'd still go into a late branch, as it is indeed late. Thanks for taking them to the late branch. If it is dropped, shall I 1. Delay bigger sets until 6.6; 2. Send fixes in a separate PR (after 6.5-rc1)? Thanks, Song
> On Jun 23, 2023, at 8:41 AM, Jens Axboe <axboe@kernel.dk> wrote: > > On 6/23/23 9:16?AM, Jens Axboe wrote: >>> I will prepare another PR with just fixes. >>> >>> Christoph, >>> >>> Please let me know if you need set #1 (deprecate file bitmap) to >>> unblock other work. Otherwise, we will delay it until 6.6. >> >> I've done a for-6.5/block-late and put your stuff there, but it can be >> dropped very easily. It doesn't really matter if Christoph's stuff can >> get pushed, it's still a lot of commits late. So regardless of that, the >> only real difference with a new PR would be that we'd drop some bits. >> It'd still go into a late branch, as it is indeed late. > > On second thought, yes let's go your suggested route. Make a branch with > JUST the fixes, and send them my way. Anything else will have to wait > for 6.6 at this point. I'll drop my late branch for now. Sounds good! Here is the updated pull request with only minor fixes. Three of the fixes are for patches in the for-6.5/block branch: - md: use mddev->external to select holder in export_rdev() - md/raid1-10: fix casting from randomized structure in raid1_submit_write() - md: fix 'delete_mutex' deadlock Thanks, Song The following changes since commit fcaa174a9c995cf0af3967e55644a1543ea07e36: scsi/sg: don't grab scsi host module reference (2023-06-23 08:28:18 -0600) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/song/md.git tags/md-next-20230623 for you to fetch changes up to a8d5fdd4d2702d0c7ec125bd3bbce3fc589afa67: raid10: avoid spin_lock from fastpath from raid10_unplug() (2023-06-23 09:41:50 -0700) ---------------------------------------------------------------- Li Nan (1): md/raid10: fix the condition to call bio_end_io_acct() Song Liu (1): md: use mddev->external to select holder in export_rdev() Yu Kuai (3): md/raid1-10: fix casting from randomized structure in raid1_submit_write() md: fix 'delete_mutex' deadlock raid10: avoid spin_lock from fastpath from raid10_unplug() drivers/md/md.c | 32 +++++++++++--------------------- drivers/md/md.h | 4 +--- drivers/md/raid1-10.c | 2 +- drivers/md/raid10.c | 6 +++--- 4 files changed, 16 insertions(+), 28 deletions(-)
On 6/23/23 11:03?AM, Song Liu wrote: > > >> On Jun 23, 2023, at 8:41 AM, Jens Axboe <axboe@kernel.dk> wrote: >> >> On 6/23/23 9:16?AM, Jens Axboe wrote: >>>> I will prepare another PR with just fixes. >>>> >>>> Christoph, >>>> >>>> Please let me know if you need set #1 (deprecate file bitmap) to >>>> unblock other work. Otherwise, we will delay it until 6.6. >>> >>> I've done a for-6.5/block-late and put your stuff there, but it can be >>> dropped very easily. It doesn't really matter if Christoph's stuff can >>> get pushed, it's still a lot of commits late. So regardless of that, the >>> only real difference with a new PR would be that we'd drop some bits. >>> It'd still go into a late branch, as it is indeed late. >> >> On second thought, yes let's go your suggested route. Make a branch with >> JUST the fixes, and send them my way. Anything else will have to wait >> for 6.6 at this point. I'll drop my late branch for now. > > Sounds good! Here is the updated pull request with only minor fixes. > Three of the fixes are for patches in the for-6.5/block branch: > > - md: use mddev->external to select holder in export_rdev() > - md/raid1-10: fix casting from randomized structure in raid1_submit_write() > - md: fix 'delete_mutex' deadlock Thanks, added to the for-6.5/block-late branch.
On Fri, Jun 23, 2023 at 03:08:52PM +0000, Song Liu wrote: > Please let me know if you need set #1 (deprecate file bitmap) to > unblock other work. Otherwise, we will delay it until 6.6. It was intended so that we don't have to make all of md depend on the new config option for buffers heads I plan to introduce. So it's a bit of a pity that we won't have it for 6.5, but not a deal breaker.
Hi Christoph, On Mon, Jun 26, 2023 at 8:51 AM Christoph Hellwig <hch@lst.de> wrote: > > On Fri, Jun 23, 2023 at 03:08:52PM +0000, Song Liu wrote: > > Please let me know if you need set #1 (deprecate file bitmap) to > > unblock other work. Otherwise, we will delay it until 6.6. > > It was intended so that we don't have to make all of md depend on > the new config option for buffers heads I plan to introduce. So it's > a bit of a pity that we won't have it for 6.5, but not a deal breaker. Thanks for the confirmation! Song
Hi Jens, Please consider pulling the following changes for md-next on top of your for-6.5/block branch. The major changes are: 1. Deprecate bitmap file support, by Christoph Hellwig; 2. Fix deadlock with md sync thread, by Yu Kuai; 3. Refactor md io accounting, by Yu Kuai. Thanks, Song The following changes since commit 648fa60fa7de3ca6f6303e1721591ad73def9cf0: block: don't return -EINVAL for not found names in devt_from_devname (2023-06-22 09:09:33 -0600) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/song/md.git tags/md-next-20230622 for you to fetch changes up to 41fb72ee7eeda723e619c6918dffaf05a55fc7dd: md/md-faulty: enable io accounting (2023-06-22 22:28:27 -0700) ---------------------------------------------------------------- Christoph Hellwig (11): md-bitmap: set BITMAP_WRITE_ERROR in write_sb_page md-bitmap: initialize variables at declaration time in md_bitmap_file_unmap md-bitmap: use %pD to print the file name in md_bitmap_file_kick md-bitmap: split file writes into a separate helper md-bitmap: rename read_page to read_file_page md-bitmap: refactor md_bitmap_init_from_disk md-bitmap: cleanup read_sb_page md-bitmap: account for mddev->bitmap_info.offset in read_sb_page md-bitmap: don't use ->index for pages backing the bitmap file md: make bitmap file support optional md: deprecate bitmap file support Li Nan (1): md/raid10: fix the condition to call bio_end_io_acct() Song Liu (1): md: use mddev->external to select holder in export_rdev() Yu Kuai (17): md/raid1-10: fix casting from randomized structure in raid1_submit_write() Revert "md: unlock mddev before reap sync_thread in action_store" md: refactor action_store() for 'idle' and 'frozen' md: add a mutex to synchronize idle and frozen in action_store() md: refactor idle/frozen_sync_thread() to fix deadlock md: wake up 'resync_wait' at last in md_reap_sync_thread() md: enhance checking in md_check_recovery() md: fix 'delete_mutex' deadlock raid10: avoid spin_lock from fastpath from raid10_unplug() md: move initialization and destruction of 'io_acct_set' to md.c md: also clone new io if io accounting is disabled raid5: fix missing io accounting in raid5_align_endio() md/raid1: switch to use md_account_bio() for io accounting md/raid10: switch to use md_account_bio() for io accounting md/md-multipath: enable io accounting md/md-linear: enable io accounting md/md-faulty: enable io accounting drivers/md/Kconfig | 10 +++++ drivers/md/dm-raid.c | 1 - drivers/md/md-bitmap.c | 338 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------------------ drivers/md/md-bitmap.h | 1 + drivers/md/md-faulty.c | 2 + drivers/md/md-linear.c | 1 + drivers/md/md-multipath.c | 1 + drivers/md/md.c | 245 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------- drivers/md/md.h | 15 +++---- drivers/md/raid0.c | 16 +------- drivers/md/raid1-10.c | 2 +- drivers/md/raid1.c | 14 +++---- drivers/md/raid1.h | 1 - drivers/md/raid10.c | 24 ++++++------ drivers/md/raid10.h | 1 - drivers/md/raid5.c | 70 +++++++++------------------------ 16 files changed, 372 insertions(+), 370 deletions(-)