mbox series

[v3,0/4] Devmap cleanups + arm64 support

Message ID cover.1558547956.git.robin.murphy@arm.com (mailing list archive)
Headers show
Series Devmap cleanups + arm64 support | expand

Message

Robin Murphy May 23, 2019, 3:03 p.m. UTC
Hi all,

Here's a refresh of my devmap stuff, now including the end goal as
well. Patches #1 to #3 are just a rebase of v2, and I hope are ready to
be picked up in the mm tree (#1 is currently doing double-duty in Dan's
subsection series as well). Patch #4 could either go via mm if Will and
Catalin agree, or could go via arm64 with a small tweak to let it build
(but otherwise do nothing) until it meets up with #3 again.

Robin.


Robin Murphy (4):
  mm/memremap: Rename and consolidate SECTION_SIZE
  mm: clean up is_device_*_page() definitions
  mm: introduce ARCH_HAS_PTE_DEVMAP
  arm64: mm: Implement pte_devmap support

 arch/arm64/Kconfig                           |  1 +
 arch/arm64/include/asm/pgtable-prot.h        |  1 +
 arch/arm64/include/asm/pgtable.h             | 19 ++++++++
 arch/powerpc/Kconfig                         |  2 +-
 arch/powerpc/include/asm/book3s/64/pgtable.h |  1 -
 arch/x86/Kconfig                             |  2 +-
 arch/x86/include/asm/pgtable.h               |  4 +-
 arch/x86/include/asm/pgtable_types.h         |  1 -
 include/linux/mm.h                           | 47 +++++++-------------
 include/linux/mmzone.h                       |  1 +
 include/linux/pfn_t.h                        |  4 +-
 kernel/memremap.c                            | 10 ++---
 mm/Kconfig                                   |  5 +--
 mm/gup.c                                     |  2 +-
 mm/hmm.c                                     |  2 -
 15 files changed, 50 insertions(+), 52 deletions(-)

Comments

Christoph Hellwig June 26, 2019, 7:35 a.m. UTC | #1
Robin, Andrew:

I have a series for the hmm tree, which touches the section size
bits, and remove device public memory support.

It might be best if we include this series in the hmm tree as well
to avoid conflicts.  Is it ok to include the rebase version of at least
the cleanup part (which looks like it is not required for the actual
arm64 support) in the hmm tree to avoid conflicts?
Mark Rutland June 26, 2019, 12:31 p.m. UTC | #2
On Wed, Jun 26, 2019 at 12:35:33AM -0700, Christoph Hellwig wrote:
> Robin, Andrew:

As a heads-up, Robin is currently on holiday, so this is all down to
Andrew's preference.

> I have a series for the hmm tree, which touches the section size
> bits, and remove device public memory support.
> 
> It might be best if we include this series in the hmm tree as well
> to avoid conflicts.  Is it ok to include the rebase version of at least
> the cleanup part (which looks like it is not required for the actual
> arm64 support) in the hmm tree to avoid conflicts?

Per the cover letter, the arm64 patch has a build dependency on the
others, so that might require a stable brnach for the common prefix.

Thanks,
Mark.
Christoph Hellwig June 26, 2019, 3:38 p.m. UTC | #3
On Wed, Jun 26, 2019 at 01:31:40PM +0100, Mark Rutland wrote:
> On Wed, Jun 26, 2019 at 12:35:33AM -0700, Christoph Hellwig wrote:
> > Robin, Andrew:
> 
> As a heads-up, Robin is currently on holiday, so this is all down to
> Andrew's preference.
> 
> > I have a series for the hmm tree, which touches the section size
> > bits, and remove device public memory support.
> > 
> > It might be best if we include this series in the hmm tree as well
> > to avoid conflicts.  Is it ok to include the rebase version of at least
> > the cleanup part (which looks like it is not required for the actual
> > arm64 support) in the hmm tree to avoid conflicts?
> 
> Per the cover letter, the arm64 patch has a build dependency on the
> others, so that might require a stable brnach for the common prefix.

I guess we'll just have to live with the merge errors then, as the
mm tree is a patch series and thus can't easily use a stable base
tree.  That is unlike Andrew wants to pull in the hmm tree as a prep
patch for the series.
Jason Gunthorpe June 26, 2019, 3:45 p.m. UTC | #4
On Wed, Jun 26, 2019 at 08:38:29AM -0700, Christoph Hellwig wrote:
> On Wed, Jun 26, 2019 at 01:31:40PM +0100, Mark Rutland wrote:
> > On Wed, Jun 26, 2019 at 12:35:33AM -0700, Christoph Hellwig wrote:
> > > Robin, Andrew:
> > 
> > As a heads-up, Robin is currently on holiday, so this is all down to
> > Andrew's preference.
> > 
> > > I have a series for the hmm tree, which touches the section size
> > > bits, and remove device public memory support.
> > > 
> > > It might be best if we include this series in the hmm tree as well
> > > to avoid conflicts.  Is it ok to include the rebase version of at least
> > > the cleanup part (which looks like it is not required for the actual
> > > arm64 support) in the hmm tree to avoid conflicts?
> > 
> > Per the cover letter, the arm64 patch has a build dependency on the
> > others, so that might require a stable brnach for the common prefix.
> 
> I guess we'll just have to live with the merge errors then, as the
> mm tree is a patch series and thus can't easily use a stable base
> tree.  That is unlike Andrew wants to pull in the hmm tree as a prep
> patch for the series.

It looks like the first three patches apply cleanly to hmm.git ..

So what we can do is base this 4 patch series off rc6 and pull the
first 3 into hmm and the full 4 into arm.git. We use this workflow often
with rdma and netdev.

Let me know and I can help orchestate this.

Jason
Andrew Morton June 27, 2019, 3:35 a.m. UTC | #5
On Wed, 26 Jun 2019 15:45:47 +0000 Jason Gunthorpe <jgg@mellanox.com> wrote:

> On Wed, Jun 26, 2019 at 08:38:29AM -0700, Christoph Hellwig wrote:
> > On Wed, Jun 26, 2019 at 01:31:40PM +0100, Mark Rutland wrote:
> > > On Wed, Jun 26, 2019 at 12:35:33AM -0700, Christoph Hellwig wrote:
> > > > Robin, Andrew:
> > > 
> > > As a heads-up, Robin is currently on holiday, so this is all down to
> > > Andrew's preference.
> > > 
> > > > I have a series for the hmm tree, which touches the section size
> > > > bits, and remove device public memory support.
> > > > 
> > > > It might be best if we include this series in the hmm tree as well
> > > > to avoid conflicts.  Is it ok to include the rebase version of at least
> > > > the cleanup part (which looks like it is not required for the actual
> > > > arm64 support) in the hmm tree to avoid conflicts?
> > > 
> > > Per the cover letter, the arm64 patch has a build dependency on the
> > > others, so that might require a stable brnach for the common prefix.
> > 
> > I guess we'll just have to live with the merge errors then, as the
> > mm tree is a patch series and thus can't easily use a stable base
> > tree.  That is unlike Andrew wants to pull in the hmm tree as a prep
> > patch for the series.
> 
> It looks like the first three patches apply cleanly to hmm.git ..
> 
> So what we can do is base this 4 patch series off rc6 and pull the
> first 3 into hmm and the full 4 into arm.git. We use this workflow often
> with rdma and netdev.
> 
> Let me know and I can help orchestate this.

Well.  Whatever works.  In this situation I'd stage the patches after
linux-next and would merge them up after the prereq patches have been
merged into mainline.  Easy.
Andrew Morton July 4, 2019, 6:53 p.m. UTC | #6
On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> > Let me know and I can help orchestate this.
> 
> Well.  Whatever works.  In this situation I'd stage the patches after
> linux-next and would merge them up after the prereq patches have been
> merged into mainline.  Easy.

All right, what the hell just happened?  A bunch of new material has
just been introduced into linux-next.  I've partially unpicked the
resulting mess, haven't dared trying to compile it yet.  To get this
far I'll need to drop two patch series and one individual patch:


mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patch

mm-sparsemem-introduce-struct-mem_section_usage.patch
mm-sparsemem-introduce-a-section_is_early-flag.patch
mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch
mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch
mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch
mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
mm-kill-is_dev_zone-helper.patch
mm-sparsemem-prepare-for-sub-section-ranges.patch
mm-sparsemem-support-sub-section-hotplug.patch
mm-document-zone_device-memory-model-implications.patch
mm-document-zone_device-memory-model-implications-fix.patch
mm-devm_memremap_pages-enable-sub-section-remap.patch
libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch
libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch

mm-sparsemem-cleanup-section-number-data-types.patch
mm-sparsemem-cleanup-section-number-data-types-fix.patch


I thought you were just going to move material out of -mm and into
hmm.git.  Didn't begin to suspect that new and quite disruptive
material would be introduced late in -rc7!!
Jason Gunthorpe July 4, 2019, 7:59 p.m. UTC | #7
On Thu, Jul 04, 2019 at 11:53:24AM -0700, Andrew Morton wrote:
> On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > > Let me know and I can help orchestate this.
> > 
> > Well.  Whatever works.  In this situation I'd stage the patches after
> > linux-next and would merge them up after the prereq patches have been
> > merged into mainline.  Easy.
> 
> All right, what the hell just happened? 

Christoph's patch series for the devmap & hmm rework finally made it
into linux-next, sorry, it took quite a few iterations on the list to
get all the reviews and tests, and figure out how to resolve some
other conflicting things. So it just made it this week.

Recall, this is the patch series I asked you about routing a few weeks
ago, as it really exceeded the small area that hmm.git was supposed to
cover. I think we are both caught off guard how big the conflict is!

> A bunch of new material has just been introduced into linux-next.
> I've partially unpicked the resulting mess, haven't dared trying to
> compile it yet.  To get this far I'll need to drop two patch series
> and one individual patch:
  
> mm-clean-up-is_device__page-definitions.patch
> mm-introduce-arch_has_pte_devmap.patch
> arm64-mm-implement-pte_devmap-support.patch
> arm64-mm-implement-pte_devmap-support-fix.patch

This one we discussed, and I thought we agreed would go to your 'stage
after linux-next' flow (see above). I think the conflict was minor
here.

> mm-sparsemem-introduce-struct-mem_section_usage.patch
> mm-sparsemem-introduce-a-section_is_early-flag.patch
> mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch
> mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch
> mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch
> mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
> mm-kill-is_dev_zone-helper.patch
> mm-sparsemem-prepare-for-sub-section-ranges.patch
> mm-sparsemem-support-sub-section-hotplug.patch
> mm-document-zone_device-memory-model-implications.patch
> mm-document-zone_device-memory-model-implications-fix.patch
> mm-devm_memremap_pages-enable-sub-section-remap.patch
> libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch
> libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch

Dan pointed to this while reviewing CH's series and said the conflicts
would be manageable, but they are certainly larger than I expected!

This series is the one that seems to be the really big trouble. I
already checked all the other stuff that Stephen resolved, and it
looks OK and managable. Just this one conflict with kernel/memremap.c
is beyond me. 

What approach do you want to take to go forward? Here are some thoughts:

CH has said he is away for the long weekend, so the path that involves
the fewest people is if Dan respins the above on linux-next and it
goes later with the arm patches above, assuming defering it for now
has no other adverse effects on -mm.

Pushing CH's series to -mm would need a respin on top of Dan's series
above and would need to carry along the whole hmm.git (about 44
patches). Signs are that this could be managed with the code currently
in the GPU trees.

If we give up on CH's series the hmm.git will not have conflicts,
however we just kick the can to the next merge window where we will be
back to having to co-ordinate amd/nouveau/rdma git trees and -mm's
patch workflow - and I think we will be worse off as we will have
totally given up on a git based work flow for this. :(

> mm-sparsemem-cleanup-section-number-data-types.patch
> mm-sparsemem-cleanup-section-number-data-types-fix.patch

Stephen used a minor conflict resolution for this one, I checked it
carefully and it looked OK.

> I thought you were just going to move material out of -mm and into
> hmm.git.  

Dan brought up a patch from Ira conflicting with CH's work and we did
handle that by moving a single patch, as well I moved several hmm
specific patches early in the cycle.

> Didn't begin to suspect that new and quite disruptive material would
> be introduced late in -rc7!!

Unfortunately a non-rebasing tree like hmm.git should only get patches
into linux-next once they are fully reviewed and done on the list. I
did not attempt to run separately patches 'under review' into
linux-next as you do. 

Actually I didn't even know this would benefit your workflow, rebasing
patches on top of linux-next is not part of the git based workflow I'm
using :(

AFAIK Dan and CH were both tracking conflicts with linux-next, so I'd
like to hear from Dan what he thinks about his series, maybe the
rebase is simple & safe for him? Dan and CH were working pretty
closely on CH's series.

Jason
Andrew Morton July 4, 2019, 8:53 p.m. UTC | #8
On Thu, 4 Jul 2019 19:59:38 +0000 Jason Gunthorpe <jgg@mellanox.com> wrote:

> On Thu, Jul 04, 2019 at 11:53:24AM -0700, Andrew Morton wrote:
> > On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > 
> > > > Let me know and I can help orchestate this.
> > > 
> > > Well.  Whatever works.  In this situation I'd stage the patches after
> > > linux-next and would merge them up after the prereq patches have been
> > > merged into mainline.  Easy.
> > 
> > All right, what the hell just happened? 
> 
> Christoph's patch series for the devmap & hmm rework finally made it
> into linux-next

We're talking about "dev_pagemap related cleanups v4", yes?

I note that linux-next contains "mm: remove the HMM config option"
which was present in Christoph's v3 series but wasn't present in v4. 
Perhaps something has gone wrong here.

> sorry, it took quite a few iterations on the list to
> get all the reviews and tests, and figure out how to resolve some
> other conflicting things. So it just made it this week.
> 
> Recall, this is the patch series I asked you about routing a few weeks
> ago, as it really exceeded the small area that hmm.git was supposed to
> cover. I think we are both caught off guard how big the conflict is!

I guess I was distracted - I should have taken a look to see how
mergable it all was.

It's a large patchset and it appears to be mainly (entirely?) code
cleanups.  I don't think such material would be appropriate for a late
-rc7 merge even if it didn't conflict with lots of other higher
priority pending functional changes and fixes!
Robin Murphy July 4, 2019, 8:54 p.m. UTC | #9
On 2019-07-04 8:59 pm, Jason Gunthorpe wrote:
> On Thu, Jul 04, 2019 at 11:53:24AM -0700, Andrew Morton wrote:
>> On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>>
>>>> Let me know and I can help orchestate this.
>>>
>>> Well.  Whatever works.  In this situation I'd stage the patches after
>>> linux-next and would merge them up after the prereq patches have been
>>> merged into mainline.  Easy.
>>
>> All right, what the hell just happened?

Aw crap, and I had this series chalked up as done...

> Christoph's patch series for the devmap & hmm rework finally made it
> into linux-next, sorry, it took quite a few iterations on the list to
> get all the reviews and tests, and figure out how to resolve some
> other conflicting things. So it just made it this week.
> 
> Recall, this is the patch series I asked you about routing a few weeks
> ago, as it really exceeded the small area that hmm.git was supposed to
> cover. I think we are both caught off guard how big the conflict is!
> 
>> A bunch of new material has just been introduced into linux-next.
>> I've partially unpicked the resulting mess, haven't dared trying to
>> compile it yet.  To get this far I'll need to drop two patch series
>> and one individual patch:
>    
>> mm-clean-up-is_device__page-definitions.patch
>> mm-introduce-arch_has_pte_devmap.patch
>> arm64-mm-implement-pte_devmap-support.patch
>> arm64-mm-implement-pte_devmap-support-fix.patch
> 
> This one we discussed, and I thought we agreed would go to your 'stage
> after linux-next' flow (see above). I think the conflict was minor
> here.

I can rebase and resend tomorrow if there's an agreement on what exactly 
to base it on - I'd really like to get this ticked off for 5.3 if at all 
possible.

Thanks,
Robin.

>> mm-sparsemem-introduce-struct-mem_section_usage.patch
>> mm-sparsemem-introduce-a-section_is_early-flag.patch
>> mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch
>> mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch
>> mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch
>> mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
>> mm-kill-is_dev_zone-helper.patch
>> mm-sparsemem-prepare-for-sub-section-ranges.patch
>> mm-sparsemem-support-sub-section-hotplug.patch
>> mm-document-zone_device-memory-model-implications.patch
>> mm-document-zone_device-memory-model-implications-fix.patch
>> mm-devm_memremap_pages-enable-sub-section-remap.patch
>> libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch
>> libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
> 
> Dan pointed to this while reviewing CH's series and said the conflicts
> would be manageable, but they are certainly larger than I expected!
> 
> This series is the one that seems to be the really big trouble. I
> already checked all the other stuff that Stephen resolved, and it
> looks OK and managable. Just this one conflict with kernel/memremap.c
> is beyond me.
> 
> What approach do you want to take to go forward? Here are some thoughts:
> 
> CH has said he is away for the long weekend, so the path that involves
> the fewest people is if Dan respins the above on linux-next and it
> goes later with the arm patches above, assuming defering it for now
> has no other adverse effects on -mm.
> 
> Pushing CH's series to -mm would need a respin on top of Dan's series
> above and would need to carry along the whole hmm.git (about 44
> patches). Signs are that this could be managed with the code currently
> in the GPU trees.
> 
> If we give up on CH's series the hmm.git will not have conflicts,
> however we just kick the can to the next merge window where we will be
> back to having to co-ordinate amd/nouveau/rdma git trees and -mm's
> patch workflow - and I think we will be worse off as we will have
> totally given up on a git based work flow for this. :(
> 
>> mm-sparsemem-cleanup-section-number-data-types.patch
>> mm-sparsemem-cleanup-section-number-data-types-fix.patch
> 
> Stephen used a minor conflict resolution for this one, I checked it
> carefully and it looked OK.
> 
>> I thought you were just going to move material out of -mm and into
>> hmm.git.
> 
> Dan brought up a patch from Ira conflicting with CH's work and we did
> handle that by moving a single patch, as well I moved several hmm
> specific patches early in the cycle.
> 
>> Didn't begin to suspect that new and quite disruptive material would
>> be introduced late in -rc7!!
> 
> Unfortunately a non-rebasing tree like hmm.git should only get patches
> into linux-next once they are fully reviewed and done on the list. I
> did not attempt to run separately patches 'under review' into
> linux-next as you do.
> 
> Actually I didn't even know this would benefit your workflow, rebasing
> patches on top of linux-next is not part of the git based workflow I'm
> using :(
> 
> AFAIK Dan and CH were both tracking conflicts with linux-next, so I'd
> like to hear from Dan what he thinks about his series, maybe the
> rebase is simple & safe for him? Dan and CH were working pretty
> closely on CH's series.
> 
> Jason
>
Andrew Morton July 4, 2019, 9:13 p.m. UTC | #10
On Thu, 4 Jul 2019 21:54:36 +0100 Robin Murphy <robin.murphy@arm.com> wrote:

> >> mm-clean-up-is_device__page-definitions.patch
> >> mm-introduce-arch_has_pte_devmap.patch
> >> arm64-mm-implement-pte_devmap-support.patch
> >> arm64-mm-implement-pte_devmap-support-fix.patch
> > 
> > This one we discussed, and I thought we agreed would go to your 'stage
> > after linux-next' flow (see above). I think the conflict was minor
> > here.
> 
> I can rebase and resend tomorrow if there's an agreement on what exactly 
> to base it on - I'd really like to get this ticked off for 5.3 if at all 
> possible.

I took another look.  Yes, it looks like the repairs were simple.

Let me now try to compile all this...
Jason Gunthorpe July 4, 2019, 9:28 p.m. UTC | #11
On Thu, Jul 04, 2019 at 01:53:32PM -0700, Andrew Morton wrote:
> On Thu, 4 Jul 2019 19:59:38 +0000 Jason Gunthorpe <jgg@mellanox.com> wrote:
> 
> > On Thu, Jul 04, 2019 at 11:53:24AM -0700, Andrew Morton wrote:
> > > On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > > 
> > > > > Let me know and I can help orchestate this.
> > > > 
> > > > Well.  Whatever works.  In this situation I'd stage the patches after
> > > > linux-next and would merge them up after the prereq patches have been
> > > > merged into mainline.  Easy.
> > > 
> > > All right, what the hell just happened? 
> > 
> > Christoph's patch series for the devmap & hmm rework finally made it
> > into linux-next
> 
> We're talking about "dev_pagemap related cleanups v4", yes?
>
> I note that linux-next contains "mm: remove the HMM config option"
> which was present in Christoph's v3 series but wasn't present in v4. 
> Perhaps something has gone wrong here.

When CH sent v4 to the list it was corrupted, v3 is the one to
reference for content.

Here is the mailing thread discussing this:

https://lore.kernel.org/linux-mm/20190702184201.GO31718@mellanox.com/

> > sorry, it took quite a few iterations on the list to
> > get all the reviews and tests, and figure out how to resolve some
> > other conflicting things. So it just made it this week.
> > 
> > Recall, this is the patch series I asked you about routing a few weeks
> > ago, as it really exceeded the small area that hmm.git was supposed to
> > cover. I think we are both caught off guard how big the conflict is!
> 
> I guess I was distracted - I should have taken a look to see how
> mergable it all was.

Okay, fair enough. I also could have also done more checks myself with
linux-next

> It's a large patchset and it appears to be mainly (entirely?) code
> cleanups.  I don't think such material would be appropriate for a late
> -rc7 merge even if it didn't conflict with lots of other higher
> priority pending functional changes and fixes!

I see your other email you resolved the conflicts - so please let me
know if you want to proceed with dropping CH's series or not, I'll
make a special effort to get that change into tomorrows linux-next if
you want (it is already 6pm here)

Regards,
Jason
Dan Williams July 4, 2019, 11:37 p.m. UTC | #12
On Thu, Jul 4, 2019 at 12:59 PM Jason Gunthorpe <jgg@mellanox.com> wrote:
>
> On Thu, Jul 04, 2019 at 11:53:24AM -0700, Andrew Morton wrote:
> > On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > > > Let me know and I can help orchestate this.
> > >
> > > Well.  Whatever works.  In this situation I'd stage the patches after
> > > linux-next and would merge them up after the prereq patches have been
> > > merged into mainline.  Easy.
> >
> > All right, what the hell just happened?
>
> Christoph's patch series for the devmap & hmm rework finally made it
> into linux-next, sorry, it took quite a few iterations on the list to
> get all the reviews and tests, and figure out how to resolve some
> other conflicting things. So it just made it this week.
>
> Recall, this is the patch series I asked you about routing a few weeks
> ago, as it really exceeded the small area that hmm.git was supposed to
> cover. I think we are both caught off guard how big the conflict is!
>
> > A bunch of new material has just been introduced into linux-next.
> > I've partially unpicked the resulting mess, haven't dared trying to
> > compile it yet.  To get this far I'll need to drop two patch series
> > and one individual patch:
>
> > mm-clean-up-is_device__page-definitions.patch
> > mm-introduce-arch_has_pte_devmap.patch
> > arm64-mm-implement-pte_devmap-support.patch
> > arm64-mm-implement-pte_devmap-support-fix.patch
>
> This one we discussed, and I thought we agreed would go to your 'stage
> after linux-next' flow (see above). I think the conflict was minor
> here.
>
> > mm-sparsemem-introduce-struct-mem_section_usage.patch
> > mm-sparsemem-introduce-a-section_is_early-flag.patch
> > mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch
> > mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch
> > mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch
> > mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
> > mm-kill-is_dev_zone-helper.patch
> > mm-sparsemem-prepare-for-sub-section-ranges.patch
> > mm-sparsemem-support-sub-section-hotplug.patch
> > mm-document-zone_device-memory-model-implications.patch
> > mm-document-zone_device-memory-model-implications-fix.patch
> > mm-devm_memremap_pages-enable-sub-section-remap.patch
> > libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch
> > libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
>
> Dan pointed to this while reviewing CH's series and said the conflicts
> would be manageable, but they are certainly larger than I expected!
>
> This series is the one that seems to be the really big trouble. I
> already checked all the other stuff that Stephen resolved, and it
> looks OK and managable. Just this one conflict with kernel/memremap.c
> is beyond me.
>
> What approach do you want to take to go forward? Here are some thoughts:
>
> CH has said he is away for the long weekend, so the path that involves
> the fewest people is if Dan respins the above on linux-next and it
> goes later with the arm patches above, assuming defering it for now
> has no other adverse effects on -mm.
>
> Pushing CH's series to -mm would need a respin on top of Dan's series
> above and would need to carry along the whole hmm.git (about 44
> patches). Signs are that this could be managed with the code currently
> in the GPU trees.
>
> If we give up on CH's series the hmm.git will not have conflicts,
> however we just kick the can to the next merge window where we will be
> back to having to co-ordinate amd/nouveau/rdma git trees and -mm's
> patch workflow - and I think we will be worse off as we will have
> totally given up on a git based work flow for this. :(

I think the problem would be resolved going forward post-v5.3 since we
won't have two tress managing kernel/memremap.c. This cycle however
there is a backlog of kernel/memremap.c changes in -mm.
Robin Murphy July 5, 2019, 11:16 a.m. UTC | #13
On 04/07/2019 22:13, Andrew Morton wrote:
> On Thu, 4 Jul 2019 21:54:36 +0100 Robin Murphy <robin.murphy@arm.com> wrote:
> 
>>>> mm-clean-up-is_device__page-definitions.patch
>>>> mm-introduce-arch_has_pte_devmap.patch
>>>> arm64-mm-implement-pte_devmap-support.patch
>>>> arm64-mm-implement-pte_devmap-support-fix.patch
>>>
>>> This one we discussed, and I thought we agreed would go to your 'stage
>>> after linux-next' flow (see above). I think the conflict was minor
>>> here.
>>
>> I can rebase and resend tomorrow if there's an agreement on what exactly
>> to base it on - I'd really like to get this ticked off for 5.3 if at all
>> possible.
> 
> I took another look.  Yes, it looks like the repairs were simple.
> 
> Let me now try to compile all this...

Thanks, the revised patches look OK to me, and I've confirmed that 
today's -next builds and boots for arm64.

Cheers,
Robin.
Jason Gunthorpe July 5, 2019, 12:32 p.m. UTC | #14
On Thu, Jul 04, 2019 at 04:37:51PM -0700, Dan Williams wrote:

> > If we give up on CH's series the hmm.git will not have conflicts,
> > however we just kick the can to the next merge window where we will be
> > back to having to co-ordinate amd/nouveau/rdma git trees and -mm's
> > patch workflow - and I think we will be worse off as we will have
> > totally given up on a git based work flow for this. :(
> 
> I think the problem would be resolved going forward post-v5.3 since we
> won't have two tress managing kernel/memremap.c. This cycle however
> there is a backlog of kernel/memremap.c changes in -mm.

IHMO there is always something :( 

CH's series had something like 5 different collisions already, and I
think we did a good job of with everything but your subsection
patches.

Jason
Jason Gunthorpe July 5, 2019, 3:47 p.m. UTC | #15
On Thu, Jul 04, 2019 at 06:28:50PM -0300, Jason Gunthorpe wrote:

> > It's a large patchset and it appears to be mainly (entirely?) code
> > cleanups.  I don't think such material would be appropriate for a late
> > -rc7 merge even if it didn't conflict with lots of other higher
> > priority pending functional changes and fixes!
> 
> I see your other email you resolved the conflicts - so please let me
> know if you want to proceed with dropping CH's series or not, I'll
> make a special effort to get that change into tomorrows linux-next if
> you want (it is already 6pm here)

I spent some time this morning looking at the various conflicts, and I
think Dan is right, they are mangable. In the sense we can forward a
merge resolution to Linus and it is not completely crazy. Most hunks
are the usual mechanical sort of conflicts.

Like Stephen, only two small conflict hunks in the memremap.c give me
any pause, and I'm confident with CH and Dan's help it can be resolved
robustly. If Linus doesn't like it then we fall back to dropping CH's
series.

So, here is a fourth idea..

We remove hmm.git entirely from your workflow (ie you revert commit
"cc5dfd59e375f Merge branch 'hmm-devmem-cleanup.4' into rdma.git hmm"
in your local version of linux-next) and I will send hmm.git to Linus
after Dan's patches and others are merged by you to Linus. With Dan
and CH's help I will forward the reviewed conflict resolution.

This will not disturb the -mm patch workflow at all, and you can put
everything back the way it was on July 3.

What do you think about this?

Jason