| Message ID | 1526653072-7153-2-git-send-email-okaya@codeaurora.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Ard, On 5/18/2018 10:17 AM, Sinan Kaya wrote: > A host bridge is allowed to remap BAR addresses using _TRA attribute in > _CRS windows. > > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > > When a VGA device is behind such a host bridge and the resource is > translated efifb driver is trying to do ioremap against bus address > rather than the resource address and is failing to probe. > > efifb: probing for efifb > efifb: cannot reserve video memory at 0x1e000000 > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > efifb: mode is 800x600x32, linelength=3200, pages=1 > efifb: scrolling: redraw > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > > Use the host bridge offset information to convert bus address to > resource address in the fixup. > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > --- I didn't see any messages about these getting picked up for 4.18. Are they queued on your own branch? Sinan
On 13 June 2018 at 16:22, Sinan Kaya <okaya@codeaurora.org> wrote: > Hi Ard, > > On 5/18/2018 10:17 AM, Sinan Kaya wrote: >> A host bridge is allowed to remap BAR addresses using _TRA attribute in >> _CRS windows. >> >> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) >> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >> >> When a VGA device is behind such a host bridge and the resource is >> translated efifb driver is trying to do ioremap against bus address >> rather than the resource address and is failing to probe. >> >> efifb: probing for efifb >> efifb: cannot reserve video memory at 0x1e000000 >> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >> efifb: mode is 800x600x32, linelength=3200, pages=1 >> efifb: scrolling: redraw >> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >> >> Use the host bridge offset information to convert bus address to >> resource address in the fixup. >> >> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >> --- > > I didn't see any messages about these getting picked up for 4.18. > > Are they queued on your own branch? > No, you never cc'ed me on them until now. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2018-06-13 11:06, Ard Biesheuvel wrote: > On 13 June 2018 at 16:22, Sinan Kaya <okaya@codeaurora.org> wrote: >> Hi Ard, >> >> On 5/18/2018 10:17 AM, Sinan Kaya wrote: >>> A host bridge is allowed to remap BAR addresses using _TRA attribute >>> in >>> _CRS windows. >>> >>> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff >>> window] (bus address [0x00100000-0x1fffffff]) >>> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >>> >>> When a VGA device is behind such a host bridge and the resource is >>> translated efifb driver is trying to do ioremap against bus address >>> rather than the resource address and is failing to probe. >>> >>> efifb: probing for efifb >>> efifb: cannot reserve video memory at 0x1e000000 >>> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >>> efifb: mode is 800x600x32, linelength=3200, pages=1 >>> efifb: scrolling: redraw >>> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >>> >>> Use the host bridge offset information to convert bus address to >>> resource address in the fixup. >>> >>> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >>> --- >> >> I didn't see any messages about these getting picked up for 4.18. >> >> Are they queued on your own branch? >> > > No, you never cc'ed me on them until now. Ouch, I hoped that you would get it via get_maintainer script. Sorry for that. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 13 June 2018 at 17:17, <okaya@codeaurora.org> wrote: > On 2018-06-13 11:06, Ard Biesheuvel wrote: >> >> On 13 June 2018 at 16:22, Sinan Kaya <okaya@codeaurora.org> wrote: >>> >>> Hi Ard, >>> >>> On 5/18/2018 10:17 AM, Sinan Kaya wrote: >>>> >>>> A host bridge is allowed to remap BAR addresses using _TRA attribute in >>>> _CRS windows. >>>> >>>> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff >>>> window] (bus address [0x00100000-0x1fffffff]) >>>> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >>>> >>>> When a VGA device is behind such a host bridge and the resource is >>>> translated efifb driver is trying to do ioremap against bus address >>>> rather than the resource address and is failing to probe. >>>> >>>> efifb: probing for efifb >>>> efifb: cannot reserve video memory at 0x1e000000 >>>> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >>>> efifb: mode is 800x600x32, linelength=3200, pages=1 >>>> efifb: scrolling: redraw >>>> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >>>> >>>> Use the host bridge offset information to convert bus address to >>>> resource address in the fixup. >>>> >>>> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >>>> --- >>> >>> >>> I didn't see any messages about these getting picked up for 4.18. >>> >>> Are they queued on your own branch? >>> >> >> No, you never cc'ed me on them until now. > > > Ouch, I hoped that you would get it via get_maintainer script. Sorry for > that. Actually, get_maintainer is right: this should go through the fbdev tree not the EFI tree Were you going to resend them? If not, I can find them in my archive and ack them, and we will ask Bartlomiej to take them for v4.19 -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2018-06-13 11:22, Ard Biesheuvel wrote: > On 13 June 2018 at 17:17, <okaya@codeaurora.org> wrote: >> On 2018-06-13 11:06, Ard Biesheuvel wrote: >>> >>> On 13 June 2018 at 16:22, Sinan Kaya <okaya@codeaurora.org> wrote: >>>> >>>> Hi Ard, >>>> >>>> On 5/18/2018 10:17 AM, Sinan Kaya wrote: >>>>> >>>>> A host bridge is allowed to remap BAR addresses using _TRA >>>>> attribute in >>>>> _CRS windows. >>>>> >>>>> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff >>>>> window] (bus address [0x00100000-0x1fffffff]) >>>>> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >>>>> >>>>> When a VGA device is behind such a host bridge and the resource is >>>>> translated efifb driver is trying to do ioremap against bus address >>>>> rather than the resource address and is failing to probe. >>>>> >>>>> efifb: probing for efifb >>>>> efifb: cannot reserve video memory at 0x1e000000 >>>>> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >>>>> efifb: mode is 800x600x32, linelength=3200, pages=1 >>>>> efifb: scrolling: redraw >>>>> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >>>>> >>>>> Use the host bridge offset information to convert bus address to >>>>> resource address in the fixup. >>>>> >>>>> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >>>>> --- >>>> >>>> >>>> I didn't see any messages about these getting picked up for 4.18. >>>> >>>> Are they queued on your own branch? >>>> >>> >>> No, you never cc'ed me on them until now. >> >> >> Ouch, I hoped that you would get it via get_maintainer script. Sorry >> for >> that. > > Actually, get_maintainer is right: this should go through the fbdev > tree not the EFI tree > > Were you going to resend them? If not, I can find them in my archive > and ack them, and we will ask Bartlomiej to take them for v4.19 I prefer ack rather than resend to be honest. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: > A host bridge is allowed to remap BAR addresses using _TRA attribute in > _CRS windows. > > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > > When a VGA device is behind such a host bridge and the resource is > translated efifb driver is trying to do ioremap against bus address > rather than the resource address and is failing to probe. > > efifb: probing for efifb > efifb: cannot reserve video memory at 0x1e000000 > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > efifb: mode is 800x600x32, linelength=3200, pages=1 > efifb: scrolling: redraw > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > > Use the host bridge offset information to convert bus address to > resource address in the fixup. > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Bartlomiej, could you please take these via the fbdev tree for v4.19? Peter already gave his ack but Sinan dropped it (presumably because of the split in v2) Sinan, does this need to go to -stable? I.e., has it ever worked before? > --- > drivers/video/fbdev/efifb.c | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c > index 6daac8d..429cc85 100644 > --- a/drivers/video/fbdev/efifb.c > +++ b/drivers/video/fbdev/efifb.c > @@ -431,6 +431,7 @@ static void efifb_fixup_resources(struct pci_dev *dev) > .end = screen_info.lfb_base + screen_info.lfb_size - 1, > .flags = IORESOURCE_MEM, > }; > + struct pci_bus_region region; > int i; > > if (efifb_pci_dev || screen_info.orig_video_isVGA != VIDEO_TYPE_EFI) > @@ -442,6 +443,10 @@ static void efifb_fixup_resources(struct pci_dev *dev) > if (!screen_res.start) > return; > > + region.start = screen_res.start; > + region.end = screen_res.end; > + pcibios_bus_to_resource(dev->bus, &screen_res, ®ion); > + > for (i = 0; i <= PCI_STD_RESOURCE_END; i++) { > struct resource *res = &dev->resource[i]; > > -- > 2.7.4 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2018-06-13 11:45, Ard Biesheuvel wrote: > On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: >> A host bridge is allowed to remap BAR addresses using _TRA attribute >> in >> _CRS windows. >> >> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff >> window] (bus address [0x00100000-0x1fffffff]) >> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >> >> When a VGA device is behind such a host bridge and the resource is >> translated efifb driver is trying to do ioremap against bus address >> rather than the resource address and is failing to probe. >> >> efifb: probing for efifb >> efifb: cannot reserve video memory at 0x1e000000 >> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >> efifb: mode is 800x600x32, linelength=3200, pages=1 >> efifb: scrolling: redraw >> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >> >> Use the host bridge offset information to convert bus address to >> resource address in the fixup. >> >> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > > Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > > Bartlomiej, could you please take these via the fbdev tree for v4.19? > Peter already gave his ack but Sinan dropped it (presumably because of > the split in v2) > > Sinan, does this need to go to -stable? I.e., has it ever worked > before? > Yes, it needs to go to stable. It never worked before. Issue was found during a graphics bring up with AST driver. It needs a 32 bit graphics card and a 32 bit translating host bridge to hit the issue. > > >> --- >> drivers/video/fbdev/efifb.c | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c >> index 6daac8d..429cc85 100644 >> --- a/drivers/video/fbdev/efifb.c >> +++ b/drivers/video/fbdev/efifb.c >> @@ -431,6 +431,7 @@ static void efifb_fixup_resources(struct pci_dev >> *dev) >> .end = screen_info.lfb_base + screen_info.lfb_size - >> 1, >> .flags = IORESOURCE_MEM, >> }; >> + struct pci_bus_region region; >> int i; >> >> if (efifb_pci_dev || screen_info.orig_video_isVGA != >> VIDEO_TYPE_EFI) >> @@ -442,6 +443,10 @@ static void efifb_fixup_resources(struct pci_dev >> *dev) >> if (!screen_res.start) >> return; >> >> + region.start = screen_res.start; >> + region.end = screen_res.end; >> + pcibios_bus_to_resource(dev->bus, &screen_res, ®ion); >> + >> for (i = 0; i <= PCI_STD_RESOURCE_END; i++) { >> struct resource *res = &dev->resource[i]; >> >> -- >> 2.7.4 >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wednesday, June 13, 2018 05:45:48 PM Ard Biesheuvel wrote: > On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: > > A host bridge is allowed to remap BAR addresses using _TRA attribute in > > _CRS windows. > > > > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > > > > When a VGA device is behind such a host bridge and the resource is > > translated efifb driver is trying to do ioremap against bus address > > rather than the resource address and is failing to probe. > > > > efifb: probing for efifb > > efifb: cannot reserve video memory at 0x1e000000 > > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > > efifb: mode is 800x600x32, linelength=3200, pages=1 > > efifb: scrolling: redraw > > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > > > > Use the host bridge offset information to convert bus address to > > resource address in the fixup. > > > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > > Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > > Bartlomiej, could you please take these via the fbdev tree for v4.19? Sure, I will queue it after the current merge window. > Peter already gave his ack but Sinan dropped it (presumably because of > the split in v2) Peter, can I (re)add your ACK to V2 patches? Best regards, -- Bartlomiej Zolnierkiewicz Samsung R&D Institute Poland Samsung Electronics -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Minor subject nit: From the caller's point of view, we must convert a bus address to a resource *always* (the caller has no knowledge of "whether it is translated by the host bridge"). On Fri, May 18, 2018 at 10:17:51AM -0400, Sinan Kaya wrote: > A host bridge is allowed to remap BAR addresses using _TRA attribute in > _CRS windows. Also, _TRA/_CRS are ACPI-specific terms and non-ACPI host bridges can also do the same sort of translation. Another trivial nit. > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > > When a VGA device is behind such a host bridge and the resource is > translated efifb driver is trying to do ioremap against bus address > rather than the resource address and is failing to probe. > > efifb: probing for efifb > efifb: cannot reserve video memory at 0x1e000000 > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > efifb: mode is 800x600x32, linelength=3200, pages=1 > efifb: scrolling: redraw > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > > Use the host bridge offset information to convert bus address to > resource address in the fixup. > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Thanks a lot for fixing this! > --- > drivers/video/fbdev/efifb.c | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c > index 6daac8d..429cc85 100644 > --- a/drivers/video/fbdev/efifb.c > +++ b/drivers/video/fbdev/efifb.c > @@ -431,6 +431,7 @@ static void efifb_fixup_resources(struct pci_dev *dev) > .end = screen_info.lfb_base + screen_info.lfb_size - 1, > .flags = IORESOURCE_MEM, > }; > + struct pci_bus_region region; > int i; > > if (efifb_pci_dev || screen_info.orig_video_isVGA != VIDEO_TYPE_EFI) > @@ -442,6 +443,10 @@ static void efifb_fixup_resources(struct pci_dev *dev) > if (!screen_res.start) > return; > > + region.start = screen_res.start; > + region.end = screen_res.end; > + pcibios_bus_to_resource(dev->bus, &screen_res, ®ion); > + > for (i = 0; i <= PCI_STD_RESOURCE_END; i++) { > struct resource *res = &dev->resource[i]; > > -- > 2.7.4 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 13 June 2018 at 18:08, Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> wrote: > On Wednesday, June 13, 2018 05:45:48 PM Ard Biesheuvel wrote: >> On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: >> > A host bridge is allowed to remap BAR addresses using _TRA attribute in >> > _CRS windows. >> > >> > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) >> > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >> > >> > When a VGA device is behind such a host bridge and the resource is >> > translated efifb driver is trying to do ioremap against bus address >> > rather than the resource address and is failing to probe. >> > >> > efifb: probing for efifb >> > efifb: cannot reserve video memory at 0x1e000000 >> > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >> > efifb: mode is 800x600x32, linelength=3200, pages=1 >> > efifb: scrolling: redraw >> > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >> > >> > Use the host bridge offset information to convert bus address to >> > resource address in the fixup. >> > >> > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >> >> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> >> >> Bartlomiej, could you please take these via the fbdev tree for v4.19? > > Sure, I will queue it after the current merge window. > >> Peter already gave his ack but Sinan dropped it (presumably because of >> the split in v2) > > Peter, can I (re)add your ACK to V2 patches? > Actually, it would be better if we could take this through the EFI tree instead, with your ack. Would you mind? There are some other efifb changes coming up, some of which depend on core EFI changes, and taking these through different trees is going to be more trouble than it's worth. https://marc.info/?l=linux-efi&m=152929425329015&w=2 from Hans, and the series [PATCH v2 0/2] efi: add support for cacheable efifb mappings that I just cc'ed you on a minute ago. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Friday, June 22, 2018 09:54:22 AM Ard Biesheuvel wrote: > On 13 June 2018 at 18:08, Bartlomiej Zolnierkiewicz > <b.zolnierkie@samsung.com> wrote: > > On Wednesday, June 13, 2018 05:45:48 PM Ard Biesheuvel wrote: > >> On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: > >> > A host bridge is allowed to remap BAR addresses using _TRA attribute in > >> > _CRS windows. > >> > > >> > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > >> > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > >> > > >> > When a VGA device is behind such a host bridge and the resource is > >> > translated efifb driver is trying to do ioremap against bus address > >> > rather than the resource address and is failing to probe. > >> > > >> > efifb: probing for efifb > >> > efifb: cannot reserve video memory at 0x1e000000 > >> > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > >> > efifb: mode is 800x600x32, linelength=3200, pages=1 > >> > efifb: scrolling: redraw > >> > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > >> > > >> > Use the host bridge offset information to convert bus address to > >> > resource address in the fixup. > >> > > >> > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > >> > >> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > >> > >> Bartlomiej, could you please take these via the fbdev tree for v4.19? > > > > Sure, I will queue it after the current merge window. > > > >> Peter already gave his ack but Sinan dropped it (presumably because of > >> the split in v2) > > > > Peter, can I (re)add your ACK to V2 patches? > > > > Actually, it would be better if we could take this through the EFI > tree instead, with your ack. Would you mind? Fine with me. > There are some other efifb changes coming up, some of which depend on > core EFI changes, and taking these through different trees is going to > be more trouble than it's worth. Best regards, -- Bartlomiej Zolnierkiewicz Samsung R&D Institute Poland Samsung Electronics -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Friday, June 22, 2018 12:07:48 PM Bartlomiej Zolnierkiewicz wrote: > On Friday, June 22, 2018 09:54:22 AM Ard Biesheuvel wrote: > > On 13 June 2018 at 18:08, Bartlomiej Zolnierkiewicz > > <b.zolnierkie@samsung.com> wrote: > > > On Wednesday, June 13, 2018 05:45:48 PM Ard Biesheuvel wrote: > > >> On 18 May 2018 at 16:17, Sinan Kaya <okaya@codeaurora.org> wrote: > > >> > A host bridge is allowed to remap BAR addresses using _TRA attribute in > > >> > _CRS windows. > > >> > > > >> > pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) > > >> > pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] > > >> > > > >> > When a VGA device is behind such a host bridge and the resource is > > >> > translated efifb driver is trying to do ioremap against bus address > > >> > rather than the resource address and is failing to probe. > > >> > > > >> > efifb: probing for efifb > > >> > efifb: cannot reserve video memory at 0x1e000000 > > >> > efifb: framebuffer at 0x1e000000, using 1920k, total 1875k > > >> > efifb: mode is 800x600x32, linelength=3200, pages=1 > > >> > efifb: scrolling: redraw > > >> > efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 > > >> > > > >> > Use the host bridge offset information to convert bus address to > > >> > resource address in the fixup. > > >> > > > >> > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > > >> > > >> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > > >> > > >> Bartlomiej, could you please take these via the fbdev tree for v4.19? > > > > > > Sure, I will queue it after the current merge window. > > > > > >> Peter already gave his ack but Sinan dropped it (presumably because of > > >> the split in v2) > > > > > > Peter, can I (re)add your ACK to V2 patches? > > > > > > > Actually, it would be better if we could take this through the EFI > > tree instead, with your ack. Would you mind? > > Fine with me. Acked-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> Best regards, -- Bartlomiej Zolnierkiewicz Samsung R&D Institute Poland Samsung Electronics -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 20 June 2018 at 00:29, Bjorn Helgaas <helgaas@kernel.org> wrote: > Minor subject nit: From the caller's point of view, we must convert a bus > address to a resource *always* (the caller has no knowledge of "whether it > is translated by the host bridge"). > > On Fri, May 18, 2018 at 10:17:51AM -0400, Sinan Kaya wrote: >> A host bridge is allowed to remap BAR addresses using _TRA attribute in >> _CRS windows. > > Also, _TRA/_CRS are ACPI-specific terms and non-ACPI host bridges can > also do the same sort of translation. Another trivial nit. > >> pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) >> pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] >> >> When a VGA device is behind such a host bridge and the resource is >> translated efifb driver is trying to do ioremap against bus address >> rather than the resource address and is failing to probe. >> >> efifb: probing for efifb >> efifb: cannot reserve video memory at 0x1e000000 >> efifb: framebuffer at 0x1e000000, using 1920k, total 1875k >> efifb: mode is 800x600x32, linelength=3200, pages=1 >> efifb: scrolling: redraw >> efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 >> >> Use the host bridge offset information to convert bus address to >> resource address in the fixup. >> >> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > > Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> > > Thanks a lot for fixing this! > Apologies for only bringing this up now, but I think this patch is wrong after all. screen_info.lfb_base is supposed to be a CPU address, and so translating it like this is wrong. If you end up with a PCI address here, you have made a mistake in hacking support for PCI outbound translations into UEFI. Other users such as UEFI itself or GRUB will treat this as a CPU physical address as well, so the kernel should not treat it any differently. >> --- >> drivers/video/fbdev/efifb.c | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c >> index 6daac8d..429cc85 100644 >> --- a/drivers/video/fbdev/efifb.c >> +++ b/drivers/video/fbdev/efifb.c >> @@ -431,6 +431,7 @@ static void efifb_fixup_resources(struct pci_dev *dev) >> .end = screen_info.lfb_base + screen_info.lfb_size - 1, >> .flags = IORESOURCE_MEM, >> }; >> + struct pci_bus_region region; >> int i; >> >> if (efifb_pci_dev || screen_info.orig_video_isVGA != VIDEO_TYPE_EFI) >> @@ -442,6 +443,10 @@ static void efifb_fixup_resources(struct pci_dev *dev) >> if (!screen_res.start) >> return; >> >> + region.start = screen_res.start; >> + region.end = screen_res.end; >> + pcibios_bus_to_resource(dev->bus, &screen_res, ®ion); >> + >> for (i = 0; i <= PCI_STD_RESOURCE_END; i++) { >> struct resource *res = &dev->resource[i]; >> >> -- >> 2.7.4 >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Ard, On 6/22/2018 7:21 AM, Ard Biesheuvel wrote: > Apologies for only bringing this up now, but I think this patch is > wrong after all. > > screen_info.lfb_base is supposed to be a CPU address, and so > translating it like this is wrong. If you end up with a PCI address > here, you have made a mistake in hacking support for PCI outbound > translations into UEFI. Other users such as UEFI itself or GRUB will > treat this as a CPU physical address as well, so the kernel should not > treat it any differently. The behavior I'm seeing is from a UEFI BIOS vendor. I did not write the code for it... I was asked to debug it. I'd like to dive into your statement about UEFI and GRUB using this address as physical addresses. AFAIK, all PCI outbound requests go through PCI IO protocol in UEFI and the translation information is hidden inside the UEFI PCI Host Bridge driver. Drivers are not allowed to access PCI resources directly especially as a memory mapped address. This particular vendor is programming the BAR address into the GOP protocol. Since the host bridge driver is doing a translation, we are hitting this issue. Is there a UEFI spec reference about the definition of this field? Sinan
On 22 June 2018 at 15:52, Sinan Kaya <okaya@codeaurora.org> wrote: > Hi Ard, > > On 6/22/2018 7:21 AM, Ard Biesheuvel wrote: >> Apologies for only bringing this up now, but I think this patch is >> wrong after all. >> >> screen_info.lfb_base is supposed to be a CPU address, and so >> translating it like this is wrong. If you end up with a PCI address >> here, you have made a mistake in hacking support for PCI outbound >> translations into UEFI. Other users such as UEFI itself or GRUB will >> treat this as a CPU physical address as well, so the kernel should not >> treat it any differently. > > The behavior I'm seeing is from a UEFI BIOS vendor. I did not write the > code for it... > > I was asked to debug it. > > I'd like to dive into your statement about UEFI and GRUB using this address > as physical addresses. > > AFAIK, all PCI outbound requests go through PCI IO protocol in UEFI and the > translation information is hidden inside the UEFI PCI Host Bridge driver. > Drivers are not allowed to access PCI resources directly especially as a > memory mapped address. > > This particular vendor is programming the BAR address into the GOP protocol. > Since the host bridge driver is doing a translation, we are hitting this > issue. > > Is there a UEFI spec reference about the definition of this field? > Yes, it is part of the PCI I/O protocol definition. FrameBufferBase is described as """ Base address of graphics linear frame buffer. Info contains information required to allow software to draw directly to the frame buffer without using Blt().Offset zero in FrameBufferBase represents the upper left pixel of the display. """ -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 22 June 2018 at 15:55, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > On 22 June 2018 at 15:52, Sinan Kaya <okaya@codeaurora.org> wrote: >> Hi Ard, >> >> On 6/22/2018 7:21 AM, Ard Biesheuvel wrote: >>> Apologies for only bringing this up now, but I think this patch is >>> wrong after all. >>> >>> screen_info.lfb_base is supposed to be a CPU address, and so >>> translating it like this is wrong. If you end up with a PCI address >>> here, you have made a mistake in hacking support for PCI outbound >>> translations into UEFI. Other users such as UEFI itself or GRUB will >>> treat this as a CPU physical address as well, so the kernel should not >>> treat it any differently. >> >> The behavior I'm seeing is from a UEFI BIOS vendor. I did not write the >> code for it... >> >> I was asked to debug it. >> >> I'd like to dive into your statement about UEFI and GRUB using this address >> as physical addresses. >> >> AFAIK, all PCI outbound requests go through PCI IO protocol in UEFI and the >> translation information is hidden inside the UEFI PCI Host Bridge driver. >> Drivers are not allowed to access PCI resources directly especially as a >> memory mapped address. >> >> This particular vendor is programming the BAR address into the GOP protocol. >> Since the host bridge driver is doing a translation, we are hitting this >> issue. >> >> Is there a UEFI spec reference about the definition of this field? >> > > Yes, it is part of the PCI I/O protocol definition. FrameBufferBase is > described as > > """ > Base address of graphics linear frame buffer. Info contains > information required to allow software to draw directly to the > frame buffer without using Blt().Offset zero in > FrameBufferBase represents the upper left pixel of the > display. > """ I just tried AMD Radeon and NVidia graphics cards on a system with non-1:1 mapped MMIO windows, and in both cases, the GOP protocol structure is populated correctly, i.e., using the CPU address not the PCIe address. EDK2 only recently gained support for MMIO translation in the host bridge driver, so I so wonder if this is a platform issue rather than a driver issue. It may be worth a try to dump the results of GetBarAttributes() of all PCI I/O protocol instances (either in UEFI or in the stub), to double check that the correct values are returned. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 6/22/2018 2:01 PM, Ard Biesheuvel wrote: >> Yes, it is part of the PCI I/O protocol definition. FrameBufferBase is >> described as >> >> """ >> Base address of graphics linear frame buffer. Info contains >> information required to allow software to draw directly to the >> frame buffer without using Blt().Offset zero in >> FrameBufferBase represents the upper left pixel of the >> display. >> """ > I just tried AMD Radeon and NVidia graphics cards on a system with > non-1:1 mapped MMIO windows, and in both cases, the GOP protocol > structure is populated correctly, i.e., using the CPU address not the > PCIe address. > > EDK2 only recently gained support for MMIO translation in the host > bridge driver, so I so wonder if this is a platform issue rather than > a driver issue. It may be worth a try to dump the results of > GetBarAttributes() of all PCI I/O protocol instances (either in UEFI > or in the stub), to double check that the correct values are returned. > Thanks for checking out other platforms. I'll mark the issue as a BIOS issue and bounce your feedback to the BIOS provider. Let's hold onto this patch for the moment.
On 22 June 2018 at 20:30, Sinan Kaya <okaya@codeaurora.org> wrote: > On 6/22/2018 2:01 PM, Ard Biesheuvel wrote: >>> Yes, it is part of the PCI I/O protocol definition. FrameBufferBase is >>> described as >>> >>> """ >>> Base address of graphics linear frame buffer. Info contains >>> information required to allow software to draw directly to the >>> frame buffer without using Blt().Offset zero in >>> FrameBufferBase represents the upper left pixel of the >>> display. >>> """ >> I just tried AMD Radeon and NVidia graphics cards on a system with >> non-1:1 mapped MMIO windows, and in both cases, the GOP protocol >> structure is populated correctly, i.e., using the CPU address not the >> PCIe address. >> >> EDK2 only recently gained support for MMIO translation in the host >> bridge driver, so I so wonder if this is a platform issue rather than >> a driver issue. It may be worth a try to dump the results of >> GetBarAttributes() of all PCI I/O protocol instances (either in UEFI >> or in the stub), to double check that the correct values are returned. >> > > Thanks for checking out other platforms. I'll mark the issue as a BIOS > issue and bounce your feedback to the BIOS provider. > I screwed up my testing, unfortunately. Both the public AMD GOP driver I tried, and the Nvidia GT218 under x86 emulation break when using MMIO translation. However, GraphicsOutputDxe in the EDK2 tree gets it right, and uses PciIo->GetBarAttributes() to get the address of the framebuffer region, which will return the CPU address not the PCI address. > Let's hold onto this patch for the moment. > Yes. I'd like to get this resolved as well, but if the drivers are dereferencing BAR values as CPU addresses, this is unlikely to be the only thing that is broken under outbound translation. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 22 June 2018 at 21:29, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > On 22 June 2018 at 20:30, Sinan Kaya <okaya@codeaurora.org> wrote: >> On 6/22/2018 2:01 PM, Ard Biesheuvel wrote: >>>> Yes, it is part of the PCI I/O protocol definition. FrameBufferBase is >>>> described as >>>> >>>> """ >>>> Base address of graphics linear frame buffer. Info contains >>>> information required to allow software to draw directly to the >>>> frame buffer without using Blt().Offset zero in >>>> FrameBufferBase represents the upper left pixel of the >>>> display. >>>> """ >>> I just tried AMD Radeon and NVidia graphics cards on a system with >>> non-1:1 mapped MMIO windows, and in both cases, the GOP protocol >>> structure is populated correctly, i.e., using the CPU address not the >>> PCIe address. >>> >>> EDK2 only recently gained support for MMIO translation in the host >>> bridge driver, so I so wonder if this is a platform issue rather than >>> a driver issue. It may be worth a try to dump the results of >>> GetBarAttributes() of all PCI I/O protocol instances (either in UEFI >>> or in the stub), to double check that the correct values are returned. >>> >> >> Thanks for checking out other platforms. I'll mark the issue as a BIOS >> issue and bounce your feedback to the BIOS provider. >> > > I screwed up my testing, unfortunately. Both the public AMD GOP driver > I tried, and the Nvidia GT218 under x86 emulation break when using > MMIO translation. However, GraphicsOutputDxe in the EDK2 tree gets it > right, and uses PciIo->GetBarAttributes() to get the address of the > framebuffer region, which will return the CPU address not the PCI > address. > >> Let's hold onto this patch for the moment. >> > > Yes. I'd like to get this resolved as well, but if the drivers are > dereferencing BAR values as CPU addresses, this is unlikely to be the > only thing that is broken under outbound translation. Note that this was fixed fairly recently in EDK2, so BIOS vendors providing UEFI firmware for ARM platforms with outbound MMIO translation should probably incorporate this EDK2 patch commit dc080d3b61e570e7a3163fc24afa6f8388d0c0bf Author: Heyi Guo <heyi.guo@linaro.org> Date: Thu Feb 8 11:13:27 2018 +0800 MdeModulePkg/PciBus: return CPU address for GetBarAttributes According to UEFI spec 2.7, PciIo->GetBarAttributes should return host address (CPU view ddress) rather than device address (PCI view address), and device address = host address + address translation offset, so we subtract translation from device address before returning. Note that this is not the only MMIO translation related change made by Heyi Guo to the generic PCI host bridge and bus drivers, but given that those did not support MMIO translation at all, I take it your affected platforms will already have their own changes to accommodate this. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2018-06-25 04:20, Ard Biesheuvel wrote: > On 22 June 2018 at 21:29, Ard Biesheuvel <ard.biesheuvel@linaro.org> > wrote: >> On 22 June 2018 at 20:30, Sinan Kaya <okaya@codeaurora.org> wrote: >>> On 6/22/2018 2:01 PM, Ard Biesheuvel wrote: >>>>> Yes, it is part of the PCI I/O protocol definition. FrameBufferBase >>>>> is >>>>> described as >>>>> >>>>> """ >>>>> Base address of graphics linear frame buffer. Info contains >>>>> information required to allow software to draw directly to the >>>>> frame buffer without using Blt().Offset zero in >>>>> FrameBufferBase represents the upper left pixel of the >>>>> display. >>>>> """ >>>> I just tried AMD Radeon and NVidia graphics cards on a system with >>>> non-1:1 mapped MMIO windows, and in both cases, the GOP protocol >>>> structure is populated correctly, i.e., using the CPU address not >>>> the >>>> PCIe address. >>>> >>>> EDK2 only recently gained support for MMIO translation in the host >>>> bridge driver, so I so wonder if this is a platform issue rather >>>> than >>>> a driver issue. It may be worth a try to dump the results of >>>> GetBarAttributes() of all PCI I/O protocol instances (either in UEFI >>>> or in the stub), to double check that the correct values are >>>> returned. >>>> >>> >>> Thanks for checking out other platforms. I'll mark the issue as a >>> BIOS >>> issue and bounce your feedback to the BIOS provider. >>> >> >> I screwed up my testing, unfortunately. Both the public AMD GOP driver >> I tried, and the Nvidia GT218 under x86 emulation break when using >> MMIO translation. However, GraphicsOutputDxe in the EDK2 tree gets it >> right, and uses PciIo->GetBarAttributes() to get the address of the >> framebuffer region, which will return the CPU address not the PCI >> address. >> >>> Let's hold onto this patch for the moment. >>> >> >> Yes. I'd like to get this resolved as well, but if the drivers are >> dereferencing BAR values as CPU addresses, this is unlikely to be the >> only thing that is broken under outbound translation. > > Note that this was fixed fairly recently in EDK2, so BIOS vendors > providing UEFI firmware for ARM platforms with outbound MMIO > translation should probably incorporate this EDK2 patch > > commit dc080d3b61e570e7a3163fc24afa6f8388d0c0bf > Author: Heyi Guo <heyi.guo@linaro.org> > Date: Thu Feb 8 11:13:27 2018 +0800 > > MdeModulePkg/PciBus: return CPU address for GetBarAttributes > > According to UEFI spec 2.7, PciIo->GetBarAttributes should return > host > address (CPU view ddress) rather than device address (PCI view > address), and > device address = host address + address translation offset, > so we subtract translation from device address before returning. > > Note that this is not the only MMIO translation related change made by > Heyi Guo to the generic PCI host bridge and bus drivers, but given > that those did not support MMIO translation at all, I take it your > affected platforms will already have their own changes to accommodate > this. Platform has been doing mmio translation for quite a while. Because all accesses go through pci io protocol, the rest of the UEFI never needed to be aware of bar address or do direct access. This is the first time I hear of direct access. Maybe, GOP is a special case. I started copying your response to the bios vendor. They are probably missing that patch. I will pass it along. -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 6/25/2018 11:52 AM, okaya@codeaurora.org wrote: >> Note that this is not the only MMIO translation related change made by >> Heyi Guo to the generic PCI host bridge and bus drivers, but given >> that those did not support MMIO translation at all, I take it your >> affected platforms will already have their own changes to accommodate >> this. > > Platform has been doing mmio translation for quite a while. Because all accesses go through pci io protocol, the rest of the UEFI never needed to be aware of bar address or do direct access. > > This is the first time I hear of direct access. Maybe, GOP is a special case. > > I started copying your response to the bios vendor. > > They are probably missing that patch. I will pass it along. I was informed that they fixed the issue in BIOS by presenting CPU physical address instead of PCI bus address in FrameBufferBase of the AST GOP UEFI driver. We can drop this patch now.
On 25 June 2018 at 19:28, Sinan Kaya <okaya@codeaurora.org> wrote: > On 6/25/2018 11:52 AM, okaya@codeaurora.org wrote: >>> Note that this is not the only MMIO translation related change made by >>> Heyi Guo to the generic PCI host bridge and bus drivers, but given >>> that those did not support MMIO translation at all, I take it your >>> affected platforms will already have their own changes to accommodate >>> this. >> >> Platform has been doing mmio translation for quite a while. Because all accesses go through pci io protocol, the rest of the UEFI never needed to be aware of bar address or do direct access. >> >> This is the first time I hear of direct access. Maybe, GOP is a special case. >> >> I started copying your response to the bios vendor. >> >> They are probably missing that patch. I will pass it along. > > I was informed that they fixed the issue in BIOS by presenting CPU physical > address instead of PCI bus address in FrameBufferBase of the AST GOP UEFI driver. > > We can drop this patch now. > Excellent! Thanks for following up with the vendor ... -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 6/25/2018 1:29 PM, Ard Biesheuvel wrote: >>> Platform has been doing mmio translation for quite a while. Because all accesses go through pci io protocol, the rest of the UEFI never needed to be aware of bar address or do direct access. >>> >>> This is the first time I hear of direct access. Maybe, GOP is a special case. >>> >>> I started copying your response to the bios vendor. >>> >>> They are probably missing that patch. I will pass it along. >> I was informed that they fixed the issue in BIOS by presenting CPU physical >> address instead of PCI bus address in FrameBufferBase of the AST GOP UEFI driver. >> >> We can drop this patch now. >> > Excellent! Thanks for following up with the vendor ... > Sure, appreciate the help.
diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c index 6daac8d..429cc85 100644 --- a/drivers/video/fbdev/efifb.c +++ b/drivers/video/fbdev/efifb.c @@ -431,6 +431,7 @@ static void efifb_fixup_resources(struct pci_dev *dev) .end = screen_info.lfb_base + screen_info.lfb_size - 1, .flags = IORESOURCE_MEM, }; + struct pci_bus_region region; int i; if (efifb_pci_dev || screen_info.orig_video_isVGA != VIDEO_TYPE_EFI) @@ -442,6 +443,10 @@ static void efifb_fixup_resources(struct pci_dev *dev) if (!screen_res.start) return; + region.start = screen_res.start; + region.end = screen_res.end; + pcibios_bus_to_resource(dev->bus, &screen_res, ®ion); + for (i = 0; i <= PCI_STD_RESOURCE_END; i++) { struct resource *res = &dev->resource[i];
A host bridge is allowed to remap BAR addresses using _TRA attribute in _CRS windows. pci_bus 0000:00: root bus resource [mem 0x80100100000-0x8011fffffff window] (bus address [0x00100000-0x1fffffff]) pci 0000:02:00.0: reg 0x10: [mem 0x8011e000000-0x8011effffff] When a VGA device is behind such a host bridge and the resource is translated efifb driver is trying to do ioremap against bus address rather than the resource address and is failing to probe. efifb: probing for efifb efifb: cannot reserve video memory at 0x1e000000 efifb: framebuffer at 0x1e000000, using 1920k, total 1875k efifb: mode is 800x600x32, linelength=3200, pages=1 efifb: scrolling: redraw efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 Use the host bridge offset information to convert bus address to resource address in the fixup. Signed-off-by: Sinan Kaya <okaya@codeaurora.org> --- drivers/video/fbdev/efifb.c | 5 +++++ 1 file changed, 5 insertions(+)