| Message ID | 20201106170036.18713-4-logang@deltatee.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Userspace P2PDMA with O_DIRECT NVMe devices | expand |
On Fri, Nov 06, 2020 at 10:00:24AM -0700, Logan Gunthorpe wrote: > Introduce pci_p2pdma_should_map_bus() which is meant to be called by > dma map functions to determine how to map a given p2pdma page. s/dma/DMA/ for consistency (also below in function comment) > pci_p2pdma_bus_offset() is also added to allow callers to get the bus > offset if they need to map the bus address. > > Signed-off-by: Logan Gunthorpe <logang@deltatee.com> > --- > drivers/pci/p2pdma.c | 46 ++++++++++++++++++++++++++++++++++++++ > include/linux/pci-p2pdma.h | 11 +++++++++ > 2 files changed, 57 insertions(+) > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > index ea8472278b11..9961e779f430 100644 > --- a/drivers/pci/p2pdma.c > +++ b/drivers/pci/p2pdma.c > @@ -930,6 +930,52 @@ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, > } > EXPORT_SYMBOL_GPL(pci_p2pdma_unmap_sg_attrs); > > +/** > + * pci_p2pdma_bus_offset - returns the bus offset for a given page > + * @page: page to get the offset for > + * > + * Must be passed a pci p2pdma page. s/pci/PCI/ > + */ > +u64 pci_p2pdma_bus_offset(struct page *page) > +{ > + struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page->pgmap); > + > + WARN_ON(!is_pci_p2pdma_page(page)); > + > + return p2p_pgmap->bus_offset; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_bus_offset); > + > +/** > + * pci_p2pdma_should_map_bus - determine if a dma mapping should use the > + * bus address > + * @dev: device doing the DMA request > + * @pgmap: dev_pagemap structure for the mapping > + * > + * Returns 1 if the page should be mapped with a bus address, 0 otherwise > + * and -1 the device should not be mapping P2PDMA pages. I think this is missing a word. I'm not really sure how to interpret the "should" in pci_p2pdma_should_map_bus(). If this returns -1, does that mean the patches *cannot* be mapped? They *could* be mapped, but you really *shouldn't*? Something else? 1 means page should be mapped with bus address. 0 means ... what, exactly? It should be mapped with some different address? Sorry these are naive questions because I don't know how all this works. > + */ > +int pci_p2pdma_should_map_bus(struct device *dev, struct dev_pagemap *pgmap) > +{ > + struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(pgmap); > + struct pci_dev *client; > + > + if (!dev_is_pci(dev)) > + return -1; > + > + client = to_pci_dev(dev); > + > + switch (pci_p2pdma_map_type(p2p_pgmap->provider, client)) { > + case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: > + return 0; > + case PCI_P2PDMA_MAP_BUS_ADDR: > + return 1; > + default: > + return -1; > + } > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_should_map_bus); > + > /** > * pci_p2pdma_enable_store - parse a configfs/sysfs attribute store > * to enable p2pdma > diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h > index 8318a97c9c61..fc5de47eeac4 100644 > --- a/include/linux/pci-p2pdma.h > +++ b/include/linux/pci-p2pdma.h > @@ -34,6 +34,8 @@ int pci_p2pdma_map_sg_attrs(struct device *dev, struct scatterlist *sg, > int nents, enum dma_data_direction dir, unsigned long attrs); > void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, > int nents, enum dma_data_direction dir, unsigned long attrs); > +u64 pci_p2pdma_bus_offset(struct page *page); > +int pci_p2pdma_should_map_bus(struct device *dev, struct dev_pagemap *pgmap); > int pci_p2pdma_enable_store(const char *page, struct pci_dev **p2p_dev, > bool *use_p2pdma); > ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev, > @@ -83,6 +85,15 @@ static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, > static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) > { > } > +static inline u64 pci_p2pdma_bus_offset(struct page *page) > +{ > + return -1; > +} > +static inline int pci_p2pdma_should_map_bus(struct device *dev, > + struct dev_pagemap *pgmap) > +{ > + return -1; > +} > static inline int pci_p2pdma_map_sg_attrs(struct device *dev, > struct scatterlist *sg, int nents, enum dma_data_direction dir, > unsigned long attrs) > -- > 2.20.1 >
On 2020-11-10 4:25 p.m., Bjorn Helgaas wrote: > On Fri, Nov 06, 2020 at 10:00:24AM -0700, Logan Gunthorpe wrote: >> Introduce pci_p2pdma_should_map_bus() which is meant to be called by >> dma map functions to determine how to map a given p2pdma page. > > s/dma/DMA/ for consistency (also below in function comment) > >> pci_p2pdma_bus_offset() is also added to allow callers to get the bus >> offset if they need to map the bus address. >> >> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> >> --- >> drivers/pci/p2pdma.c | 46 ++++++++++++++++++++++++++++++++++++++ >> include/linux/pci-p2pdma.h | 11 +++++++++ >> 2 files changed, 57 insertions(+) >> >> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c >> index ea8472278b11..9961e779f430 100644 >> --- a/drivers/pci/p2pdma.c >> +++ b/drivers/pci/p2pdma.c >> @@ -930,6 +930,52 @@ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, >> } >> EXPORT_SYMBOL_GPL(pci_p2pdma_unmap_sg_attrs); >> >> +/** >> + * pci_p2pdma_bus_offset - returns the bus offset for a given page >> + * @page: page to get the offset for >> + * >> + * Must be passed a pci p2pdma page. > > s/pci/PCI/ > >> + */ >> +u64 pci_p2pdma_bus_offset(struct page *page) >> +{ >> + struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page->pgmap); >> + >> + WARN_ON(!is_pci_p2pdma_page(page)); >> + >> + return p2p_pgmap->bus_offset; >> +} >> +EXPORT_SYMBOL_GPL(pci_p2pdma_bus_offset); >> + >> +/** >> + * pci_p2pdma_should_map_bus - determine if a dma mapping should use the >> + * bus address >> + * @dev: device doing the DMA request >> + * @pgmap: dev_pagemap structure for the mapping >> + * >> + * Returns 1 if the page should be mapped with a bus address, 0 otherwise >> + * and -1 the device should not be mapping P2PDMA pages. > > I think this is missing a word. > > I'm not really sure how to interpret the "should" in > pci_p2pdma_should_map_bus(). If this returns -1, does that mean the > patches *cannot* be mapped? They *could* be mapped, but you really > *shouldn't*? Something else? > > 1 means page should be mapped with bus address. 0 means ... what, > exactly? It should be mapped with some different address? 1 means it must be mapped with a bus address 0 means it may be mapped normally (through the IOMMU or just with a direct physical address) -1 means it cannot be mapped and should fail (ie. if it must go through the IOMMU, but the IOMMU is not in the whitelist). > Sorry these are naive questions because I don't know how all this > works. Thanks for the review. Definitely points out some questionable language that I used. I'll reword this if/when it goes further. Logan
diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index ea8472278b11..9961e779f430 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -930,6 +930,52 @@ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, } EXPORT_SYMBOL_GPL(pci_p2pdma_unmap_sg_attrs); +/** + * pci_p2pdma_bus_offset - returns the bus offset for a given page + * @page: page to get the offset for + * + * Must be passed a pci p2pdma page. + */ +u64 pci_p2pdma_bus_offset(struct page *page) +{ + struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page->pgmap); + + WARN_ON(!is_pci_p2pdma_page(page)); + + return p2p_pgmap->bus_offset; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_bus_offset); + +/** + * pci_p2pdma_should_map_bus - determine if a dma mapping should use the + * bus address + * @dev: device doing the DMA request + * @pgmap: dev_pagemap structure for the mapping + * + * Returns 1 if the page should be mapped with a bus address, 0 otherwise + * and -1 the device should not be mapping P2PDMA pages. + */ +int pci_p2pdma_should_map_bus(struct device *dev, struct dev_pagemap *pgmap) +{ + struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(pgmap); + struct pci_dev *client; + + if (!dev_is_pci(dev)) + return -1; + + client = to_pci_dev(dev); + + switch (pci_p2pdma_map_type(p2p_pgmap->provider, client)) { + case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: + return 0; + case PCI_P2PDMA_MAP_BUS_ADDR: + return 1; + default: + return -1; + } +} +EXPORT_SYMBOL_GPL(pci_p2pdma_should_map_bus); + /** * pci_p2pdma_enable_store - parse a configfs/sysfs attribute store * to enable p2pdma diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index 8318a97c9c61..fc5de47eeac4 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -34,6 +34,8 @@ int pci_p2pdma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs); void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs); +u64 pci_p2pdma_bus_offset(struct page *page); +int pci_p2pdma_should_map_bus(struct device *dev, struct dev_pagemap *pgmap); int pci_p2pdma_enable_store(const char *page, struct pci_dev **p2p_dev, bool *use_p2pdma); ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev, @@ -83,6 +85,15 @@ static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) { } +static inline u64 pci_p2pdma_bus_offset(struct page *page) +{ + return -1; +} +static inline int pci_p2pdma_should_map_bus(struct device *dev, + struct dev_pagemap *pgmap) +{ + return -1; +} static inline int pci_p2pdma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs)
Introduce pci_p2pdma_should_map_bus() which is meant to be called by dma map functions to determine how to map a given p2pdma page. pci_p2pdma_bus_offset() is also added to allow callers to get the bus offset if they need to map the bus address. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- drivers/pci/p2pdma.c | 46 ++++++++++++++++++++++++++++++++++++++ include/linux/pci-p2pdma.h | 11 +++++++++ 2 files changed, 57 insertions(+)