| Message ID | 1476278544-3397-5-git-send-email-eric.auger@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Eric, One query and a comment below. Eric Auger <eric.auger@redhat.com> writes: > We introduce the capability to (un)register MSI doorbells. > > A doorbell region is characterized by its physical address base, size, > and whether it its safe (ie. it implements IRQ remapping). A doorbell > can be per-cpu or global. We currently only care about global doorbells. > > A function returns whether all registered doorbells are safe. > > MSI controllers likely to work along with IOMMU that translate MSI > transaction must register their doorbells to allow device assignment > with MSI support. Otherwise the MSI transactions will cause IOMMU faults. > > Signed-off-by: Eric Auger <eric.auger@redhat.com> > > --- > > v13 -> v14: > - previously in msi-doorbell.h/c > --- > drivers/iommu/dma-iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++ > include/linux/dma-iommu.h | 41 ++++++++++++++++++++++++++ > 2 files changed, 116 insertions(+) > > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c > index d45f9a0..d8a7d86 100644 > --- a/drivers/iommu/dma-iommu.c > +++ b/drivers/iommu/dma-iommu.c > @@ -43,6 +43,38 @@ struct iommu_dma_cookie { > spinlock_t msi_lock; > }; > > +/** > + * struct iommu_msi_doorbell_info - MSI doorbell region descriptor > + * @percpu_doorbells: per cpu doorbell base address > + * @global_doorbell: base address of the doorbell > + * @doorbell_is_percpu: is the doorbell per cpu or global? > + * @safe: true if irq remapping is implemented > + * @size: size of the doorbell > + */ > +struct iommu_msi_doorbell_info { > + union { > + phys_addr_t __percpu *percpu_doorbells; Out of curiosity, have you come across systems that have per-cpu doorbells? I couldn't find a system that'd help solidify my understanding on it's usage. > + phys_addr_t global_doorbell; > + }; > + bool doorbell_is_percpu; > + bool safe; Although you've got the comment above, 'safe' doesn't quite convey it's purpose. Can this be renamed to something more descriptive - 'intr_remapping' or 'intr_isolation' perhaps? Thanks, Punit [...] -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Punit, On 14/10/2016 13:25, Punit Agrawal wrote: > Hi Eric, > > One query and a comment below. > > Eric Auger <eric.auger@redhat.com> writes: > >> We introduce the capability to (un)register MSI doorbells. >> >> A doorbell region is characterized by its physical address base, size, >> and whether it its safe (ie. it implements IRQ remapping). A doorbell >> can be per-cpu or global. We currently only care about global doorbells. >> >> A function returns whether all registered doorbells are safe. >> >> MSI controllers likely to work along with IOMMU that translate MSI >> transaction must register their doorbells to allow device assignment >> with MSI support. Otherwise the MSI transactions will cause IOMMU faults. >> >> Signed-off-by: Eric Auger <eric.auger@redhat.com> >> >> --- >> >> v13 -> v14: >> - previously in msi-doorbell.h/c >> --- >> drivers/iommu/dma-iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++ >> include/linux/dma-iommu.h | 41 ++++++++++++++++++++++++++ >> 2 files changed, 116 insertions(+) >> >> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c >> index d45f9a0..d8a7d86 100644 >> --- a/drivers/iommu/dma-iommu.c >> +++ b/drivers/iommu/dma-iommu.c >> @@ -43,6 +43,38 @@ struct iommu_dma_cookie { >> spinlock_t msi_lock; >> }; >> >> +/** >> + * struct iommu_msi_doorbell_info - MSI doorbell region descriptor >> + * @percpu_doorbells: per cpu doorbell base address >> + * @global_doorbell: base address of the doorbell >> + * @doorbell_is_percpu: is the doorbell per cpu or global? >> + * @safe: true if irq remapping is implemented >> + * @size: size of the doorbell >> + */ >> +struct iommu_msi_doorbell_info { >> + union { >> + phys_addr_t __percpu *percpu_doorbells; > > Out of curiosity, have you come across systems that have per-cpu > doorbells? I couldn't find a system that'd help solidify my > understanding on it's usage. This came out after a discussion With Marc. However at the moment I am not aware of any MSI controller featuring per-cpu doorbell. Not sure whether it stays relevant to keep this notion at that stage. > >> + phys_addr_t global_doorbell; >> + }; >> + bool doorbell_is_percpu; >> + bool safe; > > Although you've got the comment above, 'safe' doesn't quite convey it's > purpose. Can this be renamed to something more descriptive - > 'intr_remapping' or 'intr_isolation' perhaps? Yes definitively Thanks Eric > > Thanks, > Punit > > > [...] > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index d45f9a0..d8a7d86 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -43,6 +43,38 @@ struct iommu_dma_cookie { spinlock_t msi_lock; }; +/** + * struct iommu_msi_doorbell_info - MSI doorbell region descriptor + * @percpu_doorbells: per cpu doorbell base address + * @global_doorbell: base address of the doorbell + * @doorbell_is_percpu: is the doorbell per cpu or global? + * @safe: true if irq remapping is implemented + * @size: size of the doorbell + */ +struct iommu_msi_doorbell_info { + union { + phys_addr_t __percpu *percpu_doorbells; + phys_addr_t global_doorbell; + }; + bool doorbell_is_percpu; + bool safe; + size_t size; +}; + +struct iommu_msi_doorbell { + struct iommu_msi_doorbell_info info; + struct list_head next; +}; + +/* list of registered MSI doorbells */ +static LIST_HEAD(iommu_msi_doorbell_list); + +/* counts the number of unsafe registered doorbells */ +static uint nb_unsafe_doorbells; + +/* protects the list and nb_unsafe_doorbells */ +static DEFINE_MUTEX(iommu_msi_doorbell_mutex); + static inline struct iova_domain *cookie_iovad(struct iommu_domain *domain) { return &((struct iommu_dma_cookie *)domain->iova_cookie)->iovad; @@ -755,3 +787,46 @@ int iommu_get_dma_msi_region_cookie(struct iommu_domain *domain, return 0; } EXPORT_SYMBOL(iommu_get_dma_msi_region_cookie); + +struct iommu_msi_doorbell_info * +iommu_msi_doorbell_alloc(phys_addr_t base, size_t size, bool safe) +{ + struct iommu_msi_doorbell *db; + + db = kzalloc(sizeof(*db), GFP_KERNEL); + if (!db) + return ERR_PTR(-ENOMEM); + + db->info.global_doorbell = base; + db->info.size = size; + db->info.safe = safe; + + mutex_lock(&iommu_msi_doorbell_mutex); + list_add(&db->next, &iommu_msi_doorbell_list); + if (!db->info.safe) + nb_unsafe_doorbells++; + mutex_unlock(&iommu_msi_doorbell_mutex); + return &db->info; +} +EXPORT_SYMBOL_GPL(iommu_msi_doorbell_alloc); + +void iommu_msi_doorbell_free(struct iommu_msi_doorbell_info *dbinfo) +{ + struct iommu_msi_doorbell *db; + + db = container_of(dbinfo, struct iommu_msi_doorbell, info); + + mutex_lock(&iommu_msi_doorbell_mutex); + list_del(&db->next); + if (!db->info.safe) + nb_unsafe_doorbells--; + mutex_unlock(&iommu_msi_doorbell_mutex); + kfree(db); +} +EXPORT_SYMBOL_GPL(iommu_msi_doorbell_free); + +bool iommu_msi_doorbell_safe(void) +{ + return !nb_unsafe_doorbells; +} +EXPORT_SYMBOL_GPL(iommu_msi_doorbell_safe); diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h index 05ab5b4..9640a27 100644 --- a/include/linux/dma-iommu.h +++ b/include/linux/dma-iommu.h @@ -19,6 +19,8 @@ #ifdef __KERNEL__ #include <asm/errno.h> +struct iommu_msi_doorbell_info; + #ifdef CONFIG_IOMMU_DMA #include <linux/iommu.h> #include <linux/msi.h> @@ -70,6 +72,31 @@ void iommu_dma_map_msi_msg(int irq, struct msi_msg *msg); int iommu_get_dma_msi_region_cookie(struct iommu_domain *domain, dma_addr_t base, u64 size); +/** + * iommu_msi_doorbell_alloc - allocate a global doorbell + * @base: physical base address of the doorbell + * @size: size of the doorbell + * @safe: true is irq_remapping implemented for this doorbell + * + * Return: the newly allocated doorbell info or a pointer converted error + */ +struct iommu_msi_doorbell_info * +iommu_msi_doorbell_alloc(phys_addr_t base, size_t size, bool safe); + +/** + * iommu_msi_doorbell_free - free a global doorbell + * @db: doorbell info to free + */ +void iommu_msi_doorbell_free(struct iommu_msi_doorbell_info *db); + +/** + * iommu_msi_doorbell_safe - return whether all registered doorbells are safe + * + * Safe doorbells are those which implement irq remapping + * Return: true if all doorbells are safe, false otherwise + */ +bool iommu_msi_doorbell_safe(void); + #else struct iommu_domain; @@ -99,6 +126,20 @@ static inline int iommu_get_dma_msi_region_cookie(struct iommu_domain *domain, return -ENODEV; } +static inline struct iommu_msi_doorbell_info * +iommu_msi_doorbell_alloc(phys_addr_t base, size_t size, bool safe) +{ + return NULL; +} + +static inline void +iommu_msi_doorbell_free(struct msi_doorbell_info *db) {} + +static inline bool iommu_msi_doorbell_safe(void) +{ + return false; +} + #endif /* CONFIG_IOMMU_DMA */ #endif /* __KERNEL__ */ #endif /* __DMA_IOMMU_H */
We introduce the capability to (un)register MSI doorbells. A doorbell region is characterized by its physical address base, size, and whether it its safe (ie. it implements IRQ remapping). A doorbell can be per-cpu or global. We currently only care about global doorbells. A function returns whether all registered doorbells are safe. MSI controllers likely to work along with IOMMU that translate MSI transaction must register their doorbells to allow device assignment with MSI support. Otherwise the MSI transactions will cause IOMMU faults. Signed-off-by: Eric Auger <eric.auger@redhat.com> --- v13 -> v14: - previously in msi-doorbell.h/c --- drivers/iommu/dma-iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-iommu.h | 41 ++++++++++++++++++++++++++ 2 files changed, 116 insertions(+)