Message ID | 20250110140152.27624-4-roger.pau@citrix.com (mailing list archive) |
---|---|
State | Superseded |
Headers | show |
Series | xen: fix usage of devices behind a VMD bridge | expand |
Match subject line style again. On Fri, Jan 10, 2025 at 03:01:50PM +0100, Roger Pau Monne wrote: > Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for both MSI > and MSI-X entries globally, regardless of the IRQ chip they are using. Only > Xen sets the pci_msi_ignore_mask when routing physical interrupts over event > channels, to prevent PCI code from attempting to toggle the maskbit, as it's > Xen that controls the bit. > > However, the pci_msi_ignore_mask being global will affect devices that use MSI > interrupts but are not routing those interrupts over event channels (not using > the Xen pIRQ chip). One example is devices behind a VMD PCI bridge. In that > scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the pIRQ > one in the Xen case), and devices behind the bridge configure the MSI entries > using indexes into the VMD bridge MSI table. The VMD bridge then demultiplexes > such interrupts and delivers to the destination device(s). Having > pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI entries > for devices behind the VMD bridge. > > Move the signaling of no entry masking into the MSI domain flags, as that > allows setting it on a per-domain basis. Set it for the Xen MSI domain that > uses the pIRQ chip, while leaving it unset for the rest of the cases. > > Remove pci_msi_ignore_mask at once, since it was only used by Xen code, and > with Xen dropping usage the variable is unneeded. > > This fixes using devices behind a VMD bridge on Xen PV hardware domains. Wrap to fit in 75 columns. The first two patches talk about devices behind VMD not being usable for Xen, but this one says they now work. But this doesn't undo the code changes or comments added by the first two, so the result is a bit confusing (probably because I know nothing about Xen). Bjorn
Hi Roger, kernel test robot noticed the following build errors: [auto build test ERROR on pci/next] [also build test ERROR on pci/for-linus xen-tip/linux-next tip/irq/core linus/master v6.13-rc6 next-20250110] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Roger-Pau-Monne/xen-pci-do-not-register-devices-outside-of-PCI-segment-scope/20250110-220331 base: https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next patch link: https://lore.kernel.org/r/20250110140152.27624-4-roger.pau%40citrix.com patch subject: [PATCH 3/3] pci/msi: remove pci_msi_ignore_mask config: arm64-randconfig-003-20250111 (https://download.01.org/0day-ci/archive/20250111/202501111839.HXJGe5FL-lkp@intel.com/config) compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250111/202501111839.HXJGe5FL-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202501111839.HXJGe5FL-lkp@intel.com/ All errors (new ones prefixed by >>): >> drivers/pci/msi/msi.c:288:40: error: incomplete definition of type 'struct irq_domain' 288 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ drivers/pci/msi/msi.c:604:40: error: incomplete definition of type 'struct irq_domain' 604 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ drivers/pci/msi/msi.c:714:40: error: incomplete definition of type 'struct irq_domain' 714 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ 3 errors generated. vim +288 drivers/pci/msi/msi.c 283 284 static int msi_setup_msi_desc(struct pci_dev *dev, int nvec, 285 struct irq_affinity_desc *masks) 286 { 287 const struct irq_domain *d = dev_get_msi_domain(&dev->dev); > 288 const struct msi_domain_info *info = d->host_data; 289 struct msi_desc desc; 290 u16 control; 291 292 /* MSI Entry Initialization */ 293 memset(&desc, 0, sizeof(desc)); 294 295 pci_read_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, &control); 296 /* Lies, damned lies, and MSIs */ 297 if (dev->dev_flags & PCI_DEV_FLAGS_HAS_MSI_MASKING) 298 control |= PCI_MSI_FLAGS_MASKBIT; 299 if (info->flags & MSI_FLAG_NO_MASK) 300 control &= ~PCI_MSI_FLAGS_MASKBIT; 301 302 desc.nvec_used = nvec; 303 desc.pci.msi_attrib.is_64 = !!(control & PCI_MSI_FLAGS_64BIT); 304 desc.pci.msi_attrib.can_mask = !!(control & PCI_MSI_FLAGS_MASKBIT); 305 desc.pci.msi_attrib.default_irq = dev->irq; 306 desc.pci.msi_attrib.multi_cap = FIELD_GET(PCI_MSI_FLAGS_QMASK, control); 307 desc.pci.msi_attrib.multiple = ilog2(__roundup_pow_of_two(nvec)); 308 desc.affinity = masks; 309 310 if (control & PCI_MSI_FLAGS_64BIT) 311 desc.pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_64; 312 else 313 desc.pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_32; 314 315 /* Save the initial mask status */ 316 if (desc.pci.msi_attrib.can_mask) 317 pci_read_config_dword(dev, desc.pci.mask_pos, &desc.pci.msi_mask); 318 319 return msi_insert_msi_desc(&dev->dev, &desc); 320 } 321
Hi Roger, kernel test robot noticed the following build errors: [auto build test ERROR on pci/next] [also build test ERROR on pci/for-linus xen-tip/linux-next tip/irq/core linus/master v6.13-rc6 next-20250110] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Roger-Pau-Monne/xen-pci-do-not-register-devices-outside-of-PCI-segment-scope/20250110-220331 base: https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next patch link: https://lore.kernel.org/r/20250110140152.27624-4-roger.pau%40citrix.com patch subject: [PATCH 3/3] pci/msi: remove pci_msi_ignore_mask config: riscv-defconfig (https://download.01.org/0day-ci/archive/20250111/202501112048.6yCFh2ma-lkp@intel.com/config) compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250111/202501112048.6yCFh2ma-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202501112048.6yCFh2ma-lkp@intel.com/ All errors (new ones prefixed by >>): In file included from drivers/pci/msi/msi.c:12: In file included from include/linux/irq.h:23: In file included from arch/riscv/include/asm/irq.h:10: In file included from include/linux/interrupt.h:22: In file included from arch/riscv/include/asm/sections.h:9: In file included from include/linux/mm.h:2223: include/linux/vmstat.h:518:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion] 518 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_" | ~~~~~~~~~~~ ^ ~~~ >> drivers/pci/msi/msi.c:288:40: error: incomplete definition of type 'const struct irq_domain' 288 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ drivers/pci/msi/msi.c:604:40: error: incomplete definition of type 'const struct irq_domain' 604 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ drivers/pci/msi/msi.c:714:40: error: incomplete definition of type 'const struct irq_domain' 714 | const struct msi_domain_info *info = d->host_data; | ~^ include/linux/irq.h:130:8: note: forward declaration of 'struct irq_domain' 130 | struct irq_domain; | ^ 1 warning and 3 errors generated. vim +288 drivers/pci/msi/msi.c 283 284 static int msi_setup_msi_desc(struct pci_dev *dev, int nvec, 285 struct irq_affinity_desc *masks) 286 { 287 const struct irq_domain *d = dev_get_msi_domain(&dev->dev); > 288 const struct msi_domain_info *info = d->host_data; 289 struct msi_desc desc; 290 u16 control; 291 292 /* MSI Entry Initialization */ 293 memset(&desc, 0, sizeof(desc)); 294 295 pci_read_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, &control); 296 /* Lies, damned lies, and MSIs */ 297 if (dev->dev_flags & PCI_DEV_FLAGS_HAS_MSI_MASKING) 298 control |= PCI_MSI_FLAGS_MASKBIT; 299 if (info->flags & MSI_FLAG_NO_MASK) 300 control &= ~PCI_MSI_FLAGS_MASKBIT; 301 302 desc.nvec_used = nvec; 303 desc.pci.msi_attrib.is_64 = !!(control & PCI_MSI_FLAGS_64BIT); 304 desc.pci.msi_attrib.can_mask = !!(control & PCI_MSI_FLAGS_MASKBIT); 305 desc.pci.msi_attrib.default_irq = dev->irq; 306 desc.pci.msi_attrib.multi_cap = FIELD_GET(PCI_MSI_FLAGS_QMASK, control); 307 desc.pci.msi_attrib.multiple = ilog2(__roundup_pow_of_two(nvec)); 308 desc.affinity = masks; 309 310 if (control & PCI_MSI_FLAGS_64BIT) 311 desc.pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_64; 312 else 313 desc.pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_32; 314 315 /* Save the initial mask status */ 316 if (desc.pci.msi_attrib.can_mask) 317 pci_read_config_dword(dev, desc.pci.mask_pos, &desc.pci.msi_mask); 318 319 return msi_insert_msi_desc(&dev->dev, &desc); 320 } 321
On Fri, Jan 10, 2025 at 04:30:57PM -0600, Bjorn Helgaas wrote: > Match subject line style again. > > On Fri, Jan 10, 2025 at 03:01:50PM +0100, Roger Pau Monne wrote: > > Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for both MSI > > and MSI-X entries globally, regardless of the IRQ chip they are using. Only > > Xen sets the pci_msi_ignore_mask when routing physical interrupts over event > > channels, to prevent PCI code from attempting to toggle the maskbit, as it's > > Xen that controls the bit. > > > > However, the pci_msi_ignore_mask being global will affect devices that use MSI > > interrupts but are not routing those interrupts over event channels (not using > > the Xen pIRQ chip). One example is devices behind a VMD PCI bridge. In that > > scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the pIRQ > > one in the Xen case), and devices behind the bridge configure the MSI entries > > using indexes into the VMD bridge MSI table. The VMD bridge then demultiplexes > > such interrupts and delivers to the destination device(s). Having > > pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI entries > > for devices behind the VMD bridge. > > > > Move the signaling of no entry masking into the MSI domain flags, as that > > allows setting it on a per-domain basis. Set it for the Xen MSI domain that > > uses the pIRQ chip, while leaving it unset for the rest of the cases. > > > > Remove pci_msi_ignore_mask at once, since it was only used by Xen code, and > > with Xen dropping usage the variable is unneeded. > > > > This fixes using devices behind a VMD bridge on Xen PV hardware domains. > > Wrap to fit in 75 columns. > > The first two patches talk about devices behind VMD not being usable > for Xen, but this one says they now work. Sorry, let me try to clarify: Devices behind a VMD bridge are not known to Xen, however that doesn't mean Linux cannot use them. That's what this series achieves. By inhibiting the usage of VMD_FEAT_CAN_BYPASS_MSI_REMAP and the removal of the pci_msi_ignore_mask bodge devices behind a VMD bridge do work fine when use from a Linux Xen hardware domain. That's the whole point of the series. > But this doesn't undo the > code changes or comments added by the first two, so the result is a > bit confusing (probably because I know nothing about Xen). All patches are needed. There's probably some confusion about devices behind a VMD bridge not being known by Xen vs not being usable by Linux running under Xen as a hardware domain. With all three patches applied devices behind a VMD bridge work under Linux with Xen. Thanks, Roger.
On Mon, Jan 13, 2025 at 11:25:58AM +0100, Roger Pau Monné wrote: > On Fri, Jan 10, 2025 at 04:30:57PM -0600, Bjorn Helgaas wrote: > > On Fri, Jan 10, 2025 at 03:01:50PM +0100, Roger Pau Monne wrote: > > > Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for both MSI > > > and MSI-X entries globally, regardless of the IRQ chip they are using. Only > > > Xen sets the pci_msi_ignore_mask when routing physical interrupts over event > > > channels, to prevent PCI code from attempting to toggle the maskbit, as it's > > > Xen that controls the bit. > > > > > > However, the pci_msi_ignore_mask being global will affect devices that use MSI > > > interrupts but are not routing those interrupts over event channels (not using > > > the Xen pIRQ chip). One example is devices behind a VMD PCI bridge. In that > > > scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the pIRQ > > > one in the Xen case), and devices behind the bridge configure the MSI entries > > > using indexes into the VMD bridge MSI table. The VMD bridge then demultiplexes > > > such interrupts and delivers to the destination device(s). Having > > > pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI entries > > > for devices behind the VMD bridge. > > > > > > Move the signaling of no entry masking into the MSI domain flags, as that > > > allows setting it on a per-domain basis. Set it for the Xen MSI domain that > > > uses the pIRQ chip, while leaving it unset for the rest of the cases. > > > > > > Remove pci_msi_ignore_mask at once, since it was only used by Xen code, and > > > with Xen dropping usage the variable is unneeded. > > > > > > This fixes using devices behind a VMD bridge on Xen PV hardware domains. > > > > Wrap to fit in 75 columns. > > > > The first two patches talk about devices behind VMD not being usable > > for Xen, but this one says they now work. > > Sorry, let me try to clarify: > > Devices behind a VMD bridge are not known to Xen, however that doesn't > mean Linux cannot use them. That's what this series achieves. By > inhibiting the usage of VMD_FEAT_CAN_BYPASS_MSI_REMAP and the removal > of the pci_msi_ignore_mask bodge devices behind a VMD bridge do work > fine when use from a Linux Xen hardware domain. That's the whole > point of the series. > > > But this doesn't undo the > > code changes or comments added by the first two, so the result is a > > bit confusing (probably because I know nothing about Xen). > > All patches are needed. There's probably some confusion about devices > behind a VMD bridge not being known by Xen vs not being usable by > Linux running under Xen as a hardware domain. > > With all three patches applied devices behind a VMD bridge work under > Linux with Xen. There's certainly confusion in *my* mind because I don't know enough about Xen to understand the subtlety about devices behind VMD not being known by Xen but still being usable by Linux running under Xen. Bjorn
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index 0f2fe524f60d..b8755cde2419 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -436,7 +436,8 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { }; static struct msi_domain_info xen_pci_msi_domain_info = { - .flags = MSI_FLAG_PCI_MSIX | MSI_FLAG_FREE_MSI_DESCS | MSI_FLAG_DEV_SYSFS, + .flags = MSI_FLAG_PCI_MSIX | MSI_FLAG_FREE_MSI_DESCS | + MSI_FLAG_DEV_SYSFS | MSI_FLAG_NO_MASK, .ops = &xen_pci_msi_domain_ops, }; @@ -484,11 +485,6 @@ static __init void xen_setup_pci_msi(void) * in allocating the native domain and never use it. */ x86_init.irqs.create_pci_msi_domain = xen_create_pci_msi_domain; - /* - * With XEN PIRQ/Eventchannels in use PCI/MSI[-X] masking is solely - * controlled by the hypervisor. - */ - pci_msi_ignore_mask = 1; } #else /* CONFIG_PCI_MSI */ diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c index 3a45879d85db..cb42298f6a97 100644 --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -15,7 +15,6 @@ #include "msi.h" int pci_msi_enable = 1; -int pci_msi_ignore_mask; /** * pci_msi_supported - check whether MSI may be enabled on a device @@ -285,6 +284,8 @@ static void pci_msi_set_enable(struct pci_dev *dev, int enable) static int msi_setup_msi_desc(struct pci_dev *dev, int nvec, struct irq_affinity_desc *masks) { + const struct irq_domain *d = dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info = d->host_data; struct msi_desc desc; u16 control; @@ -295,8 +296,7 @@ static int msi_setup_msi_desc(struct pci_dev *dev, int nvec, /* Lies, damned lies, and MSIs */ if (dev->dev_flags & PCI_DEV_FLAGS_HAS_MSI_MASKING) control |= PCI_MSI_FLAGS_MASKBIT; - /* Respect XEN's mask disabling */ - if (pci_msi_ignore_mask) + if (info->flags & MSI_FLAG_NO_MASK) control &= ~PCI_MSI_FLAGS_MASKBIT; desc.nvec_used = nvec; @@ -600,12 +600,15 @@ static void __iomem *msix_map_region(struct pci_dev *dev, */ void msix_prepare_msi_desc(struct pci_dev *dev, struct msi_desc *desc) { + const struct irq_domain *d = dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info = d->host_data; + desc->nvec_used = 1; desc->pci.msi_attrib.is_msix = 1; desc->pci.msi_attrib.is_64 = 1; desc->pci.msi_attrib.default_irq = dev->irq; desc->pci.mask_base = dev->msix_base; - desc->pci.msi_attrib.can_mask = !pci_msi_ignore_mask && + desc->pci.msi_attrib.can_mask = !(info->flags & MSI_FLAG_NO_MASK) && !desc->pci.msi_attrib.is_virtual; if (desc->pci.msi_attrib.can_mask) { @@ -655,9 +658,6 @@ static void msix_mask_all(void __iomem *base, int tsize) u32 ctrl = PCI_MSIX_ENTRY_CTRL_MASKBIT; int i; - if (pci_msi_ignore_mask) - return; - for (i = 0; i < tsize; i++, base += PCI_MSIX_ENTRY_SIZE) writel(ctrl, base + PCI_MSIX_ENTRY_VECTOR_CTRL); } @@ -710,6 +710,8 @@ static int msix_setup_interrupts(struct pci_dev *dev, struct msix_entry *entries static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries, int nvec, struct irq_affinity *affd) { + const struct irq_domain *d = dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info = d->host_data; int ret, tsize; u16 control; @@ -740,15 +742,17 @@ static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries, /* Disable INTX */ pci_intx_for_msi(dev, 0); - /* - * Ensure that all table entries are masked to prevent - * stale entries from firing in a crash kernel. - * - * Done late to deal with a broken Marvell NVME device - * which takes the MSI-X mask bits into account even - * when MSI-X is disabled, which prevents MSI delivery. - */ - msix_mask_all(dev->msix_base, tsize); + if (!(info->flags & MSI_FLAG_NO_MASK)) { + /* + * Ensure that all table entries are masked to prevent + * stale entries from firing in a crash kernel. + * + * Done late to deal with a broken Marvell NVME device + * which takes the MSI-X mask bits into account even + * when MSI-X is disabled, which prevents MSI delivery. + */ + msix_mask_all(dev->msix_base, tsize); + } pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0); pcibios_free_irq(dev); diff --git a/include/linux/msi.h b/include/linux/msi.h index b10093c4d00e..59a421fc42bf 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -73,7 +73,6 @@ struct msi_msg { }; }; -extern int pci_msi_ignore_mask; /* Helper functions */ struct msi_desc; struct pci_dev; @@ -556,6 +555,8 @@ enum { MSI_FLAG_PCI_MSIX_ALLOC_DYN = (1 << 20), /* PCI MSIs cannot be steered separately to CPU cores */ MSI_FLAG_NO_AFFINITY = (1 << 21), + /* Inhibit usage of entry masking */ + MSI_FLAG_NO_MASK = (1 << 22), }; /** diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 396a067a8a56..7682c36cbccc 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -1143,7 +1143,7 @@ static bool msi_check_reservation_mode(struct irq_domain *domain, if (!(info->flags & MSI_FLAG_MUST_REACTIVATE)) return false; - if (IS_ENABLED(CONFIG_PCI_MSI) && pci_msi_ignore_mask) + if (info->flags & MSI_FLAG_NO_MASK) return false; /*
Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for both MSI and MSI-X entries globally, regardless of the IRQ chip they are using. Only Xen sets the pci_msi_ignore_mask when routing physical interrupts over event channels, to prevent PCI code from attempting to toggle the maskbit, as it's Xen that controls the bit. However, the pci_msi_ignore_mask being global will affect devices that use MSI interrupts but are not routing those interrupts over event channels (not using the Xen pIRQ chip). One example is devices behind a VMD PCI bridge. In that scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the pIRQ one in the Xen case), and devices behind the bridge configure the MSI entries using indexes into the VMD bridge MSI table. The VMD bridge then demultiplexes such interrupts and delivers to the destination device(s). Having pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI entries for devices behind the VMD bridge. Move the signaling of no entry masking into the MSI domain flags, as that allows setting it on a per-domain basis. Set it for the Xen MSI domain that uses the pIRQ chip, while leaving it unset for the rest of the cases. Remove pci_msi_ignore_mask at once, since it was only used by Xen code, and with Xen dropping usage the variable is unneeded. This fixes using devices behind a VMD bridge on Xen PV hardware domains. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> --- arch/x86/pci/xen.c | 8 ++------ drivers/pci/msi/msi.c | 36 ++++++++++++++++++++---------------- include/linux/msi.h | 3 ++- kernel/irq/msi.c | 2 +- 4 files changed, 25 insertions(+), 24 deletions(-)