| Message ID | 20170705110145.20172-1-ddiss@suse.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, 05.07.17 13:01, David Disseldorp (ddiss@suse.de) wrote: > Ceph relies on by-partuuid symlinks, in order to locate the journal > partition from a given OSD partition. For details, see > http://tracker.ceph.com/issues/19489. This appears way too broad, as it would apply to all LVM and all other devices. It appears to me Ceph should do the same as LVM does for this, and ship its own set of rules, and be careful to only match against the actual devices it creates. I hope that makes sense, Lennart
Thanks for the feedback, Lennart... On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote: > On Wed, 05.07.17 13:01, David Disseldorp (ddiss@suse.de) wrote: > > > Ceph relies on by-partuuid symlinks, in order to locate the journal > > partition from a given OSD partition. For details, see > > http://tracker.ceph.com/issues/19489. > > This appears way too broad, as it would apply to all LVM and all other > devices. > > It appears to me Ceph should do the same as LVM does for this, and > ship its own set of rules, and be careful to only match against the > actual devices it creates. We can certainly do this in a Ceph specific manner via the existing 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks are "owned" by 60-persistent-storage.rules . If you don't think the existence of these symlinks for dm paths will be of use to others then I'll go ahead and propose it as a Ceph only change. Cheers, David -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 10.07.17 11:37, David Disseldorp (ddiss@suse.de) wrote: > Thanks for the feedback, Lennart... > > On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote: > > > On Wed, 05.07.17 13:01, David Disseldorp (ddiss@suse.de) wrote: > > > > > Ceph relies on by-partuuid symlinks, in order to locate the journal > > > partition from a given OSD partition. For details, see > > > http://tracker.ceph.com/issues/19489. > > > > This appears way too broad, as it would apply to all LVM and all other > > devices. > > > > It appears to me Ceph should do the same as LVM does for this, and > > ship its own set of rules, and be careful to only match against the > > actual devices it creates. > > We can certainly do this in a Ceph specific manner via the existing > 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks > are "owned" by 60-persistent-storage.rules . > > If you don't think the existence of these symlinks for dm paths will be > of use to others then I'll go ahead and propose it as a Ceph only > change. Hmmm, so thinking about this again: "partuuid" is actually for GPT partition UUIDs if I recall recorrectly, they wouldn't be generated for DM devices anyway, since they are one layer removed from the GPT, so are you even sure this will do what you are asking for? But even if this actually works: DM links so far are created by the LVM/libdevicemapper/ packages, not by udev, and I don't think this should change. Hence, please talk to the LVM/libdevicemapper folks about this and ask them for including it. Lennart
On 07/10/2017 11:53 AM, Lennart Poettering wrote: > On Mon, 10.07.17 11:37, David Disseldorp (ddiss@suse.de) wrote: > >> Thanks for the feedback, Lennart... >> >> On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote: >> >>> On Wed, 05.07.17 13:01, David Disseldorp (ddiss@suse.de) wrote: >>> >>>> Ceph relies on by-partuuid symlinks, in order to locate the journal >>>> partition from a given OSD partition. For details, see >>>> http://tracker.ceph.com/issues/19489. >>> >>> This appears way too broad, as it would apply to all LVM and all other >>> devices. >>> >>> It appears to me Ceph should do the same as LVM does for this, and >>> ship its own set of rules, and be careful to only match against the >>> actual devices it creates. >> >> We can certainly do this in a Ceph specific manner via the existing >> 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks >> are "owned" by 60-persistent-storage.rules . >> >> If you don't think the existence of these symlinks for dm paths will be >> of use to others then I'll go ahead and propose it as a Ceph only >> change. > > Hmmm, so thinking about this again: "partuuid" is actually for GPT > partition UUIDs if I recall recorrectly, they wouldn't be generated > for DM devices anyway, since they are one layer removed from the GPT, > so are you even sure this will do what you are asking for? > > But even if this actually works: DM links so far are created by the > LVM/libdevicemapper/ packages, not by udev, and I don't think this > should change. Hence, please talk to the LVM/libdevicemapper folks > about this and ask them for including it. > Yes, please, any rules for symlinks which should be created under /dev/disk for DM devices (including all its subsystems like LVM, mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source tree: https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in Now, when we create a partition over a DM device, there's a new mapping created on top for each partition (either by calling kpartx manually or by having it created by partitioning tool directly if it supports that). So in this case, it's not the kernel directly who creates the partitions, but they're simply another DM devices created on top of the underlying DM device to represent these partitions. But I think that doesn't matter - we should still create those symlinks for people to still have a possibility to reference the device by its part uuid - I'll fix 13-dm-disk.rules to include this.
On 07/10/2017 12:14 PM, Peter Rajnoha wrote: > On 07/10/2017 11:53 AM, Lennart Poettering wrote: >> On Mon, 10.07.17 11:37, David Disseldorp (ddiss@suse.de) wrote: >> >>> Thanks for the feedback, Lennart... >>> >>> On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote: >>> >>>> On Wed, 05.07.17 13:01, David Disseldorp (ddiss@suse.de) wrote: >>>> >>>>> Ceph relies on by-partuuid symlinks, in order to locate the journal >>>>> partition from a given OSD partition. For details, see >>>>> http://tracker.ceph.com/issues/19489. >>>> >>>> This appears way too broad, as it would apply to all LVM and all other >>>> devices. >>>> >>>> It appears to me Ceph should do the same as LVM does for this, and >>>> ship its own set of rules, and be careful to only match against the >>>> actual devices it creates. >>> >>> We can certainly do this in a Ceph specific manner via the existing >>> 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks >>> are "owned" by 60-persistent-storage.rules . >>> >>> If you don't think the existence of these symlinks for dm paths will be >>> of use to others then I'll go ahead and propose it as a Ceph only >>> change. >> >> Hmmm, so thinking about this again: "partuuid" is actually for GPT >> partition UUIDs if I recall recorrectly, they wouldn't be generated >> for DM devices anyway, since they are one layer removed from the GPT, >> so are you even sure this will do what you are asking for? >> >> But even if this actually works: DM links so far are created by the >> LVM/libdevicemapper/ packages, not by udev, and I don't think this >> should change. Hence, please talk to the LVM/libdevicemapper folks >> about this and ask them for including it. >> > > Yes, please, any rules for symlinks which should be created under > /dev/disk for DM devices (including all its subsystems like LVM, > mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source > tree: > > https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in > > Now, when we create a partition over a DM device, there's a new mapping > created on top for each partition (either by calling kpartx manually or > by having it created by partitioning tool directly if it supports that). > So in this case, it's not the kernel directly who creates the > partitions, but they're simply another DM devices created on top of the > underlying DM device to represent these partitions. But I think that > doesn't matter - we should still create those symlinks for people to > still have a possibility to reference the device by its part uuid - I'll > fix 13-dm-disk.rules to include this. > Fixed here: https://sourceware.org/git/?p=lvm2.git;a=commit;h=c48149cf80c6582c2369bc7f8a33d794021d9dae
On Mon, 10 Jul 2017 12:47:24 +0200, Peter Rajnoha wrote: > On 07/10/2017 12:14 PM, Peter Rajnoha wrote: ... > > Yes, please, any rules for symlinks which should be created under > > /dev/disk for DM devices (including all its subsystems like LVM, > > mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source > > tree: > > > > https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in > > > > Now, when we create a partition over a DM device, there's a new mapping > > created on top for each partition (either by calling kpartx manually or > > by having it created by partitioning tool directly if it supports that). > > So in this case, it's not the kernel directly who creates the > > partitions, but they're simply another DM devices created on top of the > > underlying DM device to represent these partitions. But I think that > > doesn't matter - we should still create those symlinks for people to > > still have a possibility to reference the device by its part uuid - I'll > > fix 13-dm-disk.rules to include this. > > > > Fixed here: > > https://sourceware.org/git/?p=lvm2.git;a=commit;h=c48149cf80c6582c2369bc7f8a33d794021d9dae Looks good and works for me - thanks Peter. Cheers, David -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/rules/60-persistent-storage.rules b/rules/60-persistent-storage.rules index 5ab03fc27..bc0721f32 100644 --- a/rules/60-persistent-storage.rules +++ b/rules/60-persistent-storage.rules @@ -6,7 +6,7 @@ ACTION=="remove", GOTO="persistent_storage_end" SUBSYSTEM!="block", GOTO="persistent_storage_end" -KERNEL!="loop*|mmcblk*[0-9]|msblk*[0-9]|mspblk*[0-9]|nvme*|sd*|sr*|vd*|xvd*|bcache*|cciss*|dasd*", GOTO="persistent_storage_end" +KERNEL!="loop*|mmcblk*[0-9]|msblk*[0-9]|mspblk*[0-9]|nvme*|sd*|sr*|vd*|xvd*|bcache*|cciss*|dasd*|dm*", GOTO="persistent_storage_end" # ignore partitions that span the entire disk TEST=="whole_disk", GOTO="persistent_storage_end"