| Message ID | 20240816001216.26575-16-dakr@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Generic `Allocator` support for Rust | expand |
On 16.08.24 02:10, Danilo Krummrich wrote: > Currently, we can't implement `FromIterator`. There are a couple of > issues with this trait in the kernel, namely: > > - Rust's specialization feature is unstable. This prevents us to > optimze for the special case where `I::IntoIter` equals `Vec`'s > `IntoIter` type. > - We also can't use `I::IntoIter`'s type ID either to work around this, > since `FromIterator` doesn't require this type to be `'static`. > - `FromIterator::from_iter` does return `Self` instead of > `Result<Self, AllocError>`, hence we can't properly handle allocation > failures. > - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle > additional allocation flags. > > Instead, provide `IntoIter::collect`, such that we can at least convert > `IntoIter` into a `Vec` again. > > Reviewed-by: Alice Ryhl <aliceryhl@google.com> > Signed-off-by: Danilo Krummrich <dakr@kernel.org> > --- > rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 78 insertions(+) > > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs > index 3b79f977b65e..ad96f4c3af9e 100644 > --- a/rust/kernel/alloc/kvec.rs > +++ b/rust/kernel/alloc/kvec.rs > @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A> > fn as_raw_mut_slice(&mut self) -> *mut [T] { > ptr::slice_from_raw_parts_mut(self.ptr, self.len) > } > + > + fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) { > + let me = ManuallyDrop::new(self); > + let ptr = me.ptr; > + let buf = me.buf; > + let len = me.len; > + let cap = me.cap; > + (ptr, buf, len, cap) > + } > + > + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. > + /// > + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait > + /// in the kernel, namely: > + /// > + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special > + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. > + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` > + /// doesn't require this type to be `'static`. > + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence > + /// we can't properly handle allocation failures. > + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation > + /// flags. > + /// > + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a > + /// `Vec` again. I think it's great that you include this in the code, but I don't think that it should be visible in the documentation, can you move it under the `Examples` section and turn it into normal comments? > + /// > + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing > + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. > + /// > + /// # Examples > + /// > + /// ``` > + /// let v = kernel::kvec![1, 2, 3]?; > + /// let mut it = v.into_iter(); > + /// > + /// assert_eq!(it.next(), Some(1)); > + /// > + /// let v = it.collect(GFP_KERNEL); > + /// assert_eq!(v, [2, 3]); > + /// > + /// # Ok::<(), Error>(()) > + /// ``` > + pub fn collect(self, flags: Flags) -> Vec<T, A> { > + let (mut ptr, buf, len, mut cap) = self.into_raw_parts(); > + let has_advanced = ptr != buf.as_ptr(); > + > + if has_advanced { > + // SAFETY: Copy the contents we have advanced to at the beginning of the buffer. This first sentence should not be part of the SAFETY comment. > + // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is > + // guaranteed to be smaller than `buf.add(cap)`. This doesn't justify all the requirements documented in [1]. [1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety > + unsafe { ptr::copy(ptr, buf.as_ptr(), len) }; > + ptr = buf.as_ptr(); > + } > + > + // This can never fail, `len` is guaranteed to be smaller than `cap`. > + let layout = core::alloc::Layout::array::<T>(len).unwrap(); > + > + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be > + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves > + // it as it is. > + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } { > + // If we fail to shrink, which likely can't even happen, continue with the existing > + // buffer. > + Err(_) => ptr, > + Ok(ptr) => { > + cap = len; > + ptr.as_ptr().cast() > + } > + }; > + > + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to > + // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed > + // to point to the start of the backing buffer. `cap` is either the original capacity or, > + // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since > + // `into_iter` has been called on the original `Vec`. Turn this into bullet points please. --- Cheers, Benno > + unsafe { Vec::from_raw_parts(ptr, len, cap) } > + } > } > > impl<T, A> Iterator for IntoIter<T, A> > -- > 2.46.0 >
On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote: > On 16.08.24 02:10, Danilo Krummrich wrote: > > Currently, we can't implement `FromIterator`. There are a couple of > > issues with this trait in the kernel, namely: > > > > - Rust's specialization feature is unstable. This prevents us to > > optimze for the special case where `I::IntoIter` equals `Vec`'s > > `IntoIter` type. > > - We also can't use `I::IntoIter`'s type ID either to work around this, > > since `FromIterator` doesn't require this type to be `'static`. > > - `FromIterator::from_iter` does return `Self` instead of > > `Result<Self, AllocError>`, hence we can't properly handle allocation > > failures. > > - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle > > additional allocation flags. > > > > Instead, provide `IntoIter::collect`, such that we can at least convert > > `IntoIter` into a `Vec` again. > > > > Reviewed-by: Alice Ryhl <aliceryhl@google.com> > > Signed-off-by: Danilo Krummrich <dakr@kernel.org> > > --- > > rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 78 insertions(+) > > > > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs > > index 3b79f977b65e..ad96f4c3af9e 100644 > > --- a/rust/kernel/alloc/kvec.rs > > +++ b/rust/kernel/alloc/kvec.rs > > @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A> > > fn as_raw_mut_slice(&mut self) -> *mut [T] { > > ptr::slice_from_raw_parts_mut(self.ptr, self.len) > > } > > + > > + fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) { > > + let me = ManuallyDrop::new(self); > > + let ptr = me.ptr; > > + let buf = me.buf; > > + let len = me.len; > > + let cap = me.cap; > > + (ptr, buf, len, cap) > > + } > > + > > + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. > > + /// > > + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait > > + /// in the kernel, namely: > > + /// > > + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special > > + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. > > + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` > > + /// doesn't require this type to be `'static`. > > + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence > > + /// we can't properly handle allocation failures. > > + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation > > + /// flags. > > + /// > > + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a > > + /// `Vec` again. > > I think it's great that you include this in the code, but I don't think > that it should be visible in the documentation, Why not? I think this information is valuable for users of this API. > can you move it under > the `Examples` section and turn it into normal comments? > > > + /// > > + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing > > + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. > > + /// > > + /// # Examples > > + /// > > + /// ``` > > + /// let v = kernel::kvec![1, 2, 3]?; > > + /// let mut it = v.into_iter(); > > + /// > > + /// assert_eq!(it.next(), Some(1)); > > + /// > > + /// let v = it.collect(GFP_KERNEL); > > + /// assert_eq!(v, [2, 3]); > > + /// > > + /// # Ok::<(), Error>(()) > > + /// ``` > > + pub fn collect(self, flags: Flags) -> Vec<T, A> { > > + let (mut ptr, buf, len, mut cap) = self.into_raw_parts(); > > + let has_advanced = ptr != buf.as_ptr(); > > + > > + if has_advanced { > > + // SAFETY: Copy the contents we have advanced to at the beginning of the buffer. > > This first sentence should not be part of the SAFETY comment. > > > + // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is > > + // guaranteed to be smaller than `buf.add(cap)`. > > This doesn't justify all the requirements documented in [1]. > > [1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety > > > + unsafe { ptr::copy(ptr, buf.as_ptr(), len) }; > > + ptr = buf.as_ptr(); > > + } > > + > > + // This can never fail, `len` is guaranteed to be smaller than `cap`. > > + let layout = core::alloc::Layout::array::<T>(len).unwrap(); > > + > > + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be > > + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves > > + // it as it is. > > + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } { > > + // If we fail to shrink, which likely can't even happen, continue with the existing > > + // buffer. > > + Err(_) => ptr, > > + Ok(ptr) => { > > + cap = len; > > + ptr.as_ptr().cast() > > + } > > + }; > > + > > + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to > > + // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed > > + // to point to the start of the backing buffer. `cap` is either the original capacity or, > > + // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since > > + // `into_iter` has been called on the original `Vec`. > > Turn this into bullet points please. > > --- > Cheers, > Benno > > > + unsafe { Vec::from_raw_parts(ptr, len, cap) } > > + } > > } > > > > impl<T, A> Iterator for IntoIter<T, A> > > -- > > 2.46.0 > > >
On 11.09.24 02:22, Danilo Krummrich wrote: > On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote: >> On 16.08.24 02:10, Danilo Krummrich wrote: >>> + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. >>> + /// >>> + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait >>> + /// in the kernel, namely: >>> + /// >>> + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special >>> + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. >>> + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` >>> + /// doesn't require this type to be `'static`. >>> + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence >>> + /// we can't properly handle allocation failures. >>> + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation >>> + /// flags. >>> + /// >>> + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a >>> + /// `Vec` again. >> >> I think it's great that you include this in the code, but I don't think >> that it should be visible in the documentation, > > Why not? I think this information is valuable for users of this API. If you want to keep it, then I don't mind, but I would still move it underneath `Examples` and add a section header `# Implementation Details` or similar. --- Cheers, Benno >> can you move it under >> the `Examples` section and turn it into normal comments? >> >>> + /// >>> + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing >>> + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. >>> + /// >>> + /// # Examples >>> + /// >>> + /// ``` >>> + /// let v = kernel::kvec![1, 2, 3]?; >>> + /// let mut it = v.into_iter(); >>> + /// >>> + /// assert_eq!(it.next(), Some(1)); >>> + /// >>> + /// let v = it.collect(GFP_KERNEL); >>> + /// assert_eq!(v, [2, 3]); >>> + /// >>> + /// # Ok::<(), Error>(()) >>> + /// ``` >>> + pub fn collect(self, flags: Flags) -> Vec<T, A> {
On Wed, Sep 11, 2024 at 08:53:24AM +0000, Benno Lossin wrote: > On 11.09.24 02:22, Danilo Krummrich wrote: > > On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote: > >> On 16.08.24 02:10, Danilo Krummrich wrote: > >>> + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. > >>> + /// > >>> + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait > >>> + /// in the kernel, namely: > >>> + /// > >>> + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special > >>> + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. > >>> + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` > >>> + /// doesn't require this type to be `'static`. > >>> + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence > >>> + /// we can't properly handle allocation failures. > >>> + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation > >>> + /// flags. > >>> + /// > >>> + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a > >>> + /// `Vec` again. > >> > >> I think it's great that you include this in the code, but I don't think > >> that it should be visible in the documentation, > > > > Why not? I think this information is valuable for users of this API. > > If you want to keep it, then I don't mind, but I would still move it > underneath `Examples` and add a section header `# Implementation > Details` or similar. Sure, we can do that. > > --- > Cheers, > Benno > > >> can you move it under > >> the `Examples` section and turn it into normal comments? > >> > >>> + /// > >>> + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing > >>> + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. > >>> + /// > >>> + /// # Examples > >>> + /// > >>> + /// ``` > >>> + /// let v = kernel::kvec![1, 2, 3]?; > >>> + /// let mut it = v.into_iter(); > >>> + /// > >>> + /// assert_eq!(it.next(), Some(1)); > >>> + /// > >>> + /// let v = it.collect(GFP_KERNEL); > >>> + /// assert_eq!(v, [2, 3]); > >>> + /// > >>> + /// # Ok::<(), Error>(()) > >>> + /// ``` > >>> + pub fn collect(self, flags: Flags) -> Vec<T, A> { >
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs index 3b79f977b65e..ad96f4c3af9e 100644 --- a/rust/kernel/alloc/kvec.rs +++ b/rust/kernel/alloc/kvec.rs @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A> fn as_raw_mut_slice(&mut self) -> *mut [T] { ptr::slice_from_raw_parts_mut(self.ptr, self.len) } + + fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) { + let me = ManuallyDrop::new(self); + let ptr = me.ptr; + let buf = me.buf; + let len = me.len; + let cap = me.cap; + (ptr, buf, len, cap) + } + + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. + /// + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait + /// in the kernel, namely: + /// + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` + /// doesn't require this type to be `'static`. + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence + /// we can't properly handle allocation failures. + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation + /// flags. + /// + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a + /// `Vec` again. + /// + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. + /// + /// # Examples + /// + /// ``` + /// let v = kernel::kvec![1, 2, 3]?; + /// let mut it = v.into_iter(); + /// + /// assert_eq!(it.next(), Some(1)); + /// + /// let v = it.collect(GFP_KERNEL); + /// assert_eq!(v, [2, 3]); + /// + /// # Ok::<(), Error>(()) + /// ``` + pub fn collect(self, flags: Flags) -> Vec<T, A> { + let (mut ptr, buf, len, mut cap) = self.into_raw_parts(); + let has_advanced = ptr != buf.as_ptr(); + + if has_advanced { + // SAFETY: Copy the contents we have advanced to at the beginning of the buffer. + // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is + // guaranteed to be smaller than `buf.add(cap)`. + unsafe { ptr::copy(ptr, buf.as_ptr(), len) }; + ptr = buf.as_ptr(); + } + + // This can never fail, `len` is guaranteed to be smaller than `cap`. + let layout = core::alloc::Layout::array::<T>(len).unwrap(); + + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves + // it as it is. + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } { + // If we fail to shrink, which likely can't even happen, continue with the existing + // buffer. + Err(_) => ptr, + Ok(ptr) => { + cap = len; + ptr.as_ptr().cast() + } + }; + + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to + // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed + // to point to the start of the backing buffer. `cap` is either the original capacity or, + // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since + // `into_iter` has been called on the original `Vec`. + unsafe { Vec::from_raw_parts(ptr, len, cap) } + } } impl<T, A> Iterator for IntoIter<T, A>
Currently, we can't implement `FromIterator`. There are a couple of issues with this trait in the kernel, namely: - Rust's specialization feature is unstable. This prevents us to optimze for the special case where `I::IntoIter` equals `Vec`'s `IntoIter` type. - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` doesn't require this type to be `'static`. - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence we can't properly handle allocation failures. - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation flags. Instead, provide `IntoIter::collect`, such that we can at least convert `IntoIter` into a `Vec` again. Reviewed-by: Alice Ryhl <aliceryhl@google.com> Signed-off-by: Danilo Krummrich <dakr@kernel.org> --- rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+)