|
36 | 36 | //! goes out of scope and is invalidated.
|
37 | 37 | //!
|
38 | 38 | //! Note that as long as you don't use [`unsafe`], it's impossible to create or misuse a pinned
|
39 |
| -//! value in a way that will produce unsoundness. See the documentation of [`Pin<Ptr>`] for more |
| 39 | +//! value in a way that is unsound. See the documentation of [`Pin<Ptr>`] for more |
40 | 40 | //! information on the practicalities of how to pin a value and how to use that pinned value from a
|
41 | 41 | //! user's perspective without using [`unsafe`].
|
42 | 42 | //!
|
|
179 | 179 | //!
|
180 | 180 | //! The second option is a viable solution to the problem for some use cases, in particular
|
181 | 181 | //! for self-referrential types. Under this model, any type that has an address sensitive state
|
182 |
| -//! would ultimately store its data in something like a [`Box<T>`] and then carefully manage |
183 |
| -//! the access to that data internally to ensure no *moves* or other invalidation occur, then |
| 182 | +//! would ultimately store its data in something like a [`Box<T>`], carefully manage internal |
| 183 | +//! access to that data to ensure no *moves* or other invalidation occurs, and finally |
184 | 184 | //! provide a safe interface on top.
|
185 | 185 | //!
|
186 | 186 | //! There are a couple of linked disadvantages to using this model. The core issue is a lack
|
|
190 | 190 | //! rather build a shared understanding of the problem space and encode that understanding into a
|
191 | 191 | //! shared interface to solve it which everyone helps validate.
|
192 | 192 | //!
|
193 |
| -//! In addition, and the key issue that drove Rust towards developing another solution, is that |
194 |
| -//! in this model, each individual object must assume it is *on its own* to ensure that its data |
195 |
| -//! does not become *moved* or otherwise invalidated. Since there is no shared contract between |
196 |
| -//! values of different types, an object cannot assume that others interacting with it will |
197 |
| -//! properly respect the invariants around interacting with its data and must therefore protect |
198 |
| -//! it from everyone. Because of this, *composition* of address-sensitive types requires at least |
199 |
| -//! a level of pointer indirection each time a new object is added to the mix (and, practically, a |
200 |
| -//! heap allocation). |
201 |
| -//! |
202 |
| -//! This is particularly a problem when one considers, for exapmle, the implications of composing |
203 |
| -//! together the [`Future`]s which will eventaully make up an asynchronous task (including |
204 |
| -//! address-sensitive `async fn` state machines). It is plausible that there could be many layers |
205 |
| -//! of [`Future`]s composed together, including multiple layers of `async fn`s handling different |
206 |
| -//! parts of a task. It was deemed unacceptable to force indirection and allocation for each layer |
207 |
| -//! of composition in this case. |
| 193 | +//! In addition, in this model, each individual object must assume it is *on its own* to ensure |
| 194 | +//! that its data does not become *moved* or otherwise invalidated. Since there is no shared |
| 195 | +//! contract between values of different types, an object cannot assume that others interacting |
| 196 | +//! with it will properly respect the invariants around interacting with its data and must |
| 197 | +//! therefore protect it from everyone. Because of this, *composition* of address-sensitive types |
| 198 | +//! requires at least a level of pointer indirection each time a new object is added to the mix |
| 199 | +//! (and, practically, a heap allocation). |
| 200 | +//! |
| 201 | +//! This is the key thing that drove Rust towards a different model. It is particularly a problem |
| 202 | +//! when one considers, for exapmle, the implications of composing together the [`Future`]s which |
| 203 | +//! will eventaully make up an asynchronous task (including address-sensitive `async fn` state |
| 204 | +//! machines). It is plausible that there could be many layers of [`Future`]s composed together, |
| 205 | +//! including multiple layers of `async fn`s handling different parts of a task. It was deemed |
| 206 | +//! unacceptable to force indirection and allocation for each layer of composition in this case. |
208 | 207 | //!
|
209 | 208 | //! [`Pin<Ptr>`] is an implementation of the third option. It allows us to solve the issues
|
210 | 209 | //! discussed with the second option by building a *shared contractual language* around the
|
|
237 | 236 | //! value which does not actually satisfy the invariants that a pinned value must satisfy, and in
|
238 | 237 | //! this way lead to undefined behavior even in (from that point) fully safe code. Similarly, using
|
239 | 238 | //! [`unsafe`], one may get access to a bare [`&mut T`] from a [`Pin<Ptr>`] and
|
240 |
| -//! use that to invalidly *move* pinned the value out. It is the job of the user of the |
| 239 | +//! use that to invalidly *move* the pinned value out. It is the job of the user of the |
241 | 240 | //! [`unsafe`] parts of the [`Pin`] API to ensure these invariants are not violated.
|
242 | 241 | //!
|
243 | 242 | //! This differs from e.g. [`UnsafeCell`] which changes the semantics of a program's compiled
|
|
496 | 495 | //! *Moving* or otherwise invalidating an element's data would invalidate the pointers back to it
|
497 | 496 | //! which are stored in the elements ahead and behind it. Thus, in order to soundly dereference
|
498 | 497 | //! the pointers stored to the next and previous elements, we must satisfy the guarantee that
|
499 |
| -//! nothing has invalidated those pointers (which point to data which we do not own). |
| 498 | +//! nothing has invalidated those pointers (which point to data that we do not own). |
500 | 499 | //!
|
501 | 500 | //! Moreover, the [`Drop`][Drop] implementation of each element must in some way notify its
|
502 | 501 | //! predecessor and successor elements that it should be removed from the list before it is fully
|
503 | 502 | //! destroyed, otherwise the pointers back to it would again become invalidated.
|
504 | 503 | //!
|
505 | 504 | //! Crucially, this means we have to be able to rely on [`drop`] always being called before an
|
506 | 505 | //! element is invalidated. If an element could be deallocated or otherwise invalidated without
|
507 |
| -//! calling [`drop`], the pointers to it which are stored in its neighboring elements would |
| 506 | +//! calling [`drop`], the pointers to it stored in its neighboring elements would |
508 | 507 | //! become invalid, which would break the data structure.
|
509 | 508 | //!
|
510 | 509 | //! Therefore, pinning data also comes with [the "`Drop` guarantee"][drop-guarantee].
|
|
513 | 512 | //! [subtle-details]: self#subtle-details-and-the-drop-guarantee
|
514 | 513 | //! [drop-guarantee]: self#subtle-details-and-the-drop-guarantee
|
515 | 514 | //!
|
516 |
| -//! The purpose of pinning is not *just* to prevent a value from being *moved*, but rather more |
| 515 | +//! The purpose of pinning is not *just* to prevent a value from being *moved*, but more |
517 | 516 | //! generally to be able to rely on the pinned value *remaining valid **at a specific place*** in
|
518 | 517 | //! memory.
|
519 | 518 | //!
|
520 | 519 | //! To do so, pinning a value adds an *additional* invariant that must be upheld in order for use
|
521 |
| -//! of the pinned data to be valid, on top of the ones that must be upheld for a non-pinned value of |
522 |
| -//! the same type: |
| 520 | +//! of the pinned data to be valid, on top of the ones that must be upheld for a non-pinned value |
| 521 | +//! of the same type to be valid: |
523 | 522 | //!
|
524 | 523 | //! From the moment a value is pinned by constructing a [`Pin`]ning pointer to it, that value
|
525 | 524 | //! must *remain, **valid***, at that same address in memory, *until its [`drop`] handler is
|
526 | 525 | //! called.*
|
527 | 526 | //!
|
528 |
| -//! There is some subtlety to this which we have not yet talked about in detail. The invariant above |
529 |
| -//! means that, yes, |
| 527 | +//! There is some subtlety to this which we have not yet talked about in detail. The invariant |
| 528 | +//! described above means that, yes, |
530 | 529 | //!
|
531 | 530 | //! 1. The value must not be moved out of its location in memory
|
532 | 531 | //!
|
|
540 | 539 | //! ## `Drop` guarantee
|
541 | 540 | //!
|
542 | 541 | //! There needs to be a way for a pinned value to notify any code that is relying on its pinned
|
543 |
| -//! status that it is about to be destroyed, so that such code can remove its address from their |
544 |
| -//! data structures or otherwise change their behavior with the knowledge that they can no longer |
545 |
| -//! rely on that value existing at the same location. |
| 542 | +//! status that it is about to be destroyed. In this way, the dependent code can remove the |
| 543 | +//! pinned value's address from its data structures or otherwise change its behavior with the |
| 544 | +//! knowledge that it can no longer rely on that value existing at the location it was pinned to. |
546 | 545 | //!
|
547 | 546 | //! Thus, in any situation where we may want to overwrite a pinned value, that value's [`drop`] must
|
548 | 547 | //! be called beforehand (unless the pinned value implements [`Unpin`], in which case we can ignore
|
549 | 548 | //! all of [`Pin`]'s guarantees, as usual).
|
550 | 549 | //!
|
551 |
| -//! The most common storage-reuse situation is when a value on the stack is destroyed as part of a |
552 |
| -//! function return, or when heap storage is freed. In both cases, [`drop`] gets run for us |
553 |
| -//! by Rust when using standard safe code. However, for heap or otherwise custom-allocated storage, |
554 |
| -//! [`unsafe`] code must make sure to call [`ptr::drop_in_place`] before deallocating and re-using |
555 |
| -//! said storage. |
| 550 | +//! The most common storage-reuse situations occur when a value on the stack is destroyed as part |
| 551 | +//! of a function return and when heap storage is freed. In both cases, [`drop`] gets run for us |
| 552 | +//! by Rust when using standard safe code. However, for manual heap allocations or otherwise |
| 553 | +//! custom-allocated storage, [`unsafe`] code must make sure to call [`ptr::drop_in_place`] before |
| 554 | +//! deallocating and re-using said storage. |
556 | 555 | //!
|
557 |
| -//! However, reuse can happen even if no storage is (de-)allocated. For example, if we had an |
558 |
| -//! [`Option`] which contained a `Some(v)` where `v` is a pinned value, then `v` would be |
559 |
| -//! invalidated by setting that option to `None`. |
| 556 | +//! In addition, storage "re-use"/invalidation can happen even if no storage is (de-)allocated. |
| 557 | +//! For example, if we had an [`Option`] which contained a `Some(v)` where `v` is pinned, then `v` |
| 558 | +//! would be invalidated by setting that option to `None`. |
560 | 559 | //!
|
561 |
| -//! Similarly, if a [`Vec`] was used to store pinned values and [`Vec::set_len`] is used to manually |
562 |
| -//! "kill" some elements of a vector, all value "killed" would become invalidated. |
| 560 | +//! Similarly, if a [`Vec`] was used to store pinned values and [`Vec::set_len`] was used to |
| 561 | +//! manually "kill" some elements of a vector, all of the items "killed" would become invalidated, |
| 562 | +//! which would be *undefined behavior* if those items were pinned. |
563 | 563 | //!
|
564 |
| -//! Both of these cases are somewhat contrived, but it is crucial |
565 |
| -//! to remember that [`Pin`]ned data *must* be [`drop`]ped before it is invalidated, as a matter of |
566 |
| -//! soundness, not just to prevent memory leaks. As a corollary, the following code can *never* be |
567 |
| -//! made safe: |
| 564 | +//! Both of these cases are somewhat contrived, but it is crucial to remember that [`Pin`]ned data |
| 565 | +//! *must* be [`drop`]ped before it is invalidated; not just to prevent memory leaks, but as a |
| 566 | +//! matter of soundness. As a corollary, the following code can *never* be made safe: |
568 | 567 | //!
|
569 | 568 | //! ```rust
|
570 | 569 | //! # use std::mem::ManuallyDrop;
|
|
637 | 636 | //!
|
638 | 637 | //! ### Implementing [`Drop`] for pointer types which will be used as [`Pin`]ning pointers
|
639 | 638 | //!
|
640 |
| -//! It should further be noted that creating a pinning pointer of some type `Ptr` to some underlying |
641 |
| -//! `T` which is not `Unpin` *also* carries with it implications on the way that `Ptr` type must |
642 |
| -//! implement [`Drop`] (as well as [`Deref`] and [`DerefMut`])! When implementing a pointer type |
643 |
| -//! that may be used as a pinning pointer, you must also take the same care described above not to |
644 |
| -//! *move* out of or otherwise invalidate the pointee during [`Drop`], [`Deref`], or [`DerefMut`] |
| 639 | +//! It should further be noted that creating a pinning pointer of some type `Ptr` *also* carries |
| 640 | +//! with it implications on the way that `Ptr` type must implement [`Drop`] |
| 641 | +//! (as well as [`Deref`] and [`DerefMut`])! When implementing a pointer type that may be used as |
| 642 | +//! a pinning pointer, you must also take the same care described above not to *move* out of or |
| 643 | +//! otherwise invalidate the pointee during [`Drop`], [`Deref`], or [`DerefMut`] |
645 | 644 | //! implementations.
|
646 | 645 | //!
|
647 | 646 | //! ## "Assigning" pinned data
|
648 | 647 | //!
|
649 |
| -//! Although in general it is not valid to swap data through a [`Pin<Ptr>`], or assign from |
650 |
| -//! a [`Pin<Ptr>`], for the same reason that a *move* is invalid, there is no particular reason |
651 |
| -//! to disallow doing it with specialized functions, as long as they know how to update all |
652 |
| -//! uses of the pinned address (and any other `unsafe`-assumed invariants). For [`Unmovable`] ( |
653 |
| -//! from the example above) we could write |
| 648 | +//! Although in general it is not valid to swap data or assign through a [`Pin<Ptr>`] for the same |
| 649 | +//! reason that reusing a pinned object's memory is invalid, it is possible to do validly when |
| 650 | +//! implemented with special care for the needs of the exact data structure which is being |
| 651 | +//! modified. For example, the assigning function must know how to update all uses of the pinned |
| 652 | +//! address (and any other invariants necessary to satisfy validity for that type). For |
| 653 | +//! [`Unmovable`] (from the example above), we could write an assignment function like so: |
654 | 654 | //!
|
655 | 655 | //! ```
|
656 | 656 | //! # use std::pin::Pin;
|
|
689 | 689 | //! Even though we can't have the compiler do the assignment for us, it's possible to write
|
690 | 690 | //! such specialized functions for types that might need it.
|
691 | 691 | //!
|
692 |
| -//! Note that it _is_ possible to assign through a [`Pin<P>`] by way of [`Pin::set()`]. This does |
693 |
| -//! not violate any guarantees, since it will run [`drop`] on the pointee value before assigning |
694 |
| -//! the new value. |
| 692 | +//! Note that it _is_ possible to assign generically through a [`Pin<P>`] by way of [`Pin::set()`]. |
| 693 | +//! This does not violate any guarantees, since it will run [`drop`] on the pointee value before |
| 694 | +//! assigning the new value. Thus, the [`drop`] implementation still has a chance to perform the |
| 695 | +//! necessary notifications to dependent values before the memory location of the original pinned |
| 696 | +//! value is overwritten. |
695 | 697 | //!
|
696 | 698 | //! ## Projections and Structural Pinning
|
697 | 699 | //! [structural-pinning]: self#projections-and-structural-pinning
|
698 | 700 | //!
|
699 |
| -//! With ordinary structs, it is natural that we want to add *projection* methods |
700 |
| -//! that select one of the fields: |
| 701 | +//! With ordinary structs, it is natural that we want to add *projection* methods that allow |
| 702 | +//! borrowing one or more of the inner fields of a struct when the caller has access to a |
| 703 | +//! borrow of the whole struct: |
701 | 704 | //!
|
702 | 705 | //! ```
|
703 | 706 | //! # struct Field;
|
|
712 | 715 | //! ```
|
713 | 716 | //!
|
714 | 717 | //! When working with address-sensitive types, it's not obvious what the signature of these
|
715 |
| -//! functions should be. If `field` takes <code>self: [Pin]<[&mut Struct][&mut]></code>, should it return |
716 |
| -//! [`&mut Field`] or <code>[Pin]<[`&mut Field`]></code>? This question also arises with `enum`s and |
717 |
| -//! wrapper types like [`Vec<T>`], [`Box<T>`], and [`RefCell<T>`]. (This question |
718 |
| -//! applies just as well to shared references, but we'll examine the more common case |
719 |
| -//! of mutable references for illustration). |
| 718 | +//! functions should be. If `field` takes <code>self: [Pin]<[&mut Struct][&mut]></code>, should it |
| 719 | +//! return [`&mut Field`] or <code>[Pin]<[`&mut Field`]></code>? This question also arises with |
| 720 | +//! `enum`s and wrapper types like [`Vec<T>`], [`Box<T>`], and [`RefCell<T>`]. (This question |
| 721 | +//! applies just as well to shared references, but we'll examine the more common case of mutable |
| 722 | +//! references for illustration). |
720 | 723 | //!
|
721 |
| -//! It turns out that it's up to the author of `Struct` to decide which type the projection |
722 |
| -//! should produce. The choice must be *consistent* though: each field should only ever |
723 |
| -//! be projected as pinned or unpinned; both together will likely be unsound! |
| 724 | +//! It turns out that it's up to the author of `Struct` to decide which type the "projection" |
| 725 | +//! should produce. The choice must be *consistent* though: if a pin is projected to a field |
| 726 | +//! in one place, then it should very likely not be exposed elsewhere without projecting the |
| 727 | +//! pin. |
724 | 728 | //!
|
725 | 729 | //! As the author of a data structure, you get to decide for each field whether pinning
|
726 | 730 | //! "propagates" to this field or not. Pinning that propagates is also called "structural",
|
727 | 731 | //! because it follows the structure of the type.
|
728 | 732 | //!
|
729 |
| -//! The choice of whether to pin depends on how the type is being used. If unsafe code |
| 733 | +//! The choice of whether to pin depends on how the type is being used. If [`unsafe`] code |
730 | 734 | //! that consumes <code>[Pin]\<[&mut Struct][&mut]></code> also needs to take note of
|
731 | 735 | //! the address of the field itself, it may be evidence that that field is structurally
|
732 | 736 | //! pinned. Unfortunately, there are no hard-and-fast rules.
|
733 | 737 | //!
|
734 | 738 | //! ### Choosing pinning *not to be* structural for `field`...
|
735 | 739 | //!
|
736 |
| -//! While counter-intuitive, it's actually the easier choice: if a |
737 |
| -//! <code>[Pin]<[&mut] Field></code> is never created, nothing can go wrong (well, so long as no |
738 |
| -//! unsound `unsafe` code is written which expects the invariants of such a [`Pin`] to be upheld |
739 |
| -//! without actually using pinning to guarantee them)! So, if you decide that some field does not |
| 740 | +//! While counter-intuitive, it's actually the easier choice: if you do not expose a |
| 741 | +//! <code>[Pin]<[&mut] Field></code>, then no code must be written assuming that the field is |
| 742 | +//! pinned and so nothing can go wrong. So, if you decide that some field does not |
740 | 743 | //! have structural pinning, all you have to ensure is that you never create pinning
|
741 |
| -//! reference to that field. |
| 744 | +//! reference to that field. This does of course also mean that if you decide a field does not |
| 745 | +//! have structural pinning, you must not write [`unsafe`] code that assumes (invalidly) that the |
| 746 | +//! field *is* structurally pinned! |
742 | 747 | //!
|
743 | 748 | //! Fields without structural pinning may have a projection method that turns
|
744 | 749 | //! <code>[Pin]<[&mut] Struct></code> into [`&mut Field`]:
|
|
749 | 754 | //! # struct Struct { field: Field }
|
750 | 755 | //! impl Struct {
|
751 | 756 | //! fn field(self: Pin<&mut Self>) -> &mut Field {
|
752 |
| -//! // This is okay because `field` is never considered pinned. |
| 757 | +//! // This is okay because `field` is never considered pinned, therefore we do not |
| 758 | +//! // need to uphold any pinning guarantees for this field in particular. Of course, |
| 759 | +//! // we must not elsewhere assume this field *is* pinned if we choose to expose |
| 760 | +//! // such a method! |
753 | 761 | //! unsafe { &mut self.get_unchecked_mut().field }
|
754 | 762 | //! }
|
755 | 763 | //! }
|
756 | 764 | //! ```
|
757 | 765 | //!
|
758 |
| -//! You may also <code>impl [Unpin] for Struct {}</code> *even if* the type of `field` |
759 |
| -//! is not [`Unpin`]. What that type thinks about pinning is not relevant |
760 |
| -//! when no <code>[Pin]<[&mut] Field></code> is ever created. |
| 766 | +//! You may also in this situation <code>impl [Unpin] for Struct {}</code> *even if* the type of |
| 767 | +//! `field` is not [`Unpin`]. Since we have explicitly chosen not to care about pinning guarantees |
| 768 | +//! for `field`, the way `field`'s type interacts with pinning is no longer relevant in the |
| 769 | +//! context of its use in `Struct`. |
761 | 770 | //!
|
762 | 771 | //! ### Choosing pinning *to be* structural for `field`...
|
763 | 772 | //!
|
|
0 commit comments