Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sparse the default protocol for crates.io #11791

Merged
merged 1 commit into from
Mar 5, 2023

Conversation

arlosi
Copy link
Contributor

@arlosi arlosi commented Mar 2, 2023

Changes the default protocol for accessing crates.io to sparse.

The protocol can be switched back to git via the configuration key registries.crates-io.protocol = git

Closes #10965

@rustbot
Copy link
Collaborator

rustbot commented Mar 2, 2023

r? @ehuss

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added A-documenting-cargo-itself Area: Cargo's documentation A-unstable Area: nightly unstable support S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 2, 2023
@arlosi arlosi added A-sparse-registry Area: http sparse registries and removed A-documenting-cargo-itself Area: Cargo's documentation A-unstable Area: nightly unstable support labels Mar 2, 2023
@arlosi
Copy link
Contributor Author

arlosi commented Mar 2, 2023

Note that we should not complete this until after the branch for 1.70.0 to maximize testing time (6 weeks on nightly, 6 weeks on beta).

@ehuss
Copy link
Contributor

ehuss commented Mar 3, 2023

Thanks, looks good to me.
r=ehuss some time next week (after beta branch) if there aren't any objections or anything raised until then.

@weihanglo
Copy link
Member

A new nightly was branched off in the midnight. I think we can merge this?
rust-lang/rust#108740

$ rustc -vV # after `rustup update`
rustc 1.70.0-nightly (f15f0ea73 2023-03-04)
binary: rustc
commit-hash: f15f0ea73972786e426732c5b92ba9a904b866c4
commit-date: 2023-03-04
host: aarch64-apple-darwin
release: 1.70.0-nightly
LLVM version: 15.0.7

@arlosi
Copy link
Contributor Author

arlosi commented Mar 5, 2023

@bors r=ehuss

@bors
Copy link
Contributor

bors commented Mar 5, 2023

📌 Commit 78d4f2c has been approved by ehuss

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 5, 2023
@bors
Copy link
Contributor

bors commented Mar 5, 2023

⌛ Testing commit 78d4f2c with merge 234d9f6...

@bors
Copy link
Contributor

bors commented Mar 5, 2023

☀️ Test successful - checks-actions
Approved by: ehuss
Pushing 234d9f6 to master...

@bors bors merged commit 234d9f6 into rust-lang:master Mar 5, 2023
weihanglo added a commit to weihanglo/rust that referenced this pull request Mar 7, 2023
23 commits in 9880b408a3af50c08fab3dbf4aa2a972df71e951..c1334b059c6dcceab3c10c81413f79bb832c8d9d
2023-02-28 19:39:39 +0000 to 2023-03-07 19:21:50 +0000

- Add `CARGO_PKG_README` (rust-lang/cargo#11645)
- path dependency: fix cargo-util version (rust-lang/cargo#11807)
- Adding display of which target failed to compile (rust-lang/cargo#11636)
- Fix `CARGO_CFG_` vars for configs defined both with and without value (rust-lang/cargo#11790)
- Breaking endless loop on cyclic features in added dependency in cargo-add (rust-lang/cargo#11805)
- Enhance the doc of timing report with graphs (rust-lang/cargo#11798)
- Make `sparse` the default protocol for crates.io (rust-lang/cargo#11791)
- Use sha2 to calculate SHA256 (rust-lang/cargo#11795)
- gitoxide progress bar fixes (rust-lang/cargo#11800)
- Check publish_to_alt_registry publish content (rust-lang/cargo#11799)
- chore: fix missing files in autolabel trigger_files (rust-lang/cargo#11797)
- chore: Update base64 (rust-lang/cargo#11796)
- Fix some doc typos (rust-lang/cargo#11794)
- chore(ci): Enforce cargo-deny in CI (rust-lang/cargo#11761)
- Some cleanup for unstable docs (rust-lang/cargo#11793)
- gitoxide integration: fetch (rust-lang/cargo#11448)
- patch can conflict on not activated packages (rust-lang/cargo#11770)
- fix(toml): Provide a way to show unused manifest keys for dependencies (rust-lang/cargo#11630)
- Improve error for missing crate in --offline mode for sparse index (rust-lang/cargo#11783)
- feat(resolver): `-Zdirect-minimal-versions` (rust-lang/cargo#11688)
- feat: Use test name for dir when running tests (rust-lang/cargo#11738)
- Jobserver cleanup (rust-lang/cargo#11764)
- Fix help string for  "--charset" option of "cargo tree" (rust-lang/cargo#11785)

Note that some 3rd-party licensing allowed list changed due to the
introducion of `gix` dependency
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 8, 2023
Update cargo

25 commits in 9880b408a3af50c08fab3dbf4aa2a972df71e951..7d3033d2e59383fd76193daf9423c3d141972a7d
2023-02-28 19:39:39 +0000 to 2023-03-08 17:05:08 +0000

- Revert "rust-lang/cargo#11738" - Use test name for dir when running tests (rust-lang/cargo#11812)
- Update CHANGELOG for 1.68 backports (rust-lang/cargo#11810)
- Add `CARGO_PKG_README` (rust-lang/cargo#11645)
- path dependency: fix cargo-util version (rust-lang/cargo#11807)
- Adding display of which target failed to compile (rust-lang/cargo#11636)
- Fix `CARGO_CFG_` vars for configs defined both with and without value (rust-lang/cargo#11790)
- Breaking endless loop on cyclic features in added dependency in cargo-add (rust-lang/cargo#11805)
- Enhance the doc of timing report with graphs (rust-lang/cargo#11798)
- Make `sparse` the default protocol for crates.io (rust-lang/cargo#11791)
- Use sha2 to calculate SHA256 (rust-lang/cargo#11795)
- gitoxide progress bar fixes (rust-lang/cargo#11800)
- Check publish_to_alt_registry publish content (rust-lang/cargo#11799)
- chore: fix missing files in autolabel trigger_files (rust-lang/cargo#11797)
- chore: Update base64 (rust-lang/cargo#11796)
- Fix some doc typos (rust-lang/cargo#11794)
- chore(ci): Enforce cargo-deny in CI (rust-lang/cargo#11761)
- Some cleanup for unstable docs (rust-lang/cargo#11793)
- gitoxide integration: fetch (rust-lang/cargo#11448)
- patch can conflict on not activated packages (rust-lang/cargo#11770)
- fix(toml): Provide a way to show unused manifest keys for dependencies (rust-lang/cargo#11630)
- Improve error for missing crate in --offline mode for sparse index (rust-lang/cargo#11783)
- feat(resolver): `-Zdirect-minimal-versions` (rust-lang/cargo#11688)
- feat: Use test name for dir when running tests (rust-lang/cargo#11738)
- Jobserver cleanup (rust-lang/cargo#11764)
- Fix help string for  "--charset" option of "cargo tree" (rust-lang/cargo#11785)

---

~~This update is primarily for making rust-lang/cargo#11630 into 1.69~~ (will file a beta backport then). However, just look into the licenses and dependencies permitted list, it looks a bit unfortunate but inevitable I guess?

r? `@ehuss`
cc `@Muscraft`
@ehuss ehuss added this to the 1.70.0 milestone Mar 9, 2023
saethlin pushed a commit to saethlin/miri that referenced this pull request Mar 11, 2023
Update cargo

25 commits in 9880b408a3af50c08fab3dbf4aa2a972df71e951..7d3033d2e59383fd76193daf9423c3d141972a7d
2023-02-28 19:39:39 +0000 to 2023-03-08 17:05:08 +0000

- Revert "rust-lang/cargo#11738" - Use test name for dir when running tests (rust-lang/cargo#11812)
- Update CHANGELOG for 1.68 backports (rust-lang/cargo#11810)
- Add `CARGO_PKG_README` (rust-lang/cargo#11645)
- path dependency: fix cargo-util version (rust-lang/cargo#11807)
- Adding display of which target failed to compile (rust-lang/cargo#11636)
- Fix `CARGO_CFG_` vars for configs defined both with and without value (rust-lang/cargo#11790)
- Breaking endless loop on cyclic features in added dependency in cargo-add (rust-lang/cargo#11805)
- Enhance the doc of timing report with graphs (rust-lang/cargo#11798)
- Make `sparse` the default protocol for crates.io (rust-lang/cargo#11791)
- Use sha2 to calculate SHA256 (rust-lang/cargo#11795)
- gitoxide progress bar fixes (rust-lang/cargo#11800)
- Check publish_to_alt_registry publish content (rust-lang/cargo#11799)
- chore: fix missing files in autolabel trigger_files (rust-lang/cargo#11797)
- chore: Update base64 (rust-lang/cargo#11796)
- Fix some doc typos (rust-lang/cargo#11794)
- chore(ci): Enforce cargo-deny in CI (rust-lang/cargo#11761)
- Some cleanup for unstable docs (rust-lang/cargo#11793)
- gitoxide integration: fetch (rust-lang/cargo#11448)
- patch can conflict on not activated packages (rust-lang/cargo#11770)
- fix(toml): Provide a way to show unused manifest keys for dependencies (rust-lang/cargo#11630)
- Improve error for missing crate in --offline mode for sparse index (rust-lang/cargo#11783)
- feat(resolver): `-Zdirect-minimal-versions` (rust-lang/cargo#11688)
- feat: Use test name for dir when running tests (rust-lang/cargo#11738)
- Jobserver cleanup (rust-lang/cargo#11764)
- Fix help string for  "--charset" option of "cargo tree" (rust-lang/cargo#11785)

---

~~This update is primarily for making rust-lang/cargo#11630 into 1.69~~ (will file a beta backport then). However, just look into the licenses and dependencies permitted list, it looks a bit unfortunate but inevitable I guess?

r? `@ehuss`
cc `@Muscraft`
@ehuss ehuss added the relnotes Release-note worthy label Apr 9, 2023
@arlosi arlosi deleted the sparse-stable branch April 21, 2023 23:04
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Jun 3, 2023
Pkgsrc changes:
 * Adjust patches and cargo checksums to new versions.
 * Adjust to not cross-build to 8.0, due to LLVM using c++17,
   so adjust USE_LANGUAGES.

Upstream changes:

Version 1.70.0 (2023-06-01)
==========================

Language
--------
- [Relax ordering rules for `asm!` operands]
  (rust-lang/rust#105798)
- [Properly allow macro expanded `format_args` invocations to uses
  captures] (rust-lang/rust#106505)
- [Lint ambiguous glob re-exports]
  (rust-lang/rust#107880)
- [Perform const and unsafe checking for expressions in `let _ =
  expr` position.]
  (rust-lang/rust#102256)

Compiler
--------
- [Extend -Cdebuginfo with new options and named aliases]
  (rust-lang/rust#109808)
  This provides a smaller version of debuginfo for cases that only
  need line number information (`-Cdebuginfo=line-tables-only`),
  which may eventually become the default for `-Cdebuginfo=1`.
- [Make `unused_allocation` lint against `Box::new` too]
  (rust-lang/rust#104363)
- [Detect uninhabited types early in const eval]
  (rust-lang/rust#109435)
- [Switch to LLD as default linker for {arm,thumb}v4t-none-eabi]
  (rust-lang/rust#109721)
- [Add tier 3 target `loongarch64-unknown-linux-gnu`]
  (rust-lang/rust#96971)
- [Add tier 3 target for `i586-pc-nto-qnx700` (QNX Neutrino RTOS,
  version 7.0)] (rust-lang/rust#109173),
- [Insert alignment checks for pointer dereferences as debug assertions]
  (rust-lang/rust#98112)
  This catches undefined behavior at runtime, and may cause existing
  code to fail.

Refer to Rust's [platform support page][platform-support-doc]
for more information on Rust's tiered platform support.

Libraries
---------
- [Document NonZeroXxx layout guarantees]
  (rust-lang/rust#94786)
- [Windows: make `Command` prefer non-verbatim paths]
  (rust-lang/rust#96391)
- [Implement Default for some alloc/core iterators]
  (rust-lang/rust#99929)
- [Fix handling of trailing bare CR in str::lines]
  (rust-lang/rust#100311)
- [allow negative numeric literals in `concat!`]
  (rust-lang/rust#106844)
- [Add documentation about the memory layout of `Cell`]
  (rust-lang/rust#106921)
- [Use `partial_cmp` to implement tuple `lt`/`le`/`ge`/`gt`]
  (rust-lang/rust#108157)
- [Stabilize `atomic_as_ptr`]
  (rust-lang/rust#108419)
- [Stabilize `nonnull_slice_from_raw_parts`]
  (rust-lang/rust#97506)
- [Partial stabilization of `once_cell`]
  (rust-lang/rust#105587)
- [Stabilize `nonzero_min_max`]
  (rust-lang/rust#106633)
- [Flatten/inline format_args!() and (string and int) literal
  arguments into format_args!()]
  (rust-lang/rust#106824)
- [Stabilize movbe target feature]
  (rust-lang/rust#107711)
- [don't splice from files into pipes in io::copy]
  (rust-lang/rust#108283)
- [Add a builtin unstable `FnPtr` trait that is implemented for
  all function pointers]
  (rust-lang/rust#108080)
  This extends `Debug`, `Pointer`, `Hash`, `PartialEq`, `Eq`,
  `PartialOrd`, and `Ord` implementations for function pointers
  with all ABIs.

Stabilized APIs
---------------

- [`NonZero*::MIN/MAX`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroI8.html#associatedconstant.MIN)
- [`BinaryHeap::retain`]
  (https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.retain)
- [`Default for std::collections::binary_heap::IntoIter`]
  (https://doc.rust-lang.org/stable/std/collections/binary_heap/struct.IntoIter.html)
- [`Default for std::collections::btree_map::{IntoIter, Iter, IterMut}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoIter.html)
- [`Default for std::collections::btree_map::{IntoKeys, Keys}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoKeys.html)
- [`Default for std::collections::btree_map::{IntoValues, Values}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoKeys.html)
- [`Default for std::collections::btree_map::Range`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.Range.html)
- [`Default for std::collections::btree_set::{IntoIter, Iter}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_set/struct.IntoIter.html)
- [`Default for std::collections::btree_set::Range`]
  (https://doc.rust-lang.org/stable/std/collections/btree_set/struct.Range.html)
- [`Default for std::collections::linked_list::{IntoIter, Iter, IterMut}`]
  (https://doc.rust-lang.org/stable/alloc/collections/linked_list/struct.IntoIter.html)
- [`Default for std::vec::IntoIter`]
  (https://doc.rust-lang.org/stable/alloc/vec/struct.IntoIter.html#impl-Default-for-IntoIter%3CT,+A%3E)
- [`Default for std::iter::Chain`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Chain.html)
- [`Default for std::iter::Cloned`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Cloned.html)
- [`Default for std::iter::Copied`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Copied.html)
- [`Default for std::iter::Enumerate`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Enumerate.html)
- [`Default for std::iter::Flatten`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Flatten.html)
- [`Default for std::iter::Fuse`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Fuse.html)
- [`Default for std::iter::Rev`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Rev.html)
- [`Default for std::slice::Iter`]
  (https://doc.rust-lang.org/stable/std/slice/struct.Iter.html)
- [`Default for std::slice::IterMut`]
  (https://doc.rust-lang.org/stable/std/slice/struct.IterMut.html)
- [`Rc::into_inner`]
  (https://doc.rust-lang.org/stable/alloc/rc/struct.Rc.html#method.into_inner)
- [`Arc::into_inner`]
  (https://doc.rust-lang.org/stable/alloc/sync/struct.Arc.html#method.into_inner)
- [`std::cell::OnceCell`]
  (https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html)
- [`Option::is_some_and`]
  (https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.is_some_and)
- [`NonNull::slice_from_raw_parts`]
  (https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html#method.slice_from_raw_parts)
- [`Result::is_ok_and`]
  (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.is_ok_and)
- [`Result::is_err_and`]
  (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.is_err_and)
- [`std::sync::atomic::Atomic*::as_ptr`]
  (https://doc.rust-lang.org/stable/std/sync/atomic/struct.AtomicU8.html#method.as_ptr)
- [`std::io::IsTerminal`]
  (https://doc.rust-lang.org/stable/std/io/trait.IsTerminal.html)
- [`std::os::linux::net::SocketAddrExt`]
  (https://doc.rust-lang.org/stable/std/os/linux/net/trait.SocketAddrExt.html)
- [`std::os::unix::net::UnixDatagram::bind_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.bind_addr)
- [`std::os::unix::net::UnixDatagram::connect_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.connect_addr)
- [`std::os::unix::net::UnixDatagram::send_to_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.send_to_addr)
- [`std::os::unix::net::UnixListener::bind_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixListener.html#method.bind_addr)
- [`std::path::Path::as_mut_os_str`]
  (https://doc.rust-lang.org/stable/std/path/struct.Path.html#method.as_mut_os_str)
- [`std::sync::OnceLock`]
  (https://doc.rust-lang.org/stable/std/sync/struct.OnceLock.html)

Cargo
-----

- [Add `CARGO_PKG_README`]
  (rust-lang/cargo#11645)
- [Make `sparse` the default protocol for crates.io]
  (rust-lang/cargo#11791)
- [Accurately show status when downgrading dependencies]
  (rust-lang/cargo#11839)
- [Use registry.default for login/logout]
  (rust-lang/cargo#11949)
- [Stabilize `cargo logout`]
  (rust-lang/cargo#11950)

Misc
----

- [Stabilize rustdoc `--test-run-directory`]
  (rust-lang/rust#103682)

Compatibility Notes
-------------------

- [Prevent stable `libtest` from supporting `-Zunstable-options`]
  (rust-lang/rust#109044)
- [Perform const and unsafe checking for expressions in `let _ =
  expr` position.] (rust-lang/rust#102256)
- [WebAssembly targets enable `sign-ext` and `mutable-globals`
  features in codegen] (rust-lang/rust#109807)
  This may cause incompatibility with older execution environments.
- [Insert alignment checks for pointer dereferences as debug assertions]
  (rust-lang/rust#98112)
  This catches undefined behavior at runtime, and may cause existing
  code to fail.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they represent
significant improvements to the performance or internals of rustc and related
tools.

- [Upgrade to LLVM 16]
  (rust-lang/rust#109474)
- [Use SipHash-1-3 instead of SipHash-2-4 for StableHasher]
  (rust-lang/rust#107925)
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jul 10, 2023
Pkgsrc changes:
 * Adjust patches and cargo checksums to new versions.
 * Add support for NetBSD/riscv64.

Upstream changes:

Version 1.70.0 (2023-06-01)
==========================

Language
--------
- [Relax ordering rules for `asm!` operands]
  (rust-lang/rust#105798)
- [Properly allow macro expanded `format_args` invocations to uses captures]
  (rust-lang/rust#106505)
- [Lint ambiguous glob re-exports]
  (rust-lang/rust#107880)
- [Perform const and unsafe checking for expressions in `let _ =
  expr` position.]
  (rust-lang/rust#102256)

Compiler
--------
- [Extend -Cdebuginfo with new options and named aliases]
  (rust-lang/rust#109808)
  This provides a smaller version of debuginfo for cases that only
  need line number information (`-Cdebuginfo=line-tables-only`),
  which may eventually become the default for `-Cdebuginfo=1`.
- [Make `unused_allocation` lint against `Box::new` too]
  (rust-lang/rust#104363)
- [Detect uninhabited types early in const eval]
  (rust-lang/rust#109435)
- [Switch to LLD as default linker for {arm,thumb}v4t-none-eabi]
  (rust-lang/rust#109721)
- [Add tier 3 target `loongarch64-unknown-linux-gnu`]
  (rust-lang/rust#96971)
- [Add tier 3 target for `i586-pc-nto-qnx700`(QNX Neutrino RTOS, version 7.0)]
  (rust-lang/rust#109173),
- [Insert alignment checks for pointer dereferences as debug assertions]
  (rust-lang/rust#98112)
  This catches undefined behavior at runtime, and may cause existing
  code to fail.

Refer to Rust's [platform support page][platform-support-doc]
for more information on Rust's tiered platform support.

Libraries
---------
- [Document NonZeroXxx layout guarantees]
  (rust-lang/rust#94786)
- [Windows: make `Command` prefer non-verbatim paths]
  (rust-lang/rust#96391)
- [Implement Default for some alloc/core iterators]
  (rust-lang/rust#99929)
- [Fix handling of trailing bare CR in str::lines]
  (rust-lang/rust#100311)
- [allow negative numeric literals in `concat!`]
  (rust-lang/rust#106844)
- [Add documentation about the memory layout of `Cell`]
  (rust-lang/rust#106921)
- [Use `partial_cmp` to implement tuple `lt`/`le`/`ge`/`gt`]
  (rust-lang/rust#108157)
- [Stabilize `atomic_as_ptr`]
  (rust-lang/rust#108419)
- [Stabilize `nonnull_slice_from_raw_parts`]
  (rust-lang/rust#97506)
- [Partial stabilization of `once_cell`]
  (rust-lang/rust#105587)
- [Stabilize `nonzero_min_max`]
  (rust-lang/rust#106633)
- [Flatten/inline format_args!() and (string and int) literal
  arguments into format_args!()]
  (rust-lang/rust#106824)
- [Stabilize movbe target feature]
  (rust-lang/rust#107711)
- [don't splice from files into pipes in io::copy]
  (rust-lang/rust#108283)
- [Add a builtin unstable `FnPtr` trait that is implemented for
  all function pointers]
  (rust-lang/rust#108080)
  This extends `Debug`, `Pointer`, `Hash`, `PartialEq`, `Eq`,
  `PartialOrd`, and `Ord` implementations for function pointers
  with all ABIs.


Stabilized APIs
---------------

- [`NonZero*::MIN/MAX`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroI8.html#associatedconstant.MIN)
- [`BinaryHeap::retain`]
  (https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.retain)
- [`Default for std::collections::binary_heap::IntoIter`]
  (https://doc.rust-lang.org/stable/std/collections/binary_heap/struct.IntoIter.html)
- [`Default for std::collections::btree_map::{IntoIter, Iter, IterMut}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoIter.html)
- [`Default for std::collections::btree_map::{IntoKeys, Keys}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoKeys.html)
- [`Default for std::collections::btree_map::{IntoValues, Values}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.IntoKeys.html)
- [`Default for std::collections::btree_map::Range`]
  (https://doc.rust-lang.org/stable/std/collections/btree_map/struct.Range.html)
- [`Default for std::collections::btree_set::{IntoIter, Iter}`]
  (https://doc.rust-lang.org/stable/std/collections/btree_set/struct.IntoIter.html)
- [`Default for std::collections::btree_set::Range`]
  (https://doc.rust-lang.org/stable/std/collections/btree_set/struct.Range.html)
- [`Default for std::collections::linked_list::{IntoIter, Iter, IterMut}`]
  (https://doc.rust-lang.org/stable/alloc/collections/linked_list/struct.IntoIter.html)
- [`Default for std::vec::IntoIter`]
  (https://doc.rust-lang.org/stable/alloc/vec/struct.IntoIter.html#impl-Default-for-IntoIter%3CT,+A%3E)
- [`Default for std::iter::Chain`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Chain.html)
- [`Default for std::iter::Cloned`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Cloned.html)
- [`Default for std::iter::Copied`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Copied.html)
- [`Default for std::iter::Enumerate`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Enumerate.html)
- [`Default for std::iter::Flatten`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Flatten.html)
- [`Default for std::iter::Fuse`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Fuse.html)
- [`Default for std::iter::Rev`]
  (https://doc.rust-lang.org/stable/std/iter/struct.Rev.html)
- [`Default for std::slice::Iter`]
  (https://doc.rust-lang.org/stable/std/slice/struct.Iter.html)
- [`Default for std::slice::IterMut`]
  (https://doc.rust-lang.org/stable/std/slice/struct.IterMut.html)
- [`Rc::into_inner`]
  (https://doc.rust-lang.org/stable/alloc/rc/struct.Rc.html#method.into_inner)
- [`Arc::into_inner`]
  (https://doc.rust-lang.org/stable/alloc/sync/struct.Arc.html#method.into_inner)
- [`std::cell::OnceCell`]
  (https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html)
- [`Option::is_some_and`]
  (https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.is_some_and)
- [`NonNull::slice_from_raw_parts`]
  (https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html#method.slice_from_raw_parts)
- [`Result::is_ok_and`]
  (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.is_ok_and)
- [`Result::is_err_and`]
  (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.is_err_and)
- [`std::sync::atomic::Atomic*::as_ptr`]
  (https://doc.rust-lang.org/stable/std/sync/atomic/struct.AtomicU8.html#method.as_ptr)
- [`std::io::IsTerminal`]
  (https://doc.rust-lang.org/stable/std/io/trait.IsTerminal.html)
- [`std::os::linux::net::SocketAddrExt`]
  (https://doc.rust-lang.org/stable/std/os/linux/net/trait.SocketAddrExt.html)
- [`std::os::unix::net::UnixDatagram::bind_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.bind_addr)
- [`std::os::unix::net::UnixDatagram::connect_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.connect_addr)
- [`std::os::unix::net::UnixDatagram::send_to_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixDatagram.html#method.send_to_addr)
- [`std::os::unix::net::UnixListener::bind_addr`]
  (https://doc.rust-lang.org/stable/std/os/unix/net/struct.UnixListener.html#method.bind_addr)
- [`std::path::Path::as_mut_os_str`]
  (https://doc.rust-lang.org/stable/std/path/struct.Path.html#method.as_mut_os_str)
- [`std::sync::OnceLock`]
  (https://doc.rust-lang.org/stable/std/sync/struct.OnceLock.html)

Cargo
-----

- [Add `CARGO_PKG_README`]
  (rust-lang/cargo#11645)
- [Make `sparse` the default protocol for crates.io]
  (rust-lang/cargo#11791)
- [Accurately show status when downgrading dependencies]
  (rust-lang/cargo#11839)
- [Use registry.default for login/logout]
  (rust-lang/cargo#11949)
- [Stabilize `cargo logout`]
  (rust-lang/cargo#11950)

Misc
----

- [Stabilize rustdoc `--test-run-directory`]
  (rust-lang/rust#103682)

Compatibility Notes
-------------------

- [Prevent stable `libtest` from supporting `-Zunstable-options`]
  (rust-lang/rust#109044)
- [Perform const and unsafe checking for expressions in `let _ =
  expr` position.]
  (rust-lang/rust#102256)
- [WebAssembly targets enable `sign-ext` and `mutable-globals`
  features in codegen]
  (rust-lang/rust#109807)
  This may cause incompatibility with older execution environments.
- [Insert alignment checks for pointer dereferences as debug assertions]
  (rust-lang/rust#98112)
  This catches undefined behavior at runtime, and may cause existing
  code to fail.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they represent
significant improvements to the performance or internals of rustc and related
tools.

- [Upgrade to LLVM 16]
  (rust-lang/rust#109474)
- [Use SipHash-1-3 instead of SipHash-2-4 for StableHasher]
  (rust-lang/rust#107925)
har7an added a commit to har7an/zellij that referenced this pull request Dec 29, 2023
which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html
har7an added a commit to zellij-org/zellij that referenced this pull request Jan 8, 2024
* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR #3039.
har7an added a commit to har7an/zellij that referenced this pull request Jan 8, 2024
which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html
imsnif pushed a commit to aidanhs/zellij that referenced this pull request Jan 18, 2024
* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.
har7an added a commit to har7an/zellij that referenced this pull request Jan 20, 2024
which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html
imsnif added a commit to zellij-org/zellij that referenced this pull request Jan 22, 2024
* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See #2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR #3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* fix: add some robustness and future proofing

* fix e2e tests

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>
imsnif pushed a commit to aidanhs/zellij that referenced this pull request Jan 22, 2024
* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.
imsnif added a commit to aidanhs/zellij that referenced this pull request Jan 22, 2024
…3032)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See zellij-org#2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (zellij-org#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (zellij-org#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* fix: add some robustness and future proofing

* fix e2e tests

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>
imsnif added a commit to zellij-org/zellij that referenced this pull request Jan 22, 2024
… reducing size of TerminalCharacter (#3043)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See #2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* refactor: Unify viewport transfer: transfer + cursor update together

* perf: Reduce size of TerminalCharacter from 72 to 60 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
964092 kB to 874020 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~15s to ~12.5s

* fix(build): Don't unconditionally rebuild zellij-utils

* refactor: Remove Copy impl on TerminalCharacter

* perf: Rc styles to reduce TerminalCharacter from 60 to 24 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
845156 kB to 478396 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~12.5s to ~7s

* perf: Remove RcCharacterStyles::Default, allow enum niche optimisation

This reduces TerminalCharacter from 24 to 16 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
478396 kB to 398108 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~7s to ~4.75s

* docs: link anchor omission from reset_all is deliberate

reset_all is only used from ansi params, and ansi params don't control
link anchor

* fix: Remove no-op on variable that gets immediately dropped

* refactor: Simplify replace_character_at logic

The original condition checked absolute_x_index was in bounds, then
used the index to manipulate it. This is equivalent to getting a
ref to the character at that position and manipulating directly

* chore: Run xtask format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR #3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* xtask: Disable pusing during publish (#3040)

* xtask: Add `--no-push` flag to `publish`

which can be used when simulating releases to work without a writable
git fork of the zellij code.

* xtask: Fix borrow issues

* xtask/pipe: Require lockfile in publish

to avoid errors from invalid dependency versions.

* CHANGELOG: Add PR #3040.

* fix(terminal): some real/saved cursor bugs during resize (#3032)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See #2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR #3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* fix: add some robustness and future proofing

* fix e2e tests

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>

* fix integer overflow again (oops)

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>
xuanyuan300 pushed a commit to trysthout/zellij that referenced this pull request Jan 23, 2024
* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.
xuanyuan300 pushed a commit to trysthout/zellij that referenced this pull request Jan 23, 2024
…3032)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See zellij-org#2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (zellij-org#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (zellij-org#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* fix: add some robustness and future proofing

* fix e2e tests

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>
xuanyuan300 pushed a commit to trysthout/zellij that referenced this pull request Jan 23, 2024
… reducing size of TerminalCharacter (zellij-org#3043)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See zellij-org#2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* refactor: Unify viewport transfer: transfer + cursor update together

* perf: Reduce size of TerminalCharacter from 72 to 60 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
964092 kB to 874020 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~15s to ~12.5s

* fix(build): Don't unconditionally rebuild zellij-utils

* refactor: Remove Copy impl on TerminalCharacter

* perf: Rc styles to reduce TerminalCharacter from 60 to 24 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
845156 kB to 478396 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~12.5s to ~7s

* perf: Remove RcCharacterStyles::Default, allow enum niche optimisation

This reduces TerminalCharacter from 24 to 16 bytes

With a 10MB single line catted into a fresh terminal, VmRSS goes from
478396 kB to 398108 kB (as reported by /proc/<pid>/status) before/after
this patch

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~7s to ~4.75s

* docs: link anchor omission from reset_all is deliberate

reset_all is only used from ansi params, and ansi params don't control
link anchor

* fix: Remove no-op on variable that gets immediately dropped

* refactor: Simplify replace_character_at logic

The original condition checked absolute_x_index was in bounds, then
used the index to manipulate it. This is equivalent to getting a
ref to the character at that position and manipulating directly

* chore: Run xtask format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (zellij-org#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (zellij-org#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* xtask: Disable pusing during publish (zellij-org#3040)

* xtask: Add `--no-push` flag to `publish`

which can be used when simulating releases to work without a writable
git fork of the zellij code.

* xtask: Fix borrow issues

* xtask/pipe: Require lockfile in publish

to avoid errors from invalid dependency versions.

* CHANGELOG: Add PR zellij-org#3040.

* fix(terminal): some real/saved cursor bugs during resize (zellij-org#3032)

* refactor: Simplify transfer_rows_from_viewport_to_lines_above

next_lines is always consolidated to a single Row, which immediately
gets removed - we can remove some dead code as a result

* perf: Batch remove rows from the viewport for performance

Given a 1MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~9s to ~3s

* perf: Optimize Row::drain_until by splitting chars in one step

Given a 10MB line catted into the terminal, a toggle-fullscreen +
toggle-fullscreen + close-pane + `run true` goes from ~23s to ~20s

* refactor: Simplify `if let` into a `.map`

* refactor: There are only new saved coordinates when there were old ones

* refactor: Unify viewport transfer: use common variable names

* fix: Use same saved cursor logic in height resize as width

See zellij-org#2182 for original introduction that only added it in one branch,
this fixes an issue where the saved cursor was incorrectly reset when
the real cursor was

* fix: Correct saved+real cursor calculations when reflowing long lines

* fix: Don't create canonical lines if cursor ends on EOL after resize

Previously if a 20 character line were split into two 10 character
lines, the cursor would be placed on the line after the two lines.
New characters would then be treated as a new canonical line. This
commit fixes this by biasing cursors to the end of the previous line.

* fix: for cursor index calculation in lines that are already wrapped

* chore: test for real/saved cursor position being handled separately

* chore: Apply cargo format

* chore(repo): update issue templates

* Bump rust version to 1.75.0 (zellij-org#3039)

* rust-toolchain: Bump toolchain version to 1.69.0

which, compared to the previous 1.67.0, has the following impacts on
`zellij`:

- [Turn off debuginfo for build deps][2]: Increases build time (on my
  machine) from ~230 s in 1.67.0 to ~250 s now, *which is unexpected*

This version also changes [handling of the `default-features` flag][3]
when specifying dependencies in `Cargo.toml`. If a dependent crate
requires `default-features = true` on a crate that is required as
`default-features = false` further up the dependency tree, the `true`
setting "wins". We only specify `default-features = false` for three
crates total:

- `names`: This is used only by us
- `surf`: This is used only by us
- `vte`: This is also required by `strip-ansi-escapes`, but that has
  `default-features = false` as well

How this affects our transitive dependencies is unknown at this point.

[2]: rust-lang/cargo#11252
[3]: rust-lang/cargo#11409

* rust-toolchain: Bump toolchain version to 1.70.0

which, compared to the previous 1.69.0, as the following impacts on
`zellij`:

1. [Enable sparse registry checkout for crates.io by default][1]

This drastically increases the time to first build on a fresh rust
installation/a rust installation with a clean cargo registry cache.
Previously it took about 75s to populate the deps/cache (with `cargo
fetch --locked` and ~100 MBit/s network), whereas now the same process
takes ~10 s.

2. [The `OnceCell` type is now part of std][2]

In theory, this would allow us to cut a dependency from `zellij-utils`,
but the `once_cell` crate is pulled in by another 16 deps, so there's no
point in attempting it right now.

Build times and binary sizes are unaffected by this change compared to
the previous 1.69.0 toolchain.

[1]: rust-lang/cargo#11791
[2]: https://doc.rust-lang.org/stable/std/cell/struct.OnceCell.html

* rust-toolchain: Bump toolchain version to 1.75.0

which, compared to the previous 1.70.0, has the following impacts on
`zellij`:

1. [cross-crate inlining][8]

This should increase application performance, as functions can now be
inlined across crates.

2. [`async fn` in traits][9]

This would allow us to drop the `async_trait` dependency, but it is
currently still required by 3 other dependencies.

Build time in debug mode (on my own PC) is cut down from 256s to 189s
(for a clean build). Build time in release mode is cut down from 473s to
391s (for a clean build). Binary sizes only change minimally (825 MB ->
807 MB in debug, 29 MB -> 30 MB in release).

[8]: rust-lang/rust#116505
[9]: rust-lang/rust#115822

* chore: Apply rustfmt.

* CHANGELOG: Add PR zellij-org#3039.

* feat(plugins): introduce 'pipes', allowing users to pipe data to and control plugins from the command line (zellij-org#3066)

* prototype - working with message from the cli

* prototype - pipe from the CLI to plugins

* prototype - pipe from the CLI to plugins and back again

* prototype - working with better cli interface

* prototype - working after removing unused stuff

* prototype - working with launching plugin if it is not launched, also fixed event ordering

* refactor: change message to cli-message

* prototype - allow plugins to send messages to each other

* fix: allow cli messages to send plugin parameters (and implement backpressure)

* fix: use input_pipe_id to identify cli pipes instead of their message name

* fix: come cleanups and add skip_cache parameter

* fix: pipe/client-server communication robustness

* fix: leaking messages between plugins while loading

* feat: allow plugins to specify how a new plugin instance is launched when sending messages

* fix: add permissions

* refactor: adjust cli api

* fix: improve cli plugin loading error messages

* docs: cli pipe

* fix: take plugin configuration into account when messaging between plugins

* refactor: pipe message protobuf interface

* refactor: update(event) -> pipe

* refactor - rename CliMessage to CliPipe

* fix: add is_private to pipes and change some naming

* refactor - cli client

* refactor: various cleanups

* style(fmt): rustfmt

* fix(pipes): backpressure across multiple plugins

* style: some cleanups

* style(fmt): rustfmt

* style: fix merge conflict mistake

* style(wording): clarify pipe permission

* docs(changelog): introduce pipes

* fix: add some robustness and future proofing

* fix e2e tests

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>

* fix integer overflow again (oops)

---------

Co-authored-by: Aram Drevekenin <aram@poor.dev>
Co-authored-by: har7an <99636919+har7an@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sparse-registry Area: http sparse registries relnotes Release-note worthy S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sparse registry: transition plan for index.crates.io as default
5 participants