Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for more packages to DeepSize #18

Merged
merged 1 commit into from
Nov 19, 2021
Merged

feat: Add support for more packages to DeepSize #18

merged 1 commit into from
Nov 19, 2021

Conversation

pmnoxx
Copy link
Contributor

@pmnoxx pmnoxx commented Nov 11, 2021

I would like to add support for a few types from tokio, actix.

  • use tokio::net::TcpStream
  • use actix::Addr

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 11, 2021

@matklad Please, let me know whenever it's a good idea to split this change.
I can split it into 4 parts

  • more array sizes
  • std
  • tokio
  • actix

@matklad
Copy link

matklad commented Nov 12, 2021

For array sizes, #16 is already split off.

I guess the rest looks fine, but thats for @Aeledfyr to decide :)

I would probably suggest to refactor the code a bit to make sure that per-crate impls live in crate-specific file (ie, alloc.rs, std.rs, tokio.rs, actix.rs), but that is orthogonal to adding the impls at all.

@Aeledfyr
Copy link
Owner

This looks good. One issue though is with adding feature dependencies:

tokio = { version = "^1.1", optional = true, features = ["full"] }

By adding a dependency on tokio with the full feature set, I believe that we enable that feature set in all crates that depend on deepsize, even if they don't enable the tokio feature in deepsize.

Is there any way to properly deal with this? For example, it might be possible to have a tokio-net feature that depends on tokio with the net feature enabled, but that only works if the feature flags of optional dependencies are ignored when the dependency isn't explicitly enabled.

@Aeledfyr
Copy link
Owner

Splitting the std_time and std_net parts out from this (into another PR?) would be a good idea, since they are easy to merge without dealing with dependency resolution.

Also, most of those implementations could use the known_deep_size macro to reduce verbosity, like the chrono impl does:

known_deep_size!(0;
NaiveDate, NaiveTime, NaiveDateTime, IsoWeek,
Duration, Month, Weekday,
FixedOffset, Local, Utc,
{T: TimeZone} DateTime<T>, {T: TimeZone} Date<T>,
);

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 13, 2021

@Aeledfyr Ok, I made a split #19

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 14, 2021

@Aeledfyr Can you take a look again?

@Aeledfyr
Copy link
Owner

There are still some issues with dependency resolution. This PR fails to compile with tokio being the only feature (cargo check --features tokio), since it doesn't enable the net feature. However, if you compile it with all features (cargo check --all-features) it will work, because actix enables the net feature in tokio.

Also, the actix dependency should be specified with default-features=false to avoid enabling its macro feature, for dependencies attempting to reduce compile time.

I think the cleanest way to solve the tokio feature issue would be to make tokio an optional feature with default-features=false, and to make a feature tokio_net (or something similar) that enables tokio, tokio/net (to enable the feature in tokio), and enables the DeepSize implementations.

As a basic example of how I think it could work:

[dependencies]
tokio = { version = "^1.1", optional = true, default-features = false }
# default-features is technically unnecessary here as tokio
# has no default features, but adding it is good practice for
# a library
# ...

[features]
tokio_net = ["tokio", "tokio/net"]
# ...
#[cfg(feature = "tokio_net")]
mod tokio_net_impl {
    // ...
}

I haven't verified that this works, but it seems like it should. I also haven't verified that this won't lead to accidentally enabling features in dependant's crates, but from what I can tell this requires the feature to be explicitly enabled for the tokio/net feature to be enabled.

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 14, 2021

@Aeledfyr Yes, it works. Thanks.

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 16, 2021

@Aeledfyr I updated the PR, can you take a look?

@pmnoxx pmnoxx changed the title Add support for more packages to DeepSize FEATURE Add support for more packages to DeepSize Nov 16, 2021
@pmnoxx pmnoxx changed the title FEATURE Add support for more packages to DeepSize feat: Add support for more packages to DeepSize Nov 17, 2021
@Aeledfyr
Copy link
Owner

Sorry about the delay; I believe the actix dependency should also have default-features = false, to avoid enabling the macros feature in actix for dependencies.

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Nov 19, 2021

Sorry about the delay; I believe the actix dependency should also have default-features = false, to avoid enabling the macros feature in actix for dependencies.

@miguelfrde Thanks, I updated the PR.


tokio = { version = "^1.1", optional = true, default-features = false }
actix = { version = "^0.11.0", optional = true, default-features = false }

@Aeledfyr
Copy link
Owner

I can't verify that the feature resolution works entirely properly, but from cargo tree -e features and cargo expand, it looks like this works, and looks good to merge.

@Aeledfyr Aeledfyr merged commit 5deebe6 into Aeledfyr:master Nov 19, 2021
@pmnoxx pmnoxx deleted the add-support-for-more-packages branch November 19, 2021 02:07
near-bulldozer bot pushed a commit to near/nearcore that referenced this pull request Nov 23, 2021
We want to track of sizes of messages sent from `PeerActor` to `PeerManagerActor`.
We will stop reading from given `PeerActor` once we reach the limit.

To achieve this a new type `ActixMessageWrapper` was introduced. It wraps around messages sent to peer, and contains metadata needed for stat counting.

This is a draft:
- [x] Add a wrapper for sending messages from`PeerActor` to `PeerManagerActor`
- [x] Modify `PeerActor` to use 
- [x] Add a wrapper for reply's to the above messages (wating for `DeepSizeOf`)
- [ ] Wait for `DeepSizeOf` (another PR)
- [ ] Add a wrapper for messages to Peer - for example for routing (another PR)
- [ ] Add a wrapper for messages to EdgeVerifierActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to RoutingTableActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to ClientActor - for example for routing (another PR)
- [ ] Extensions to `performance_stats` to track per thread limits (another PR)
- [ ] Extension to track bandwidth (another PR)

This PR is based on, not yet merged, #4927

Resolves #5155

Blocked by Aeledfyr/deepsize#18
pmnoxx added a commit to near/nearcore that referenced this pull request Nov 23, 2021
We want to track of sizes of messages sent from `PeerActor` to `PeerManagerActor`.
We will stop reading from given `PeerActor` once we reach the limit.

To achieve this a new type `ActixMessageWrapper` was introduced. It wraps around messages sent to peer, and contains metadata needed for stat counting.

This is a draft:
- [x] Add a wrapper for sending messages from`PeerActor` to `PeerManagerActor`
- [x] Modify `PeerActor` to use
- [x] Add a wrapper for reply's to the above messages (wating for `DeepSizeOf`)
- [ ] Wait for `DeepSizeOf` (another PR)
- [ ] Add a wrapper for messages to Peer - for example for routing (another PR)
- [ ] Add a wrapper for messages to EdgeVerifierActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to RoutingTableActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to ClientActor - for example for routing (another PR)
- [ ] Extensions to `performance_stats` to track per thread limits (another PR)
- [ ] Extension to track bandwidth (another PR)

This PR is based on, not yet merged, #4927

Resolves #5155

Blocked by Aeledfyr/deepsize#18
pmnoxx added a commit to near/nearcore that referenced this pull request Nov 23, 2021
We want to track of sizes of messages sent from `PeerActor` to `PeerManagerActor`.
We will stop reading from given `PeerActor` once we reach the limit.

To achieve this a new type `ActixMessageWrapper` was introduced. It wraps around messages sent to peer, and contains metadata needed for stat counting.

This is a draft:
- [x] Add a wrapper for sending messages from`PeerActor` to `PeerManagerActor`
- [x] Modify `PeerActor` to use
- [x] Add a wrapper for reply's to the above messages (wating for `DeepSizeOf`)
- [ ] Wait for `DeepSizeOf` (another PR)
- [ ] Add a wrapper for messages to Peer - for example for routing (another PR)
- [ ] Add a wrapper for messages to EdgeVerifierActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to RoutingTableActor - for example for routing (another PR)
- [ ] Add a wrapper for messages to ClientActor - for example for routing (another PR)
- [ ] Extensions to `performance_stats` to track per thread limits (another PR)
- [ ] Extension to track bandwidth (another PR)

This PR is based on, not yet merged, #4927

Resolves #5155

Blocked by Aeledfyr/deepsize#18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants