Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockservice & exchange & bitswap: add non variadic NotifyNewBlock #242

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

Jorropo
Copy link
Contributor

@Jorropo Jorropo commented Mar 30, 2023

Variadicts in go are just syntactic sugar around passing a slice, that means all go memory reachability rules apply, this force the compiler to heap allocate the variadic slice for virtual call, because the implementation is allowed to leak the slice (and go's interprocedural optimisations do not cover virtuals).

Passing a block without variadic will pass the itab either on the stack or decomposed through registers. Skipping having to allocate a slice.

@Jorropo Jorropo self-assigned this Mar 30, 2023
@Jorropo Jorropo requested a review from a team as a code owner March 30, 2023 16:48
@Jorropo Jorropo force-pushed the notify-new-block branch 3 times, most recently from 5344f5d to 4ff4bd2 Compare March 30, 2023 17:02
@codecov
Copy link

codecov bot commented Mar 30, 2023

Codecov Report

Attention: Patch coverage is 90.90909% with 2 lines in your changes missing coverage. Please review.

Project coverage is 60.32%. Comparing base (19a402b) to head (730b9bd).

Files with missing lines Patch % Lines
blockservice/blockservice.go 50.00% 1 Missing and 1 partial ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #242      +/-   ##
==========================================
- Coverage   60.36%   60.32%   -0.04%     
==========================================
  Files         243      243              
  Lines       30953    30968      +15     
==========================================
- Hits        18684    18682       -2     
- Misses      10612    10626      +14     
- Partials     1657     1660       +3     
Files with missing lines Coverage Δ
bitswap/bitswap.go 69.23% <100.00%> (+1.78%) ⬆️
bitswap/client/client.go 90.22% <100.00%> (+2.10%) ⬆️
bitswap/server/internal/decision/engine.go 91.44% <ø> (ø)
bitswap/server/server.go 64.42% <100.00%> (+0.50%) ⬆️
exchange/offline/offline.go 88.88% <100.00%> (+1.01%) ⬆️
blockservice/blockservice.go 78.00% <50.00%> (-0.23%) ⬇️

... and 9 files with indirect coverage changes

Comment on lines +16 to +17
// NotifyNewBlock tells the exchange that a new block is available and can be served.
NotifyNewBlock(ctx context.Context, blocks blocks.Block) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea how widespread the dependency on this interface (and therefore the result of the breakage) is and the benefits of implementing this? For example, does this show up at all in profiles as a meaningful thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think everyone relies either on the offline exchange (in order to wrap a blockstore) or bitswap. I don't belive many implementations exists, even in outside codebases.

Even if someone else implement this, just forwarding the call to NotifyNewBlocks with a slice of 1, this is legal and easy to write.
This just aims to get over golang's lacking interprocedural virtual optimisations, it does not change any feature.

Most of our other APIs have both single and multiple versions for this reason (Put vs PutMany, GetBlock vs GetBlocks, ...).

@BigLep
Copy link
Contributor

BigLep commented Mar 30, 2023

@Jorropo : I don't see an issue linked to this PR. In the absence of this, can you share more on what's motivating this PR?

@Jorropo
Copy link
Contributor Author

Jorropo commented Mar 30, 2023

@BigLep the PR and Commit bodies:

Variadicts in go are just syntactic sugar around passing a slice, that means all go memory reachability rules apply, this force the compiler to heap allocate the variadic slice for virtual call, because the implementation is allowed to leak the slice (and go's interprocedural optimisations do not cover virtuals).
Passing a block without variadic will pass the itab either on the stack or decomposed through registers. Skipping having to allocate a slice.

I can add the explanation about keeping it in-line with our other exchange APIs.


This seems like a good usecase to try the idea @guseggert is cooking (some automated code migrator), that would add the NotifyNewBlock method (which redirects to NotifyNewBlocks) automatically if you run it.

Copy link
Member

@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong opinions on this. It seems a relatively small change that can bring memory benefits - although arguably low. If others agree, I'm fine to have it merged.

FYSA: I rebased the PR and added the updated exchange for the gateway examples.

@BigLep
Copy link
Contributor

BigLep commented Apr 6, 2023

2023-04-06 conversation: will defer for now. Want to have clarity on how we handle breaking changes in Boxo.

@Jorropo Jorropo removed their assignment Mar 4, 2024
@gammazero gammazero added the need/maintainers-input Needs input from the current maintainer(s) label Sep 20, 2024
@gammazero
Copy link
Contributor

This is ready to merge. Let's decide if we want this or not.

Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm? As noted in https://github.com/ipfs/boxo/pull/242/files#r1153577309 this follows existing convention from other places (Put vs PutMany, GetBlock vs GetBlocks etc), and simplifies code slightly. If someone has custom implementation, should be trivial to migrate.

@aschmahmann any concerns?

Jorropo and others added 5 commits September 27, 2024 20:49
Variadicts in go are just syntactic sugar around passing a slice, that
means all go memory reachability rules apply, this force the compiler to
heap allocate the variadic slice for virtual call, because the
implementation is allowed to leak the slice (and go's interprocedural
optimisations do not cover virtuals).

Passing a block without variadic will pass the itab either on the stack
or decomposed through registers. Skipping having to allocate a slice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/maintainers-input Needs input from the current maintainer(s)
Projects
No open projects
Status: 🔎 In Review
Development

Successfully merging this pull request may close these issues.

7 participants