Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge mysten_infra into sui repo #5955

Merged
merged 291 commits into from
Nov 9, 2022
Merged

Merge mysten_infra into sui repo #5955

merged 291 commits into from
Nov 9, 2022

Conversation

sadhansood
Copy link
Contributor

@sadhansood sadhansood commented Nov 9, 2022

Merge mysten-infra into sui. Followed steps used to merge narwhal closely here

huitseeker and others added 30 commits November 8, 2022 16:14
…nfra#79)

Improve the default and testing subscribers to include more information
like the line number as well as outputting span creation and close
events.
In many cases, e.g. cli tools like the wallet, a tool may want to print
out some useful information to stdout. Today we currently log to stdout
which makes it very difficult to distinguish between output we want the
user to actually use and log messages. In order to address this this
patch changes our logging infrastructure to log to stderr by default
instead of stdout.
…log messages (MystenLabs/mysten-infra#85)

* Remove unhelpful log messages

* Don't use ansi colors unless stderr is a tty
…-infra#83)

This commit is introducing an interface to act as a callback for performed requests against the server. Then those callbacks can be used for metrics reporting purposes.
As this is liable to overload memory, the present PR introduces an optional Predicate argument (a `Fn(&(Key, Value)) -> bool`) which allows filtering a subset of the table's items instead of the whole table.
…infra#94)

Also fix our misuse of layering various optional tracing layers.
Improve our panic handler to do the following:

Also call the stdlib's panic handler so backtraces aren't swallowed
flush stdout and stderr
enable configuring if the panic handler should crash the process
…nLabs/mysten-infra#101)

* add a ultility function to convert MultiAddr to SocketAddr

* fixup! add a ultility function to convert MultiAddr to SocketAddr

* add negative test

* fixup! add negative test
@sadhansood sadhansood changed the title Sadhan/merge_mysten_infra Merge merge_mysten_infra into sui repo Nov 9, 2022
@sadhansood sadhansood changed the title Merge merge_mysten_infra into sui repo Merge mysten_infra into sui repo Nov 9, 2022
@sadhansood sadhansood force-pushed the sadhan/merge_mysten_infra branch from 8d20b47 to a327fa8 Compare November 9, 2022 02:51
@bmwill
Copy link
Contributor

bmwill commented Nov 9, 2022

Please move all these crates into the top level crates directory and not inside a special mysten-infra one.

Copy link
Member

@mwtian mwtian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Really looking forward to this.

Cargo.toml Outdated
@@ -40,7 +40,29 @@ members = [
"crates/test-utils",
"crates/workspace-hack",
"crates/x",
"narwhal/config", "narwhal/consensus", "narwhal/crypto", "narwhal/dag", "narwhal/examples", "narwhal/executor", "narwhal/network", "narwhal/node", "narwhal/primary", "narwhal/storage", "narwhal/test-utils", "narwhal/types", "narwhal/worker",
"mysten-infra/crates/component",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use just one directory level for crates, e.g. mysten/ or infra/? Using crates/ as suggested would also work, but it will make fixing the Dockerfiles harder.

@sadhansood
Copy link
Contributor Author

Moved all of them under the top level crates dir, cc @bmwill @mwtian

@mwtian
Copy link
Member

mwtian commented Nov 9, 2022

Moved all of them under the top level crates dir, cc @bmwill @mwtian

Can you try the build docker image workflows in sui ops on the new branch? We may need to fix a few things.

@sadhansood
Copy link
Contributor Author

sadhansood commented Nov 9, 2022

Moved all of them under the top level crates dir, cc @bmwill @mwtian

Can you try the build docker image workflows in sui ops on the new branch? We may need to fix a few things.

seems to be working fine to me (this is on the new branch)

@mwtian
Copy link
Member

mwtian commented Nov 9, 2022

Moved all of them under the top level crates dir, cc @bmwill @mwtian

Can you try the build docker image workflows in sui ops on the new branch? We may need to fix a few things.

seems to be working fine to me (this is on the new branch)

Great, how about the Narwhal image build?

@@ -16,7 +16,6 @@ jsonrpsee = { version = "0.15.1", features = ["full"] }
tracing = { version = "0.1.36", features = ["log"] }
clap = { version = "3.1.14", features = ["derive"] }
reqwest = { version = "0.11.11", features = ["blocking", "json"] }
telemetry-subscribers.workspace = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the workspace dependency setup? It will make it easier to put narwhal back into a separate repo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are zero plans for putting narwhal back into a separate repo. This is not going to happen any time within the next 1-2 years

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any harm in using workspace dependency as before. There are people who want to experiment with narwhal in their projects too.

Copy link
Member

@mwtian mwtian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted offline. Good to go!

@sadhansood sadhansood merged commit 01d6d31 into main Nov 9, 2022
@sadhansood sadhansood deleted the sadhan/merge_mysten_infra branch November 9, 2022 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.