Skip to content
This repository has been archived by the owner on Oct 18, 2023. It is now read-only.

Abstract sqld networking #650

Merged
merged 2 commits into from
Sep 12, 2023
Merged

Abstract sqld networking #650

merged 2 commits into from
Sep 12, 2023

Conversation

MarinPostma
Copy link
Collaborator

@MarinPostma MarinPostma commented Sep 4, 2023

This PR aims to abstract networking for sqld so that we can simulate it and start writing tests using turmoil for network simulation.

I took the opportunity to clean up the configuration process and remove useless stuff that accumulated with time.

The --backend flag was removed because it is unused.

The net module contains network abstractions used throughout the crate.

@MarinPostma MarinPostma force-pushed the refactor-generic-networking branch 13 times, most recently from b76c2f1 to cec7108 Compare September 7, 2023 15:35
@MarinPostma MarinPostma marked this pull request as ready for review September 7, 2023 15:35
@MarinPostma MarinPostma force-pushed the refactor-generic-networking branch 6 times, most recently from 73a6d75 to 1a3cd5d Compare September 8, 2023 14:44
@MarinPostma MarinPostma changed the title refactor for generic networking Abstract sqld networking Sep 8, 2023
@MarinPostma MarinPostma force-pushed the refactor-generic-networking branch from 1a3cd5d to d132025 Compare September 8, 2023 15:09
Copy link
Collaborator

@LucioFranco LucioFranco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I think we should merged then do the follow ups in another PR to avoid conflicts.

sqld/src/config.rs Outdated Show resolved Hide resolved
sqld/src/hrana/ws/mod.rs Outdated Show resolved Hide resolved
continue;
};
let socket: Box<dyn Conn> = Box::new(conn);
let _: Result<_, _> = accept_tx.send(Accept { socket, peer_addr }).await;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we close the acceptor if the other end has disappeared?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which acceptor are you referring to?

Comment on lines 211 to 224
pub auth: Arc<Auth>,
pub http_acceptor: Option<A>,
pub hrana_ws_acceptor: Option<A>,
pub namespaces: NamespaceStore<M>,
pub idle_shutdown_kicker: Option<IdleShutdownKicker>,
pub stats: Stats,
pub proxy_service: P,
pub replication_service: S,
pub disable_default_namespace: bool,
pub disable_namespaces: bool,
pub max_response_size: u64,
pub enable_console: bool,
pub self_url: Option<String>,
pub join_set: &'a mut JoinSet<anyhow::Result<()>>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to avoid all these pub? I think this just makes refactoring code later on really hard since these items leak pretty easily.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you suggest?

sqld/src/http/mod.rs Outdated Show resolved Hide resolved
pub struct Server<C = HttpConnector, A = AddrIncoming> {
pub path: Arc<Path>,
pub db_config: DbConfig,
pub user_api_config: UserApiConfig<A>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should call it HttpServer (matches the mod) and RpcServer (like you have already). I think that reads the best.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idk, it's not only an HTTP server, if anything, it's a SqldServer, but since we're changing the name, I avoid including sqld into anything. Any other ideas?

sqld/src/lib.rs Outdated Show resolved Hide resolved
sqld/src/namespace/mod.rs Show resolved Hide resolved
sqld/src/test/bottomless.rs Outdated Show resolved Hide resolved
@MarinPostma MarinPostma force-pushed the refactor-generic-networking branch from 86fc033 to 3166f30 Compare September 12, 2023 10:35
@MarinPostma
Copy link
Collaborator Author

@LucioFranco, merging this to move forward, I can address your points in followups if you want.

@MarinPostma MarinPostma added this pull request to the merge queue Sep 12, 2023
Merged via the queue into main with commit 93de94d Sep 12, 2023
@MarinPostma MarinPostma deleted the refactor-generic-networking branch September 12, 2023 11:25
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants