rpk: introduce --sandbox flag in rpk redpanda start

[r-vasquez]

Cover letter

Introduces --sandbox flag for rpk redpanda start command, it will bundle the following flags:

• --check=false
• --node-id 0
• --smp 1
• --reserve-memory 0M
• --overprovisioned
• --unsafe-bypass-fsync=true

Fixes #5208

#5997 contains the latest release notes.

Release notes

• none

[emaxerrno]

@jcsp and @dotnwat anything else we can set here to make redpanda start faster, any seastar flags here to tune out, maybe disabling filesystem consistency checking?

I am pretty sure we are missing some seastar flags here.

[dotnwat]

What about --unsafe-bypass-fsync=true?

[emaxerrno]

What about --unsafe-bypass-fsync=true?

Exactly!!!! This one too.

[emaxerrno]

@r-vasquez we May need to change the flag name because kubernetes has a developerMode variable too that means something different.

So we should think how this mode would interact with other modes and it may need a different name.

Cc @vladoschreiner

[jcsp]

Where do we warn the user that their data is unsafe in this mode? I think it needs to be very clear and explicit.

[jcsp]

This mode should set topic_partitions_per_shard to about 1000, to put a nice safe limit on any users that try to use their mini-me redpandas beyond small scales.

(that property is new in 22.2.x & defaults to 7000, but that default value is only for backward compat reasons to avoid breaking any existing systems: when we introduce a new mode it is an opportunity to set a conservative limit).

[jcsp]

Related thought: we should look at producing a separate container image ("redpanda-sandbox" or similar) that defaults to this mode, so that we can then make the regular redpanda image strictly configured for production (and then in the production image we can do things like refusing to run on bad setups).

[mmedenjak]

@ajfabbri there's this dev mode and I see there's the discussion about full disk handling thresholds:

• Add test for full disk threshold config, leave default as-is #5216 (comment)
• rpk: add storage space command w/ auto-config option #5248

I think we should account full disk threshold values for dev mode. Any thoughts?

[emaxerrno]

This mode should set topic_partitions_per_shard to about 1000, to put a nice safe limit on any users that try to use their mini-me redpandas beyond small scales.

(that property is new in 22.2.x & defaults to 7000, but that default value is only for backward compat reasons to avoid breaking any existing systems: when we introduce a new mode it is an opportunity to set a conservative limit).

This makes sense. @r-vasquez we should rename to —sandbox and use this limit too.

[r-vasquez]

@jcsp is topic_partitions_per_shard a node property that can be picked up from redpanda.yaml (redpanda.topic_partitions_per_shard) ? If it's a cluster property, is there a way to pass a value in redpanda start?

w.r.t topic_partitions_per_shard we can print a warning in rpk :

• This is a setup for development purposes only, in dev mode your clusters may run unrealistically fast and data can be corrupted any time your computer shuts down uncleanly. (Maybe add a recommendation for production setup?)

@emaxerrno will rename the flag to --sandbox then.

[jcsp]

@jcsp is topic_partitions_per_shard a node property that can be picked up from redpanda.yaml (redpanda.topic_partitions_per_shard) ? If it's a cluster property, is there a way to pass a value in redpanda start?

You can either use the admin API, or on first startup write a .bootstrap.yaml to the same directory that redpanda.yaml is in. The bootstrap file is a single yaml dict of property name to value (no toplevel redpanda section)

[r-vasquez]

Update:

• Rename flag from --dev to --sandbox.
• Add --unsafe-bypass-fsync=true
• Add a warning in rpk when you are running in sandbox mode.
• Rebased.

Created #5661 to follow up the work to add cluster properties into this bundle (also suggested a couple of properties there)