Skip to content

Refactor ov to create a persistent container #4490

Closed

Description

Problem

ov currently creates a new container each time it runs (and automatically removes the previous container). That mostly works, but we could get a lot more out of ov if we treated it more like a persistent virtual machine than an immutable one-off environment. Crucially, it also means that if you run ov <some long running command> in one terminal, and ov bash in another, you're looking at two distinct and separate containers. Theoretically most state should be maintained in bound directories but that might not be the case, for example, with tmp files. This can lead to some confusing behaviour, and makes the development environment more tedious to debug inside of.

Additionally, it makes it impossible to, for example, defer installing certain system dependencies only when needed. That would be useful in the case of Playwright, which requires a lot of host dependencies to run browsers, that shouldn't be installed in the base container.

Description

To accomplish this, we need the container to stay running, and then use exec to run commands inside the container. This could actually allow us to significantly simplify how we handle certain situations, and would speed up repeated script execution in almost all cases.

To do this, implement the following:

  1. Change the container's entrypoint to an infinite while loop that calls sleep 1 in each iteration
  2. Move the old entrypoint script to docker/dev_env/setup_env.sh and add an entry to ov for setup-env to run that script
  3. Update docker/dev_env/run.sh to do the following:
  4. Check if the container exists (even if it is stopped), If it does, and is stopped, start it. Then (after it is started), use docker exec to run the command in the container. You'll still need to set up some of the same args as run, so extract those shared args into a separate shared_args array
  5. If the container does not exist yet, use docker run to create and start it with the current run_args variables. Make sure to pass -d as well, so that the command detaches, as the entrypoint is now the infinite loop + sleep script. Then, defer to the same docker exec method used for the other branch.

Alternatives

We could promote the idea of running ov bash to start a "session", and then execute all commands inside that bash prompt. This could have several benefits, including the ability to set up just's auto-complete, easier alias definitions, etc. That might still be the way to go, but even then, I think it would be easier to reason about a persistent environment rather than an ephemeral one.

Additional context

I've marked this as "medium" because it would be great to have this to simplify the playwright setup. Right now, playwright runs inside another container, and has a bunch of semi-hacky dependency directory mapping to make pnpm install inside the container fast. But #4257 shows problems with this approach, because some of the new dependencies brought in by nuxt 3 are not entirely cross platform (they have native binaries). Plus, it adds yet another heavy container to be downloaded (all of ubuntu), and you need to download an entire new huge container each time we update Playwright. Instead, we could run Playwright directly in ov. That already solves the problem we wanted to solve by running Playwright inside docker (namely, always take snapshots with the same browser version & target OS), without introducing the additional layer of redirection. But, because Playwright requires a heap of system dependencies to run browsers, and because it would be nice not to add those dependencies to the base Dockerfile (as they are heavy and not used outside of this specific case), it would be profoundly beneficial to be able to modify the real environment inside the container and persist those changes. Running dnf install in ov currently won't persist the updates in each container run. We could map more volumes to try to do that, but then I think we're fighting Docker, and a persistent container would be a lot easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Labels

🟨 priority: mediumNot blocking but should be addressed soon🤖 aspect: dxConcerns developers' experience with the codebase🧰 goal: internal improvementImprovement that benefits maintainers, not users🧱 stack: infraRelated to the Terraform config and other infrastructure

Type

No type

Projects

  • Status

    ✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions