Add support for container restart policy by JaewonHur · Pull Request #1258 · apple/container

JaewonHur · 2026-02-24T01:56:25Z

This PR initiates the implementation for adding restart feature (#286), which can be extended to support auto-start later (#158, #1201). The first milestone is to implement restart policies no, on-failure, and always (which behaves similar to docker-restart-policy).

To support restart policy, following persistent and runtime state variables should be added:

In ContainerCreateOption (persistent)
- RestartPolicy: remember restart policy across the container life cycle.
In ContainerState (run-time)
- StoppedState(startError: Error?, exitStatus: ExitStatus?): determine restart logic when container exits
- manualStopped: Bool: remember if the container is stopped by user
  - True on first container creation, False on loadAtBoot
- backOff: Duration?: remember backoff timer.
  - Default to nil
  - Set 100ms on first non-manual exit (and then doubled until 10 seconds)
  - Set nil on startError or manualStopped

A new state should be added in container life cycle:

restarting: restarted containers first go to this state and waits for backOff to start.

Implementation steps are described below:

[ContainersService] Add state variables for restart policy
[ContainerResource, ContainerCommand] Add restart policy and expose command line.
[ContainersService] Add restarting state, where the restarting containers first visit, and proceeds to bootstrap (after backOff) then startProcess.
[ContainersService] Implement restart logic in handleContainerExit
- [restart == no]
  - Idle (stay at stopped).
- [restart == on-failure]
  - startError || manualStopped || exitStatus == nil
    - Idle
  - exitStatues != nil && exitStatus != 0
    - Restart
- [restart == always]
  - startError || manualStopped
    - Idle
  - Else
    - Restart
Add stability call, which marks the container to be stable 10 seconds after running---i.e., resetting the backOff.
- It might be something like stabilityMonitor (similar to ExitMonitor) that registers callback on successful startProcess.

Type of Change

Bug fix
New feature
Breaking change
Documentation update

Motivation and Context

Users can setup auto-restarting containers using this feature.

Testing

Tested locally
Added/updated tests
Added/updated docs

Containers move to .bootstrapped state after assigning SandboxClient. Following invariant holds: 1. If .stopped, state.client = nil 2. If .bootstrapped, state.client != nil. It'd better to tightly couple the state and client.

Then, add `restarting` state to ContainerStatus

ContainersService spawn an unstructured Task for restart scheduler, which receives stopped container ids through a async queue and restart them if matching conditions. While it doesn't run under lock, it doesn't introduce a new level of race, as all the actual operations changing container state (e.g., `bootstrap`) are lock protected.

This reuses ExitMonitor, which was not initially for this. It might be better to come up with generic Monitor structure?

saehejkang · 2026-02-25T05:14:07Z

@JaewonHur Busy week and I hope to get to review this here soon. I saw your comment and I am all for following the direction we take in this PR. Though it might be worthwhile to get started on the discussion on how we want to handle test cases and maybe work together on that piece?

doStop cannot guarantee the container is stopped in short term.

JaewonHur · 2026-02-25T07:28:19Z

Hi @saehejkang, sorry for moving forward on my own.

I just did minimal implementation today, but I'll definitely put our name together here.
This PR now just contains prototype implementation, and anything can be changed! Please take a look :)

Further things to do might be:

supporting --restart systemStart (or make always to start on system start?)
Add restart counter for onFailure.
Also, testing for restarting container across system can be added.

saehejkang

I went over things as best as I could for now. Lots of changes that I have not gotten a chance to dig deeper into (the scheduler). Don't worry about taking over and wrapping this up!

saehejkang · 2026-02-26T02:20:21Z

Sources/ContainerResource/Container/ContainerCreateOptions.swift

    }

-    public static let `default` = ContainerCreateOptions(autoRemove: false)
+    public init(from decoder: Decoder) throws {


Why was the decoder not being used before when it was just autoRemove?

I actually have no idea :)
Maybe, wanted to keep the code as simple as possible at that time?

saehejkang · 2026-02-26T02:22:45Z

Sources/ContainerResource/Container/ContainerStatus.swift

+    case stopped
+    /// The object is waiting to be restarted.
+    case restarting
+    /// The object is currently bootstrapped.


What does the bootstrapped status mean? The container or sandbox is stopped but ready to be started?

bootstrapped means (by itself) the container is bootstrapped (i.e., bootstrap function has been successfully executed on that container).

Before this update, the state was same stopped whether it is bootstrapped or not. The only difference was SandboxClient is allocated or not---I personally felt that somewhat weird, but left there.

But after adding restarting state, it becomes more weird, since container state should go restarting (waiting for backOff), then stopped (after bootstrap) then running`.

BTW, now we split the RuntimeStatus to ContainerStatus and SandboxStatus, so it only applies to containers.

While writing this comment, I found that I missed setting the container restarting in scheduler lol.

saehejkang · 2026-02-26T03:23:19Z

Sources/Services/ContainerAPIService/Server/Containers/ContainersService.swift

        return results
    }

+    public func runRestartScheduler() throws {


Never thought I would see a scheduler again after OS in college 😆.

saehejkang · 2026-02-26T03:31:58Z

supporting --restart systemStart (or make always to start on system start?)

Solid foundation work was done with the restart policy and I am sure it could follow the same pattern?

Add restart counter for onFailure.

What would be the use case for this? Could it be used to automatically stop trying to restart a container after a certain count?

Also, testing for restarting container across system can be added.

Hopefully we can use something like #1176 to be a sort of final e2e test of all the policies.

JaewonHur · 2026-02-26T17:56:29Z

Solid foundation work was done with the restart policy and I am sure it could follow the same pattern?

Thanks, if you make PR to my branch, let me merge it :)

What would be the use case for this? Could it be used to automatically stop trying to restart a container after a certain count?

Yes. Docker supports that functionality (https://docs.docker.com/engine/containers/start-containers-automatically/)

JaewonHur added 2 commits February 23, 2026 17:02

Add StoppedState and manualStopped in ContainerState

7033950

Create and expose restart policy

ae1aad8

github-actions bot added the cli label Feb 24, 2026

JaewonHur mentioned this pull request Feb 24, 2026

[containers]: add system-start flag for auto-start of containers #1201

Draft

7 tasks

JaewonHur added 7 commits February 24, 2026 10:51

Add RuntimeStatus.bootstrapped

0cecfbe

Containers move to .bootstrapped state after assigning SandboxClient. Following invariant holds: 1. If .stopped, state.client = nil 2. If .bootstrapped, state.client != nil. It'd better to tightly couple the state and client.

Split RuntimeStatus to ContainerStatus and SandboxStatus

3b675c4

Then, add `restarting` state to ContainerStatus

Add backOff and wait before restarting

5552746

Fix backOff update

743a2a4

Add stability call to reset backOff

75f35ac

This reuses ExitMonitor, which was not initially for this. It might be better to come up with generic Monitor structure?

Add TestCLIRunRestart

78cfc1a

Use doKill instead of doStop

aa251f5

doStop cannot guarantee the container is stopped in short term.

saehejkang reviewed Feb 26, 2026

View reviewed changes

Proceed containers to restarting state

0304d28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for container restart policy#1258

Add support for container restart policy#1258
JaewonHur wants to merge 11 commits intoapple:mainfrom
JaewonHur:container-restart

JaewonHur commented Feb 24, 2026 •

edited

Loading

Uh oh!

saehejkang commented Feb 25, 2026

Uh oh!

JaewonHur commented Feb 25, 2026

Uh oh!

saehejkang left a comment •

edited

Loading

Uh oh!

saehejkang Feb 26, 2026

Uh oh!

JaewonHur Feb 26, 2026

Uh oh!

saehejkang Feb 26, 2026

Uh oh!

JaewonHur Feb 26, 2026

Uh oh!

JaewonHur Feb 26, 2026

Uh oh!

saehejkang Feb 26, 2026

Uh oh!

saehejkang commented Feb 26, 2026

Uh oh!

JaewonHur commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

JaewonHur commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of Change

Motivation and Context

Testing

Uh oh!

saehejkang commented Feb 25, 2026

Uh oh!

JaewonHur commented Feb 25, 2026

Uh oh!

saehejkang left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

saehejkang Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

JaewonHur Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

saehejkang Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

JaewonHur Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

JaewonHur Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

saehejkang Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

saehejkang commented Feb 26, 2026

Uh oh!

JaewonHur commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JaewonHur commented Feb 24, 2026 •

edited

Loading

saehejkang left a comment •

edited

Loading