after replacing default config file, removing the MC causes the config file to disappear

[miabbott]

Using OpenShift 4.1.0-rc.5:

1. Create a MC that replaces /etc/containers/policy.json. In the example below, this is the default contents of /etc/containers/policy.json run through jq -c and requoted:

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 50-policy-json
spec:
  config:
    ignition:
      version: 2.2.0
    storage:
      files:
      - contents:
          source: data:,%7B%22default%22%3A%5B%7B%22type%22%3A%22insecureAcceptAnything%22%7D%5D%2C%22transports%22%3A%7B%22docker-daemon%22%3A%7B%22%22%3A%5B%7B%22type%22%3A%22insecureAcceptAnything%22%7D%5D%7D%7D%7D
        filesystem: root
        mode: 0644
        path: /etc/containers/policy.json

2. Wait for nodes to be updated
3. Remove the MC
4. Wait for nodes to be updated
5. Check the nodes and find that /etc/containers/policy.json is missing

FWIW, /etc/containers/policy.json is owned by the containers-common package

[cgwalters]

To solve this generally, on an ostree-based system we need to restore the defaults from /usr/etc when the new MC stops modifying a file created by the old one. But we don't have /usr/etc on classic yum-based systems 😢 ...I guess we could instead copy the originals to .mcdorig when we update a file?

[jlebon]

Hmm, is this a liability for future updates? Right now, we need to make sure that any new MC we generate does not stop modifying pre-existing files until this is fixed, it seems? (I.e. re-encoding the default file, rather than removing the file entry entirely). Which... seems like a very easy bit to trip up on 😬

[runcom]

I'm adding a test (and a fix for this following Colin's suggestion)

[runcom]

PR here #797