Forms unlinked from datasets still show up in dataset metadata

[ktuite]

One requirement for deleting a dataset is that it is unlinked from all forms, both forms that write to the dataset, and forms that consume the dataset (as an attachment). While working on this, I noticed it was hard to unlink a dataset from a form in the latter case.

In this test, there's a form that has a "goodone.csv" attachment that automatically links to the "goodone" dataset. Updating the form to use a regular csv file instead of the dataset didn't fully unlink the dataset. When querying the dataset itself, the form still appeared in the linked forms, preventing the dataset from being deleted.

it('should unlink dataset from the form reflect unlinking in dataset metadata', testService(async (service) => {
        const asAlice = await service.login('alice');

        await asAlice.post('/v1/projects/1/datasets')
          .send({ name: 'goodone' })
          .expect(200);

        await asAlice.post('/v1/projects/1/forms?publish=true')
          .send(testData.forms.withAttachments)
          .set('Content-Type', 'application/xml')
          .expect(200);

        // make draft of form
        await asAlice.post('/v1/projects/1/forms/withAttachments/draft')
          .expect(200);

        // unlink by overwriting csv
        await asAlice.post('/v1/projects/1/forms/withAttachments/draft/attachments/goodone.csv')
          .send('test,csv\n1,2')
          .set('Content-Type', 'text/csv')
          .expect(200);

        // draft attachments should not be linked to datasets
        await asAlice.get('/v1/projects/1/forms/withAttachments/draft/attachments')
          .then(({ body }) => {
            body.map(attachment => attachment.datasetExists).should.eql([false, false]);
          });

        // publish draft
        await asAlice.post('/v1/projects/1/forms/withAttachments/draft/publish?version=2')
          .expect(200);

        // attachments should still not reference any blobs
        await asAlice.get('/v1/projects/1/forms/withAttachments/attachments')
          .then(({ body }) => {
            body.map(attachment => attachment.datasetExists).should.eql([false, false]);
          });

        // check dataset
        await asAlice.get('/v1/projects/1/datasets/goodone')
          .then(({ body }) => {
            // PROBLEM: A link to the dataset is still maintained because it exists in an old form attachment
            body.linkedForms.length.should.equal(0);
          });
      }));

Some questions that have come up:

1. How should a dataset be unlinked from a form like this?

Making a draft of the form and replacing the dataset attachment with a CSV worked.
Some other things I thought would work included:

This PATCH command to supposedly unlink the dataset. The API docs say this can be done, but it seemed like after doing this, the link between form attachment and dataset still existed?

 asAlice.patch('/v1/projects/1/forms/withAttachments/draft/attachments/goodone.csv')
      .send({ dataset: false })

The DELETE command to supposedly delete the attachment.

asAlice.delete('/v1/projects/1/forms/withAttachments/draft/attachments/goodone.csv')

This test projects/:id/forms/:formId/draft/attachment/:name PATCH > should unlink dataset from the form is kind of confusing because it uses the old style of having to make a dataset through a form. It looks like the form in question is always a draft... the dataset is linked and then immediately unlinked and then when the form is published, it correctly doesn't have the dataset attached.

There's a similar thing going on with projects/:id/forms/:formId/draft/attachment/:name DELETE > should unlink dataset from the form.

Basically, my hypothesis is that these unlinking endpoints work for forms that have never been published but may not work for forms that have been published.

But maybe that's fine... maybe uploading a CSV is the only way that users should really be expected to unlink a dataset.

Another way that should be verified is changing the form definition enough so it doesn't use that attachment name. Maybe it uses a different attachment name instead, or maybe it no longer has any attachments. These cases should also be tested to make sure the original dataset can be properly unlinked.

2. Does deleting a form unlink it?

Does the dataset metadata stuff check for deleted forms?

If you're able to delete a dataset after deleting a form, what does that mean for restoring the form? If you're not able to do so, what is it like to be stuck with a dataset you can't delete because a soft-deleted form still has a reference to it?