`kiveapi.add_dataset()` creates uploaded datasets with no retained file data

[Donaim]

kiveapi.add_dataset() can create datasets that are marked as uploaded and have an MD5 checksum, but do not retain the uploaded file. The resulting dataset has dataset_file=null, filename=null, filesize=null, has_data=false, and is_purged=true.

This appears to be a compatibility bug between kiveapi.add_dataset() and the server-side dataset serializer.

Observed Behavior

Calling kiveapi.add_dataset() with a local file creates a dataset like this:

{
  "id": 62,
  "name": "hello1",
  "dataset_file": null,
  "externalfiledirectory": null,
  "external_path": "",
  "filename": null,
  "filesize": null,
  "filesize_display": "0 bytes",
  "MD5_checksum": "1ebbd3e34237af26da5dc08a4e440464",
  "has_data": false,
  "is_redacted": false,
  "is_purged": true,
  "uploaded": true
}

The MD5 checksum indicates that the server read the uploaded file, but the file itself was not saved.

Later, any attempt to use the dataset as real input can fail with:

ValueError: Dataset has no dataset_file or external_path.

Expected Behavior

kiveapi.add_dataset() with a file handle should create a usable file-backed dataset by default.

Expected dataset properties:

{
  "dataset_file": "Datasets/...",
  "filename": "hello1.txt",
  "filesize": 123,
  "has_data": true,
  "is_purged": false,
  "uploaded": true
}

Reproduction

Start with an empty Kive database.

Run something equivalent to:

import kiveapi

session = kiveapi.KiveAPI("http://example-kive")
session.login("kive", "password")

with open("hello1.txt", "rb") as handle:
    dataset = session.add_dataset(
        name="hello1",
        description="",
        handle=handle,
        users=[],
        groups=["Everyone"],
    )

print(dataset.raw)

Then inspect the dataset:

GET /api/datasets/<id>/

Observed result:

{
  "dataset_file": null,
  "filename": null,
  "filesize": null,
  "has_data": false,
  "is_purged": true,
  "uploaded": true
}

Likely Cause

In DatasetSerializer.create():

keep_file = True
file_path = validated_data.get("external_path", "")
...
keep_file = validated_data.get("save_in_db", keep_file)

If save_in_db is present in validated_data as False even though the client did not submit it, then this overrides the intended default keep_file=True.

This may be related to the serializer field change from:

save_in_db = serializers.NullBooleanField(write_only=True, required=False)

to:

save_in_db = serializers.BooleanField(
    write_only=True,
    required=False,
    allow_null=True,
)