Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mongo_normalized fails to read a 0 length event stream #729

Open
tacaswell opened this issue Aug 30, 2022 · 0 comments
Open

mongo_normalized fails to read a 0 length event stream #729

tacaswell opened this issue Aug 30, 2022 · 0 comments

Comments

@tacaswell
Copy link
Contributor

Expected Behavior

Current Behavior

MongoAdapter._build_event_stream(self, run_start_uid, stream_name, is_complete)
   1466 # We need each of the sub-dicts to have a consistent length. If
   1467 # Events are still being added, we need to choose a consistent
   1468 # cutoff. If not, we need to know the length anyway. Note that this
   1469 # is not the same thing as the number of Event documents in the
   1470 # stream because seq_num may be repeated, nonunique.
   1471 cursor = self._event_collection.aggregate(
   1472     [
   1473         {"$match": {"descriptor": {"$in": event_descriptor_uids}}},
   (...)
   1480     ]
   1481 )
-> 1482 (result,) = cursor
   1483 cutoff_seq_num = (
   1484     1 + result["highest_seq_num"]
   1485 )  # `1 +` because we use a half-open interval
   1486 object_names = event_descriptors[0]["object_keys"]

ValueError: not enough values to unpack (expected 1, got 0)

Possible Solution

We need to deal with the possibility that the cursor that comes out of

cursor = self._event_collection.aggregate(
[
{"$match": {"descriptor": {"$in": event_descriptor_uids}}},
{
"$group": {
"_id": "descriptor",
"highest_seq_num": {"$max": "$seq_num"},
},
},
]
)
(result,) = cursor
may have 0 or 1 values in it (rather than always exactly 1 value).

There are a couple other such unpackings that may need to be fixed as well.

Steps to Reproduce (for bugs)

  1. try create the header object for a run that contains an empty stream (have a descriptor, but no events)

Context

had to change from a comprehension to a for loop so I could put a try/except in, but in general this is fatal to any process that wants to look at the meta data of runs that may have failed or been cancelled mid-collection between the descriptor going out and the first event going out.

The ability to pre-declare streams being proposed in bluesky will make it much easier to end up in this situation.

Your Environment

in the 2022-3.0-py39-tiled environment

$ conda list | grep databroker
databroker                2.0.0b8            pyhd8ed1ab_1    conda-forge
$ conda list | grep tiled
# packages in environment at /nsls2/conda/envs/2022-3.0-py39-tiled:
tiled                     0.1.0a70             hd8ed1ab_0    conda-forge
tiled-array               0.1.0a70             hd8ed1ab_0    conda-forge
tiled-base                0.1.0a70         py39hf3d152e_0    conda-forge
tiled-client              0.1.0a70             hd8ed1ab_0    conda-forge
tiled-compression         0.1.0a70             hd8ed1ab_0    conda-forge
tiled-dataframe           0.1.0a70             hd8ed1ab_0    conda-forge
tiled-formats             0.1.0a70             hd8ed1ab_0    conda-forge
tiled-minimal-base        0.1.0a70             hd8ed1ab_0    conda-forge
tiled-server              0.1.0a70         py39hf3d152e_0    conda-forge
tiled-sparse              0.1.0a70             hd8ed1ab_0    conda-forge
tiled-xarray              0.1.0a70             hd8ed1ab_0    conda-forge
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant