Closed
Description
Version
23.01
Which installation method(s) does this occur on?
Docker
Describe the bug.
Production DFP examples fail with KeyError when changing config.ae.userid_column_name and config.ae.timestamp_column_name from defaults ("username", "timestamp").
Minimum reproducible example
Change "timestamp" to "ts" in line 187 of `dfp_azure_pipeline.py`:
config.ae.timestamp_column_name = "timestamp"Relevant log output
Traceback (most recent call last):
File "dfp_azure_pipeline.py", line 318, in <module>
run_pipeline(obj={}, auto_envvar_prefix='DFP', show_default=True, prog_name="dfp")
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "dfp_azure_pipeline.py", line 314, in run_pipeline
pipeline.run()
File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 517, in run
asyncio.run(self._do_run())
File "/opt/conda/envs/morpheus/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/opt/conda/envs/morpheus/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 495, in _do_run
await self.join()
File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 251, in join
await self._mrc_executor.join_async()
File "/my_data/gitrepos/efajardo-nv/Morpheus/examples/digital_fingerprinting/production/morpheus/dfp/stages/dfp_split_users_stage.py", line 121, in extract_users
message[self._config.ae.timestamp_column_name].min(),
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/pandas/core/frame.py", line 3458, in __getitem__
indexer = self.columns.get_loc(key)
File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
raise KeyError(key) from err
KeyError: 'ts'Code of Conduct
- I agree to follow Morpheus' Code of Conduct
- I have searched the open bugs and have found no duplicates for this bug report
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
