Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem this feature solves

Some of the post-processing stages (add-class & add-scores) need to insert new columns into the Dataframe. The problem is this requires acquiring the gil and introduces a potential race condition.

Describe your ideal solution

Since the add-class & add-scores stages know which columns they need to create, they should be able to advertise the names & types of the columns.

During pipeline build stage, these can be queried, and published to the source stage, so that they can be allocated in the Dataframe prior to emitting into the pipeline.

Describe any alternatives you have considered

No response

Additional context

Idea came out of a discussion at the Morpheus meeting as a way to avoid locking in the during pipeline execution.

Code of Conduct

• I agree to follow this project's Code of Conduct
• I have searched the open feature requests and have found no duplicates for this feature request