Open
Description
Is this a new feature, an improvement, or a change to existing functionality?
Improvement
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
The first iteration of Sherlock nodes did not utilize MRC in order to support using coroutines in Morpheus before they are fully supported in MRC. For future compatibility, all Sherlock nodes should be converted to use MRC modules to take advantage of coroutine nodes when they are supported in MRC.
Describe your ideal solution
Sherlock nodes create a subgraph within the LLMEngine
class. This subgraph can be converted to MRC nodes and connected via edges where each Sherlock nodes is a MRC node. This will allow for fully async pipelining of the LLMEngine
(i.e. allowing simultaneous execution of two sibling nodes in the LLMEngine
which is not possible now).
Completion Criteria
- All 4 LLM examples run exactly as they did before without any code changes to the files in the examples directory (changes to nodes in
morpheus/llm
are ok) - Performance of the examples before the change and after the change match (within reason). See #sherlock per
- All existing tests pass
- The
LLMEngine
can be run both inside of a pipeline (where nodes will be converted to MRC modules) as well as outside of a pipeline (exactly as they are now)- This is to avoid requiring an entire pipeline for debugging and testing of the
LLMEngine
- This is to avoid requiring an entire pipeline for debugging and testing of the
Additional context
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
- I have searched the open feature requests and have found no duplicates for this feature request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Type
Projects
Status
In Progress