Skip to content

[FEA]: Convert Sherlock nodes to modules #1425

Open

Description

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

The first iteration of Sherlock nodes did not utilize MRC in order to support using coroutines in Morpheus before they are fully supported in MRC. For future compatibility, all Sherlock nodes should be converted to use MRC modules to take advantage of coroutine nodes when they are supported in MRC.

Describe your ideal solution

Sherlock nodes create a subgraph within the LLMEngine class. This subgraph can be converted to MRC nodes and connected via edges where each Sherlock nodes is a MRC node. This will allow for fully async pipelining of the LLMEngine (i.e. allowing simultaneous execution of two sibling nodes in the LLMEngine which is not possible now).

Completion Criteria

  • All 4 LLM examples run exactly as they did before without any code changes to the files in the examples directory (changes to nodes in morpheus/llm are ok)
  • Performance of the examples before the change and after the change match (within reason). See #sherlock per
  • All existing tests pass
  • The LLMEngine can be run both inside of a pipeline (where nodes will be converted to MRC modules) as well as outside of a pipeline (exactly as they are now)
    • This is to avoid requiring an entire pipeline for debugging and testing of the LLMEngine

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I have searched the open feature requests and have found no duplicates for this feature request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

feature requestNew feature or requestsherlockIssues/PRs related to Sherlock workflows and components

Type

No type

Projects

  • Status

    In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions