Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roadmap: Implement Remote engine #1662

Closed
21 of 22 tasks
Tracked by #3786
nguyenhoangthuan99 opened this issue Nov 11, 2024 · 0 comments · Fixed by #1666
Closed
21 of 22 tasks
Tracked by #3786

roadmap: Implement Remote engine #1662

nguyenhoangthuan99 opened this issue Nov 11, 2024 · 0 comments · Fixed by #1666
Assignees
Labels
type: epic A major feature or initiative
Milestone

Comments

@nguyenhoangthuan99
Copy link
Contributor

nguyenhoangthuan99 commented Nov 11, 2024

Problem

This epic is implementation of Remote engine

Success Criteria

  • Cortex supports remote engine for using remote provider (openai, anthropic, ...)
  • Cortex supports engine managements for remote engine
  • Cortex supports model managements for remote model (gpt-4o, claude, ...)
  • Cortex supports API for Chat Completions, Get models and route request to correct remote provider

Tasklist

  • Implement jinja2 parser and chat completion requests route
  • Implement Get models request route
  • Implement Request Transform
  • Implement Response Transform
  • Implement Engine management
    • Engine table in cortex.db
    • Upsert Remote engine (name & variant would be unique as composite key)
    • Get Remote engine by Id
    • Get Remote engine by name and variant (optional)
    • Delete Remote engine
    • Update API /v1/engines to support remote engine
  • Implement model management (cortex.cpp)
    • Add remote model.yml
    • Update Remote model.yml
    • List, Get remote models
    • load, unload model
    • chat completion with non stream request
    • chat completion with stream request
    • Integrate with new engine implementation
  • Try using transform template with Claude model
  • Support CLI for remote engines
  • Docs
@nguyenhoangthuan99 nguyenhoangthuan99 added the type: epic A major feature or initiative label Nov 11, 2024
@nguyenhoangthuan99 nguyenhoangthuan99 self-assigned this Nov 11, 2024
@github-project-automation github-project-automation bot moved this to Investigating in Jan & Cortex Nov 11, 2024
@nguyenhoangthuan99 nguyenhoangthuan99 moved this from Investigating to In Progress in Jan & Cortex Nov 11, 2024
@dan-menlo dan-menlo changed the title epic: Implement Remote engine roadmap: Implement Remote engine Nov 28, 2024
@vansangpfiev vansangpfiev moved this from In Progress to In Review in Jan & Cortex Dec 1, 2024
@github-project-automation github-project-automation bot moved this from Eng Review to QA in Jan & Cortex Dec 5, 2024
@gabrielle-ong gabrielle-ong added this to the v1.0.5 milestone Dec 6, 2024
@TC117 TC117 modified the milestones: v1.0.5, v1.0.6, v1.0.7 Dec 26, 2024
@TC117 TC117 modified the milestones: v1.0.7, v1.0.9 Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: epic A major feature or initiative
Projects
Status: QA
Development

Successfully merging a pull request may close this issue.

6 participants