Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Draft
Changes from 1 commit
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
60b13bb
wip: download uv
gau-nernst Feb 14, 2025
3ddce8c
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 14, 2025
f9817c8
fix: has_value -> has_error
gau-nernst Feb 14, 2025
2dbc296
move uv stuff to python_engine. use uv to start process
gau-nernst Feb 18, 2025
eec24bd
redirect stdout/stderr
gau-nernst Feb 18, 2025
26fdbd3
simplify code
gau-nernst Feb 18, 2025
3ba7994
rename python engine interface
gau-nernst Feb 19, 2025
5e7125f
use PythonEngineI
gau-nernst Feb 19, 2025
c5da0ee
more checks to match all EngineV variants
gau-nernst Feb 19, 2025
3c097fb
improve Python load model
gau-nernst Feb 19, 2025
84db8b0
consolidate process-related functions
gau-nernst Feb 19, 2025
8ee815c
update PythonModelConfig. add UnloadModel
gau-nernst Feb 19, 2025
29f5344
implement PythonEngine::GetModels
gau-nernst Feb 19, 2025
75ce355
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 19, 2025
7949dcc
implement getModelStatus. add some notes
gau-nernst Feb 19, 2025
e2f0323
add router for python
gau-nernst Feb 19, 2025
607d2cb
call PythonEngine destructor
gau-nernst Feb 19, 2025
f58b773
remove unused method
gau-nernst Feb 19, 2025
bf23c9f
remove unnecessary headers
gau-nernst Feb 19, 2025
d7818d5
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 19, 2025
8ebee7c
remove unused stuff
gau-nernst Feb 20, 2025
8f36adc
download uv directly from github release
gau-nernst Feb 20, 2025
5ebfbb7
check for entrypoint
gau-nernst Feb 20, 2025
5d310d1
only record model size for llama.cpp
gau-nernst Feb 20, 2025
c4c622c
don't include headers
gau-nernst Feb 20, 2025
fc0369c
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 21, 2025
6b59878
don't use std::optional to support < c++17
gau-nernst Feb 21, 2025
250a2ac
fix stringstream usage
gau-nernst Feb 21, 2025
bb38a56
define pid_t for windows
gau-nernst Feb 21, 2025
723c5db
explicit call .string() on filesystem::path to support windows
gau-nernst Feb 21, 2025
26ec20a
include extra_args in entrypoint
gau-nernst Feb 21, 2025
376deeb
add python engine install test
gau-nernst Feb 21, 2025
a9ed820
add start time
gau-nernst Feb 21, 2025
db82134
add back python engine to default supported engine so that cortex eng…
gau-nernst Feb 21, 2025
64f5451
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 25, 2025
79464a2
format
gau-nernst Feb 25, 2025
1768826
run uv sync after model download
gau-nernst Feb 26, 2025
7627eac
download CUDA for python engine
gau-nernst Feb 26, 2025
06503c0
add .exe for windows
gau-nernst Feb 26, 2025
176f878
destroy file action in posix
gau-nernst Feb 26, 2025
48f50f5
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 27, 2025
f7bddc2
revert name change to avoid conflict
gau-nernst Feb 27, 2025
728e7eb
check for NULL before destroy file action
gau-nernst Feb 27, 2025
560b9fe
fix windows
gau-nernst Feb 27, 2025
f481c2f
fix windows
gau-nernst Feb 27, 2025
48e2015
fix windows subprocess
gau-nernst Feb 27, 2025
f02fc93
update test
gau-nernst Feb 27, 2025
c104d77
Merge branch 'dev' into thien/python_engine
gau-nernst Feb 27, 2025
9918672
more robust checks and cleanup
gau-nernst Feb 28, 2025
99a0035
support engines uninstall
gau-nernst Mar 3, 2025
b96fd69
follow reverse proxy example
gau-nernst Mar 3, 2025
62da415
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 3, 2025
e2e2ccc
update uv to 0.6.3
gau-nernst Mar 3, 2025
57c30d3
support engines list
gau-nernst Mar 3, 2025
408d66b
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 4, 2025
5dbb5c5
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 12, 2025
4a4cd1d
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 13, 2025
49df6af
remove checks against supportedEngines
gau-nernst Mar 17, 2025
f1dcdde
remove supportedEngines check for more commands
gau-nernst Mar 17, 2025
1a7576b
Merge branch 'dev' into thien/cli_engines_install
gau-nernst Mar 17, 2025
64124b3
Merge branch 'thien/cli_engines_install' into thien/python_engine
gau-nernst Mar 17, 2025
f030615
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 17, 2025
13652ca
init vllm engine
gau-nernst Mar 17, 2025
4d13014
fix issues with progress streaming
gau-nernst Mar 17, 2025
5c451d8
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 18, 2025
591d461
support download HF model
gau-nernst Mar 18, 2025
c3d41bf
use / for HF model
gau-nernst Mar 18, 2025
dc42ddd
fix thread-unsafe
gau-nernst Mar 18, 2025
13d9e3f
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 18, 2025
70151e2
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 19, 2025
73fe3e5
remove methods
gau-nernst Mar 19, 2025
7bf287d
remove old remnants
gau-nernst Mar 19, 2025
2a2b607
support models list. add --relocatable for venv
gau-nernst Mar 19, 2025
fffc686
preparation works for start model
gau-nernst Mar 19, 2025
cea8020
add sync download util. add vLLM version config. some boilerplate cod…
gau-nernst Mar 19, 2025
86d4c01
list engines
gau-nernst Mar 19, 2025
ec8b36d
load and unload model
gau-nernst Mar 19, 2025
9226110
retrieve cortex port from yaml file
gau-nernst Mar 19, 2025
eeccd3a
add env vars support. log stdout and stderr
gau-nernst Mar 20, 2025
6fe7ae8
add GetModelStatus and GetModels
gau-nernst Mar 20, 2025
074a04a
fix typo
gau-nernst Mar 20, 2025
cd55d64
Merge branch 'dev' into thien/python_engine
gau-nernst Mar 21, 2025
368a4f3
add non-stream chat completions
gau-nernst Mar 21, 2025
c0e0fca
Merge pull request #2186 from menloresearch/s/chore/sync-dev
vansangpfiev Mar 27, 2025
e141891
Merge branch 'main' into thien/python_engine
gau-nernst Apr 1, 2025
807b201
add uninstall cmd
gau-nernst Apr 1, 2025
d38eca8
support streaming
gau-nernst Apr 1, 2025
7e002cd
fix cortex run
gau-nernst Apr 1, 2025
1ebbbdb
wait for vLLM server to be up
gau-nernst Apr 2, 2025
b5d8315
use health check for some stuff
gau-nernst Apr 2, 2025
5feda51
add some notes. support embeddings. support some extra vLLM args
gau-nernst Apr 2, 2025
5eea345
remove old tests. some chores
gau-nernst Apr 2, 2025
2bde26a
remove unused function
gau-nernst Apr 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
explicit call .string() on filesystem::path to support windows
  • Loading branch information
gau-nernst committed Feb 21, 2025
commit 723c5db5646a23d2fc11d7f40e7a09ce41554ab7
15 changes: 9 additions & 6 deletions engine/extensions/python-engine/python_engine.cc
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ cpp::result<void, std::string> DownloadUv(std::shared_ptr<DownloadService>& down
// try to unzip the downloaded file
const std::string download_path = finishedTask.items[0].localPath.string();

archive_utils::ExtractArchive(download_path, py_bin_path, true);
archive_utils::ExtractArchive(download_path, py_bin_path.string(), true);
set_permission_utils::SetExecutePermissionsRecursive(py_bin_path);
std::filesystem::remove(download_path);
};
Expand All @@ -70,7 +70,9 @@ cpp::result<void, std::string> DownloadUv(std::shared_ptr<DownloadService>& down

std::string GetUvPath() {
// NOTE: do I need to add .exe for windows?
return file_manager_utils::GetCortexDataPath() / "python_engine" / "bin" / "uv";
const auto path = file_manager_utils::GetCortexDataPath()
/ "python_engine" / "bin" / "uv";
return path.string();
}
bool IsUvInstalled() {
return std::filesystem::exists(GetUvPath());
Expand Down Expand Up @@ -145,20 +147,21 @@ void PythonEngine::LoadModel(
pid_t pid;
try {
config::PythonModelConfig py_cfg;
py_cfg.ReadFromYaml(model_dir / "model.yml");
py_cfg.ReadFromYaml((model_dir / "model.yml").string());

if (py_cfg.entrypoint == "") {
throw std::runtime_error("Missing entrypoint in model.yml");
}

// NOTE: model_dir / entrypoint assumes a Python script
// TODO: figure out if we can support arbitrary CLI (but still launch by uv)
std::vector<std::string> command{GetUvPath(), "run", model_dir / py_cfg.entrypoint};
const std::string entrypoint = (model_dir / py_cfg.entrypoint).string();
std::vector<std::string> command{GetUvPath(), "run", entrypoint};
for (const auto& item : py_cfg.extra_args)
command.push_back(item);

const std::string stdout_path = model_dir / "stdout.txt";
const std::string stderr_path = model_dir / "stderr.txt";
const std::string stdout_path = (model_dir / "stdout.txt").string();
const std::string stderr_path = (model_dir / "stderr.txt").string();

// create empty stdout.txt and stderr.txt for redirection
if (!std::filesystem::exists(stdout_path)) std::ofstream(stdout_path).flush();
Expand Down
Loading