Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FractalComputeConfig inherit from BaseSettings instead of BaseModel #729

Open
hadim opened this issue Jun 22, 2023 · 2 comments
Open

FractalComputeConfig inherit from BaseSettings instead of BaseModel #729

hadim opened this issue Jun 22, 2023 · 2 comments

Comments

@hadim
Copy link

hadim commented Jun 22, 2023

The main reason would be to allow parsing env variables instead of providing a config YAML file (more convenient in a container-based env for example).

@hadim
Copy link
Author

hadim commented Jun 22, 2023

For now I am using the following patch: hadim@494a78a

It works with the below config for example:

export QCF_COMPUTE_BASE_FOLDER=/tmp/qcf_compute
export QCF_COMPUTE_CLUSTER=manager_demo_1
export QCF_COMPUTE_LOGLEVEL=INFO
export QCF_COMPUTE_UPDATE_FREQUENCY=30

export QCF_COMPUTE_SERVER_FRACTAL_URI=http://127.0.0.1:7777
export QCF_COMPUTE_SERVER_USERNAME=compute_user
export QCF_COMPUTE_SERVER_PASSWORD=compute_password
export QCF_COMPUTE_SERVER_VERIFY=false

export QCF_COMPUTE_ENVIRONMENTS_USE_MANAGER_ENVIRONMENT=true
export QCF_COMPUTE_ENVIRONMENTS_CONDA='[]'
export QCF_COMPUTE_ENVIRONMENTS_APPTAINER='[]'

export QCF_COMPUTE_EXECUTORS='{"local": {"type": "local", "queue_tags": ["demo"], "cores_per_worker": 16, "memory_per_worker": 16, "max_workers": 1}}'

The only thing that is not ideal is QCF_COMPUTE_EXECUTORS but I haven't spend too much time trying to make it work with multiple env variables instead of a single JSON string.

@hadim
Copy link
Author

hadim commented Jun 22, 2023

I had to add another patch to make it works (and the tests to pass): hadim@ee83118

Also somehow I also need to set QCF_COMPUTE_SERVER or it fails cuz of pydantic error.

export QCF_COMPUTE_BASE_FOLDER=/tmp/qcf_compute
export QCF_COMPUTE_CLUSTER=manager_demo_1
export QCF_COMPUTE_LOGLEVEL=INFO
export QCF_COMPUTE_UPDATE_FREQUENCY=30

export QCF_COMPUTE_SERVER='{}'  # somehow this is needed....
export QCF_COMPUTE_SERVER_FRACTAL_URI=http://127.0.0.1:7777
export QCF_COMPUTE_SERVER_USERNAME=compute_user
export QCF_COMPUTE_SERVER_PASSWORD=compute_password
export QCF_COMPUTE_SERVER_VERIFY=false

export QCF_COMPUTE_ENVIRONMENTS_USE_MANAGER_ENVIRONMENT=true
export QCF_COMPUTE_ENVIRONMENTS_CONDA='[]'
export QCF_COMPUTE_ENVIRONMENTS_APPTAINER='[]'

export QCF_COMPUTE_EXECUTORS='{"local": {"type": "local", "queue_tags": ["demo"], "cores_per_worker": 16, "memory_per_worker": 16, "max_workers": 1}}'

It works well for me.

@bennybp bennybp changed the title [next branch] should FractalComputeConfig inherit from BaseSettings instead of BaseModel FractalComputeConfig inherit from BaseSettings instead of BaseModel Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant