Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fail when mantra license error is seen #164

Merged
merged 1 commit into from
Jun 26, 2024

Conversation

epmog
Copy link
Contributor

@epmog epmog commented Jun 14, 2024

What was the problem/requirement? (What/Why)

Houdini jobs weren't failing properly if a license error occurred when launching hython:

INFO: Running command: hython "/Users/morgane/Library/Application Support/hatch/env/virtual/deadline-cloud-for-houdini/Bp-p5MaR/deadline-cloud-for-houdini/lib/python3.10/site-packages/deadline/houdini_adaptor/HoudiniClient/houdini_client.py"
STDERR: No licenses could be found to run this application.
STDERR: 	Please check for a valid license server host
STDERR: Details: http://localhost:1715: No server found running. Please start the server before retrying.

What was the solution? (How)

capture the license error and fail the job

What is the impact of this change?

Jobs properly fail!

How was this change tested?

Ran hython locally without a license connection and added a unit test

$ houdini-openjd run --init-data "{'scene_file': 'C:/asdfasdf', 'render_node': 'mantra1', 'version': '19.5'}"
INFO: Applying user-level configuration: /Users/morgane/.openjd/adaptors/runtime/configuration.json
INFO: Applying user-level configuration: /Users/morgane/.openjd/adaptors/HoudiniAdaptor/HoudiniAdaptor.json
INFO: Loading 'init_data' schema from /Users/morgane/Library/Application Support/hatch/env/virtual/deadline-cloud-for-houdini/Bp-p5MaR/deadline-cloud-for-houdini/lib/python3.10/site-packages/deadline/houdini_adaptor/HoudiniAdaptor/schemas/init_data.schema.json
INFO: Loading 'run_data' schema from /Users/morgane/Library/Application Support/hatch/env/virtual/deadline-cloud-for-houdini/Bp-p5MaR/deadline-cloud-for-houdini/lib/python3.10/site-packages/deadline/houdini_adaptor/HoudiniAdaptor/schemas/run_data.schema.json
openjd_progress: 0
openjd_status: Initializing Houdini
INFO: Setting HOUDINI_PATHMAP to: 
INFO: Running command: hython "/Users/morgane/Library/Application Support/hatch/env/virtual/deadline-cloud-for-houdini/Bp-p5MaR/deadline-cloud-for-houdini/lib/python3.10/site-packages/deadline/houdini_adaptor/HoudiniClient/houdini_client.py"
STDERR: No licenses could be found to run this application.
STDERR: 	Please check for a valid license server host
STDERR: Details: http://localhost:1715: No server found running. Please start the server before retrying.
STDERR: 
STDERR: 
openjd_fail: Error encountered while starting adaptor: Houdini encountered a license error: No licenses could be found to run this application.
ERROR: Error running the adaptor: Houdini encountered a license error: No licenses could be found to run this application.
ERROR: Entrypoint failed: Houdini encountered a license error: No licenses could be found to run this application.

Was this change documented?

N/A

Is this a breaking change?

Nope


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@epmog epmog force-pushed the handle_license_error branch from 80af643 to 05ebe2a Compare June 18, 2024 22:28
@epmog epmog marked this pull request as ready for review June 18, 2024 22:28
@epmog epmog requested a review from a team as a code owner June 18, 2024 22:28
@epmog epmog force-pushed the handle_license_error branch from 05ebe2a to 4d1febd Compare June 18, 2024 22:38
@epmog epmog added the bug Something isn't working label Jun 18, 2024
completed_regexes = [re.compile(".*Finished Rendering.*")]
progress_regexes = [re.compile(".*ALF_PROGRESS ([0-9]+)%.*")]
error_regexes = [re.compile(".*Error: .*|.*\\[Error\\].*", re.IGNORECASE)]
license_regexes = [
# generic runtime error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be under the license error regex list if it's a generic error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went back and forth on this a few times. It was previously just a license error, so I'm not changing functionality here.

Putting it with the generic errors would then make the "strict error handling" apply to it which I'm not sure we want.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

Comment on lines -279 to -285
shutil_usage = shutil.disk_usage(os.getcwd())
self._exc_info = RuntimeError(
f"{match.group(0)}\n"
"This error is typically associated with a licensing error"
" when using Houdini. Check your licensing configuration.\n"
f"Free disc space: {shutil_usage.free//1024//1024}M\n"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why did we report the free disk space when we suspected license errors before?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was due to copy/pasting some hyper-specific maya license handling. They erroneously report licensing errors on the cloud by saying they ran out of disk space. So we did a few attempts at improving it to ensure that the user could easily verify that was not the case.

Signed-off-by: Morgan Epp <60796713+epmog@users.noreply.github.com>
@epmog epmog force-pushed the handle_license_error branch from 4d1febd to 298b5ec Compare June 26, 2024 21:16
@epmog epmog merged commit 6012e2b into aws-deadline:mainline Jun 26, 2024
15 checks passed
@epmog epmog deleted the handle_license_error branch June 26, 2024 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants