Add e2e tests for template#25
Conversation
jrichter1
requested review from
gabemontero,
maysunfaisal and
thepetk
as code owners
gabemontero
left a comment
gabemontero
left a comment
There was a problem hiding this comment.
cool stuff @jrichter1
most of my comments are not just for you, but @thepetk @maysunfaisal as well, and center around coupling this with our rolling demo setup to help automation ... env var names and clarification around cluster login / credentials were bits that could apply to what you have here
| - oc/kubectl cli tool | ||
| - Running cluster with: | ||
| - RHDH/backstage instance | ||
| - including a configured static token (or disabled API authentication if you like to live dangerously) |
There was a problem hiding this comment.
@thepetk - any sense on how hard would it be if we added some sort of optional env var flag that defines a static token for the backstage instance ?
if we had that, seems like in theory we could use https://github.com/redhat-ai-dev/ai-rolling-demo-gitops/blob/main/docs/SETUP_GUIDE.md and run these e2e's
pretty sure the rolling demo setup covers all the req's @jrichter1 has here
could be the basis for PR induced execution of the e2e's as well
a possible parking lot convo as well
There was a problem hiding this comment.
Yeah +1 I feel too a quick discussion will help
There was a problem hiding this comment.
adding to the Feb 5 parking lot agenda (there isn't a Tuesday call next week)
| export RHDH_TOKEN= | ||
|
|
||
| # Namespace the application will be deployed to | ||
| export APP_NAMESPACE= |
There was a problem hiding this comment.
the overlap is pretty good already, but if we align the env vars here with https://github.com/redhat-ai-dev/ai-rolling-demo-gitops/blob/main/docs/SETUP_GUIDE.md that could facilitate use of the rolling demo gitops for setting up the OCP cluster
| # Base URL of RHDH | ||
| export RHDH_URL= | ||
| # Static token configured for RHDH. Tests will assume unauthenticated API access if left empty | ||
| export RHDH_TOKEN= |
There was a problem hiding this comment.
this would be the static token add on to rolling gitops IIUC
e2e/.env-template
Outdated
| @@ -0,0 +1,37 @@ | |||
| # Base URL of RHDH | |||
| export RHDH_URL= | |||
There was a problem hiding this comment.
for example, in rolling gitops, this is RHDH_BASE_URL
| } | ||
|
|
||
| async registerTemplate(url: string) { | ||
| const response = await this.axiosInstance.post(`${this.RHDHUrl}/api/catalog/locations`, { |
There was a problem hiding this comment.
populating our template by default via rolling demo gitops could help not having to register the template @thepetk @maysunfaisal
that said, I certainly see no problems with having this code here @jrichter1 in the event a template in question is not available
maysunfaisal
left a comment
maysunfaisal
left a comment
There was a problem hiding this comment.
PASS tests/lls-template.test.ts (842.516 s)
Llama stack agentic sample template
✓ Load the template if not imported (167 ms)
✓ Create necessary secrets (832 ms)
✓ Template runs (20486 ms)
✓ Creates source and gitops repos (568 ms)
✓ Creates and merges a pull request in source repo (8660 ms)
✓ Gitops applications are created (226 ms)
✓ Build pipeline finishes green (810619 ms)
✓ Deployments are created (161 ms)
Test Suites: 1 passed, 1 total
Tests: 8 passed, 8 total
Snapshots: 0 total
Time: 843.573 s
Ran all test suites.
generally lgtm, i will wait for you to address other comments
|
@jrichter1 I think if you can get env var name consistency with what you have and wlhat we have over in https://github.com/redhat-ai-dev/ai-rolling-demo-gitops/blob/main/docs/SETUP_GUIDE.md per #25 (comment) I'll be good to merge this the name consistency would facilitate integration with our rolling gitops, and doesn't hurt anything if some reason we decide not to pair your e2e's with that in some for of automated test. thanks again for these tests |
Adjusted what I could find. Let me know if I missed something. |
gabemontero
left a comment
gabemontero
left a comment
There was a problem hiding this comment.
I'm not finding any missed updates @jrichter1 .... of course we can always adjust later if we both missed something
thanks for the update
I'll merge momentarily
4 participants
What does this PR do?:
Adds automated tests for the Llama Stack Agentic AI Workflow template, which
Which issue(s) this PR fixes:
https://issues.redhat.com/browse/RHIDP-11391
PR acceptance criteria:
Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened and linked to this PR, if they are not in the PR scope due to various constraints.
Tested and Verified
Documentation (READMEs, Product Docs, Blogs, Education Modules, etc.)
How to test changes / Special notes to the reviewer: