Skip to content

Add missing endpoints and separate generated code#324

Merged
IgorChvyrov-sm merged 90 commits intomainfrom
refactoring_separate_generated_code
Sep 24, 2025
Merged

Add missing endpoints and separate generated code#324
IgorChvyrov-sm merged 90 commits intomainfrom
refactoring_separate_generated_code

Conversation

@IgorChvyrov-sm
Copy link
Contributor

@IgorChvyrov-sm IgorChvyrov-sm commented Aug 22, 2025

Changes:

  1. Generated Missing API endpoints:

    1. admin_resource_api:
      1. /admin/cache/clear/{taskDefName
      2. /admin/redisUsage
      3. /admin/sweep/requeue/{workflowId}
      4. /admin/consistency/verifyAndRepair/{workflowId}
      5. /admin/task/{tasktype}
    2. application_resource_api:
      1. POST /applications/{applicationId}/accessKeys/{keyId}/status
    3. environment_resource_api:
      1. PUT /environment/{key}
      2. DELETE /environment/{key}
      3. DELETE /environment/{name}/tags
      4. GET /environment/{key}
      5. GET /environment
      6. GET /environment/{name}/tags
      7. PUT /environment/{name}/tags
    4. event_execution_resource_api:
      1. GET /event/execution
      2. GET /event/execution/{event}
    5. event_message_resource_api:
      1. GET /event/message
      2. GET /event/message/{event}
    6. event_resource_api:
      1. DELETE /event/{name}/tags
      2. GET /event/handler/{name}
      3. PUT /event/queue/config/{queueType}/{queueName}
      4. PUT /event/{name}/tags
      5. DELETE /event/name
      6. GET /event/handler/
      7. GET /event/queue/connectivity
    7. group_resource_api:
      1. POST /groups/{groupId}/users
      2. DELETE /groups/{groupId}/users
    8. incomming_webhook_resource_api:
      1. GET /webhook/{id}
      2. POST /webhook/{id}
    9. integration_resource_api:
      1. GET /integrations/
      2. POST /integrations/eventStats/{type}
      3. POST /integrations/
    10. limits_resource_api:
      1. GET /limits
    11. metrics_resource_api:
      1. GET /metrics/task/{taskName}
    12. metrics_token_resource_api:
      1. GET /metrics/token
    13. prompt_resource_api:
      1. POST /prompts/
    14. queue_admin_resource_api:
      1. GET /queue/
      2. GET /queue/size
    15. scheduler_bulk_resource_api:
      1. PUT /scheduler/bulk/pause
      2. PUT /scheduler/bulk/resume
    16. scheduler_resource_api:
      1. DELETE /scheduler/schedules/{name}/tags
    17. secret_resource_api:
      1. GET /secrets/clearLocalCache
      2. GET /secrets/clearRedisCache
    18. user_resource_api:
      1. GET /users/{userId}/checkPermissions
    19. version_resource_api:
      1. GET /version
    20. webhook_resource_api
    21. workflow_bulk_resource_api:
      1. POST /workflow/bulk/delete
  2. Created Resource API adapters

  3. Created Models adapters

  4. Introduced proxy package to achieve backward compatibility

  5. Added integration provider functionality to OrkesIntegrationClient

  6. Added integration tests for Orkes clients

@IgorChvyrov-sm IgorChvyrov-sm marked this pull request as ready for review August 28, 2025 09:30
@IgorChvyrov-sm IgorChvyrov-sm changed the base branch from feature_async_client_pt2 to main August 28, 2025 09:31
@IgorChvyrov-sm IgorChvyrov-sm changed the title Refactoring separate generated code Add missing endpoints and separate generated code Sep 1, 2025
@nthmost-orkes nthmost-orkes self-requested a review September 18, 2025 18:04
@nthmost-orkes nthmost-orkes self-assigned this Sep 18, 2025
nthmost-orkes
nthmost-orkes previously approved these changes Sep 18, 2025
Copy link

@nthmost-orkes nthmost-orkes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for posterity (and to help myself since I'm new!), I'm going to write a few notes here.

I can see this is a PR consisting overwhelmingly of mechanical refactoring, with structural improvement to support the SDK Roadmap:

https://docs.google.com/document/d/12fdAkJfBsBhLM7jS87uGKnFyDl2gCro6dTIgcWxL-BY/edit?tab=t.0#heading=h.3g6oli2c87ry

The adapter pattern creates 3 files for every API (codegen + adapter + proxy).

Some insights generated by Claude Code:

✏️ Real Changes (<10% of files and LOC)

  1. Import path updates in ~50 example files:
    - conductor.client.ai.configuration → conductor.shared.ai.enums
    - conductor.client.http.models → conductor.client.adapters.models
  2. New documentation:
    - CLIENT_REGENERATION_GUIDE.md (315 lines)
  3. Real new functionality:
    - Integration API support (a handful of actual implementation files)
    - Some adapter methods like TaskAdapter.to_task_result()

💡 Stats:

  • ~695 files (80%) are boilerplate: Generated code + stub adapters + proxy re-exports
  • ~150 files (17%) are import updates: Examples and tests updating import paths
  • ~30 files (3%) are actual new code: Integration features and documentation

I'm can approve this PR on the basis that e2e tests exist (even if they're incomplete), so if these underlying structural changes don't break anything, it's probably fine. 👍🏼

(And very minor thing that need not hold up the PR: from future imports should probably be removed going forward.)

HOWEVER I have a concern about potential overengineering that I'd like to clear up @IgorChvyrov-sm before approving this.

@nthmost-orkes nthmost-orkes dismissed their stale review September 18, 2025 19:10

rethinking this after seeing #326 -- concerns about architecture and possible impacts to SDK-user (developer) experience

@IgorChvyrov-sm
Copy link
Contributor Author

Just for posterity (and to help myself since I'm new!), I'm going to write a few notes here.

I can see this is a PR consisting overwhelmingly of mechanical refactoring, with structural improvement to support the SDK Roadmap:

https://docs.google.com/document/d/12fdAkJfBsBhLM7jS87uGKnFyDl2gCro6dTIgcWxL-BY/edit?tab=t.0#heading=h.3g6oli2c87ry

The adapter pattern creates 3 files for every API (codegen + adapter + proxy).

Some insights generated by Claude Code:

✏️ Real Changes (<10% of files and LOC)

  1. Import path updates in ~50 example files:
    • conductor.client.ai.configuration → conductor.shared.ai.enums
    • conductor.client.http.models → conductor.client.adapters.models
  2. New documentation:
    • CLIENT_REGENERATION_GUIDE.md (315 lines)
  3. Real new functionality:
    • Integration API support (a handful of actual implementation files)
    • Some adapter methods like TaskAdapter.to_task_result()

💡 Stats:

  • ~695 files (80%) are boilerplate: Generated code + stub adapters + proxy re-exports
  • ~150 files (17%) are import updates: Examples and tests updating import paths
  • ~30 files (3%) are actual new code: Integration features and documentation

I'm can approve this PR on the basis that e2e tests exist (even if they're incomplete), so if these underlying structural changes don't break anything, it's probably fine. 👍🏼

(And very minor thing that need not hold up the PR: from future imports should probably be removed going forward.)

HOWEVER I have a concern about potential overengineering that I'd like to clear up @IgorChvyrov-sm before approving this.

We can discuss the SDK architecture (feel free to ping me if you have any questions).

Regarding future statements, we support Python from version 3.7 so I don't understand why they need to be removed. Could you elaborate on this point in more detail?

@nthmost-orkes
Copy link

Regarding future statements, we support Python from version 3.7 so I don't understand why they need to be removed. Could you elaborate on this point in more detail?

Sure, this is a philosophy of maintenance and code scannability (fast-readability) thing. Here are the 3 statements that use from future...:

  1. src/conductor/client/orkes/api/tags_api.py:13 - Contains from future import absolute_import
  2. src/conductor/client/orkes/orkes_prompt_client.py:1 - Contains from future import absolute_import, annotations
  3. src/conductor/client/orkes/orkes_integration_client.py:1 - Contains from future import absolute_import

For now we have to keep the annotations import (it looks like to me) but from future import absolute_import doesn't have any effect in Python 3.7 onwards (this is already the default behavior).

We're not supporting versions of Python before 3.7, so any compatibility code doing something that a previous version of Python need is just cruft. Sure, it won't add much weight to the package, but the implication for a human seeing a from future import is that there's some extra backward compatibility we're performing here, and that's just not the case.

Also, consider that the following statement doesn't cause any harm either:

assert 1 == 1

The only difference is that a linter would complain about this, but it wouldn't pick up on a useless from future... statement.

So, the philosophy I'm adhering to here is "don't leave in code that doesn't serve a useful purpose." (I did say it was a very minor thing!)

@nthmost-orkes
Copy link

nthmost-orkes commented Sep 22, 2025

Hey so just as we discussed in Slack @IgorChvyrov-sm, here's my breakdown of the developer experience concern I have about this PR.

State of the arch conversation reflected in this PR:

This change theoretically enables the SDK to more easily keep up with Conductor API changes.

This change makes it so that there are 3 files per API call.

For the developer, this change may have the following effects:

Import Pattern Confusion 🔀

Before: Single import path**

from conductor.client.workflow_client import WorkflowClient

After: Three possible import paths**

Option 1: Proxy layer (backward compatible)
from conductor.client.http.api.workflow_resource_api import WorkflowResourceApi

Option 2: Adapter layer
from conductor.client.adapters.api.workflow_resource_api_adapter import WorkflowResourceApiAdapter

Option 3: Codegen layer (raw)
from conductor.client.codegen.api.workflow_resource_api import WorkflowResourceApi

Impact

Developers unsure which to use, potential for inconsistent usage across codebase. Future architectural changes (if desired) may cause more issues for developers.

Debugging Struggles 🔍

Stack traces now go through 3 layers for every API call:

  File "my_app.py", line 10, in create_workflow
  File "conductor/client/http/api/workflow_resource_api.py", line 4  # Just a re-export
  File "conductor/client/adapters/api/workflow_resource_api_adapter.py", line 5  # Empty pass-through
  File "conductor/client/codegen/api/workflow_resource_api.py", line 234  # Actual implementation

Impact

May be harder to trace errors, longer stack traces, more cognitive overhead.

Maintenance Commitment Ambiguities

We're promising to keep a "proxy layer" to enable backwards compatibility for the arch change, but the implication is that this is a bridge between the "old" and the "new".

I'm concerned that the "new" isn't well thought out from the developer perspective, so we'll either end up using that proxy layer forever or changing the architecture again. (And if we keep the proxy layer, we get the added maintenance burden of maintaining the proxy layer.)

Thanks for reading and considering, @IgorChvyrov-sm -- i know this PR has been weeks in the making.

Let me know if there are any flaws in my reasoning, i'm still getting my footing here!

@IgorChvyrov-sm
Copy link
Contributor Author

Hey so just as we discussed in Slack @IgorChvyrov-sm, here's my breakdown of the developer experience concern I have about this PR.

State of the arch conversation reflected in this PR:

This change theoretically enables the SDK to more easily keep up with Conductor API changes.

This change makes it so that there are 3 files per API call.

For the developer, this change may have the following effects:

Import Pattern Confusion 🔀

Before: Single import path**

from conductor.client.workflow_client import WorkflowClient

After: Three possible import paths**

Option 1: Proxy layer (backward compatible) from conductor.client.http.api.workflow_resource_api import WorkflowResourceApi

Option 2: Adapter layer from conductor.client.adapters.api.workflow_resource_api_adapter import WorkflowResourceApiAdapter

Option 3: Codegen layer (raw) from conductor.client.codegen.api.workflow_resource_api import WorkflowResourceApi

Impact

Developers unsure which to use, potential for inconsistent usage across codebase. Future architectural changes (if desired) may cause more issues for developers.

Debugging Struggles 🔍

Stack traces now go through 3 layers for every API call:

  File "my_app.py", line 10, in create_workflow
  File "conductor/client/http/api/workflow_resource_api.py", line 4  # Just a re-export
  File "conductor/client/adapters/api/workflow_resource_api_adapter.py", line 5  # Empty pass-through
  File "conductor/client/codegen/api/workflow_resource_api.py", line 234  # Actual implementation

Impact

May be harder to trace errors, longer stack traces, more cognitive overhead.

Maintenance Commitment Ambiguities

We're promising to keep a "proxy layer" to enable backwards compatibility for the arch change, but the implication is that this is a bridge between the "old" and the "new".

I'm concerned that the "new" isn't well thought out from the developer perspective, so we'll either end up using that proxy layer forever or changing the architecture again. (And if we keep the proxy layer, we get the added maintenance burden of maintaining the proxy layer.)

Thanks for reading and considering, @IgorChvyrov-sm -- i know this PR has been weeks in the making.

Let me know if there are any flaws in my reasoning, i'm still getting my footing here!

Hey @nthmost-orkes!

I agree with you that we should take the time to consider the experience of developers who will be using the SDK. I plan to conduct research on imports to make them less confusing in future versions and prepare a document that we can discuss to arrive at the best solution. At this stage, to maintain backward compatibility and avoid breaking changes, we will have to keep the architecture presented in this PR.

Copy link

@nthmost-orkes nthmost-orkes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, approving on the basis that we'll be keeping the proxy layer in place for the sake of developer experience.

We can remove this proxy later if and only if we decide on an SDK-user-friendly vision for usage of the conductor API, since this new architecture has some usability issues.

@IgorChvyrov-sm IgorChvyrov-sm merged commit 0c2959a into main Sep 24, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants