Skip to content

Async activity support and describe interceptor #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 1, 2022

Conversation

cretz
Copy link
Member

@cretz cretz commented Jun 1, 2022

What was changed

Added async activity support and made the client describe call use an interceptor

Checklist

  1. Closes Async activity support #24

@cretz cretz marked this pull request as ready for review June 1, 2022 17:04
@cretz cretz requested a review from a team June 1, 2022 17:04
# because the one passed in may be a method or some other partial
# that represents the real callable instead of what the decorator
# used.
return dataclasses.replace(ret, fn=fn)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm probably missing context but why here and not in from_callable or even when generating __temporal_activity_definition?

Copy link
Member Author

@cretz cretz Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm probably missing context but why here and not in from_callable

You're not missing any context, I should have put this there. Will change on next commit (or on its own assuming all else is good).

when generating __temporal_activity_definition

This is what I do, but the problem is for activity methods (which we should support) the decorator is invoked on the unbound method but the callable coming in for from_callable is at call time has self bound. I want to have the bound method as fn.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

class HeartbeatAsyncActivityInput:
"""Input for :py:meth:`OutboundInterceptor.heartbeat_async_activity`."""

id_or_token: Union[Tuple[str, str, str], bytes]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can use namedtuple here or some other more meaningful identifier than 3 strings

Copy link
Member Author

@cretz cretz Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was just being lazy. I will change to a dataclass named AsyncActivityIDReference or similar (saw FullActivityId in TS).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@@ -1352,12 +1443,36 @@ async def terminate_workflow(self, input: TerminateWorkflowInput) -> None:
"""Called for every :py:meth:`WorkflowHandle.terminate` call."""
await self.next.terminate_workflow(input)

### Async activity calls
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice that you made these interceptable, I wonder if anyone will use this feature though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect not, but it was easy enough.

Copy link
Member

@bergundy bergundy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

Take a look at TS, we throw a special error when an activity is not found.
There are also tests that verify this.

@cretz
Copy link
Member Author

cretz commented Jun 1, 2022

Take a look at TS, we throw a special error when an activity is not found.

I noticed that this is done with workflow not found too whereas I just throw the gRPC exception. My concern is not found can mean more than that. For example, with the latest server changes in master, if we delete a namespace and you call "start workflow", you'll get a WorkflowNotFound error in TS which doesn't make sense. The only way to differentiate namespace not found from normal not found is to check the details. And note that is "normal not found" because the server has no concept of "workflow" or "activity" not found so you'll never know for sure.

I have opened temporalio/features#59 for discussion.

@mfateev
Copy link
Member

mfateev commented Jun 1, 2022

In Java, we have two ways to create the async completion handler. One is to be used by a separate process, and the second is to be used by the same process. The second way obeys the limit of parallel activity executions.

@cretz
Copy link
Member Author

cretz commented Jun 1, 2022

In Java [...] obeys the limit of parallel activity executions.

After off-PR discussion, it was deemed that since a Python's async activity doesn't eat a valuable OS thread resource, there isn't value in this advanced form of in-process async completion

@cretz cretz force-pushed the async-activity branch from 75144b1 to 42c22d5 Compare June 1, 2022 19:00
@cretz cretz merged commit 4657453 into temporalio:main Jun 1, 2022
@cretz cretz deleted the async-activity branch June 1, 2022 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Async activity support
3 participants