-
Notifications
You must be signed in to change notification settings - Fork 279
Added the plugin guide #3990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Added the plugin guide #3990
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📖 Docs PR preview links
|
drewhoskins-temporal
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! I made a few suggested edits and tagged Tim in for a few.
(Bearing in mind that in some cases I was editing my own content. :-) )
docs/develop/plugins-guide.mdx
Outdated
| Temporal's Activity retry mechanism gives applications the benefits of durable execution. | ||
| For example, Temporal will keep track of the exponential backoff delay even if the Worker crashes. Since Temporal can’t tell when a Worker crashes, Workflows rely on the [start_to_close_timeout](https://docs.temporal.io/encyclopedia/detecting-activity-failures#start-to-close-timeout) to know how long to wait before assuming that an Activity is inactive. | ||
|
|
||
| Be cautious when doing retries within your Activity because it lengthens this overall timeout to be longer than the longest possible retry sequence. That means it takes too long to recover from other causes of failure. Such internal retries also prevent users from counting failure metrics and make it harder for users to debug in Temporal UI when something is wrong. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This ended up feeling a bit long and redundant on my re-read.
| Be cautious when doing retries within your Activity because it lengthens this overall timeout to be longer than the longest possible retry sequence. That means it takes too long to recover from other causes of failure. Such internal retries also prevent users from counting failure metrics and make it harder for users to debug in Temporal UI when something is wrong. | |
| Be cautious when doing retries within your Activity because it lengthens the needed Activity timeout. Such internal retries also prevent users from counting failure metrics and make it harder for users to debug in Temporal UI when something is wrong. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same deal as my above comments, not sure this whole section even needs to be here at all. We're just repeating advice we give elsewhere in the docs - none of this is specific to plugins
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, as above, @flippedcoder and I will resolve this in a follow-up PR.
docs/develop/plugins-guide.mdx
Outdated
| You can provide a library with functionality for use within a Workflow. Your library will call elements you include in your Plugin Activities, Child Workflows, Signals, Updates, Queries, Nexus Operations, Interceptors, Data Converters, and any other code as long as it follows these requirements: | ||
|
|
||
| - It should be [deterministic](https://docs.temporal.io/workflows#deterministic-constraints), running the same way every time it’s executed. Non-deterministic code should go in Activities or Nexus Operations. | ||
| - It should be used in the Python [sandbox](https://docs.temporal.io/develop/python/python-sdk-sandbox). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is python-specific. Could we maybe do this in a tab? Or otherwise reformat into an SDK-specific section or something?
s/used/usable/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few questions here:
- Do we have a code example for how a plugin would be used in the Python sandbox?
- Do we have sandboxes for the other SDKs?
- Are we going to recommend using a sandbox for all SDKs?
This will help me figure out how to structure this in the doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good questions.
- I like the idea of putting a code snippet for sandbox in python. @tconley1428 can you provide?
- Only python has a sandbox.
- Other languages may have their own common/special gotchas. @tconley1428 will know better, esp. in typescript which he is working on.
Maybe for now we should just include a one-off delineator, and then we can refactor once the typescript stuff comes in?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sandbox is enabled by default, all existing samples already have it.
Typescript also has a sandbox which is actually more robust, but it is not optional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- It should be used in the Python sandbox.
Yeah this doesn't make a lot of sense. When you use it in a workflow, which is the whole context here, it is either sandboxed or not based on worker configuration. This isn't something that needs to be specified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tim's code--perhaps we should make this a snippet?
def workflow_runner(runner: Optional[WorkflowRunner]) -> WorkflowRunner:
if not runner:
raise ValueError("No WorkflowRunner provided to the OpenAI plugin.")
# If in sandbox, add additional passthrough
if isinstance(runner, SandboxedWorkflowRunner):
return dataclasses.replace(
runner,
restrictions=runner.restrictions.with_passthrough_modules("mcp"),
)
return runner
SimplePlugin(..., workflow_runner=workflow_runner)
docs/develop/plugins-guide.mdx
Outdated
|
|
||
| - It should be [deterministic](https://docs.temporal.io/workflows#deterministic-constraints), running the same way every time it’s executed. Non-deterministic code should go in Activities or Nexus Operations. | ||
| - It should be used in the Python [sandbox](https://docs.temporal.io/develop/python/python-sdk-sandbox). | ||
| - It should be designed to handle being restarted, resumed, or executed independently of where or how it originally began without losing correctness or state consistency. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - It should be designed to handle being restarted, resumed, or executed independently of where or how it originally began without losing correctness or state consistency. | |
| - It should be designed to handle being restarted, resumed, or executed in a different process from where it originally began without losing correctness or state consistency. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tconley1428, can you provide details about how to test this by disabling the workflow cache, starting with python? My thought is we can add a "Testing your plugin" section at the bottom and then link to that here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
☝🏽 I like this idea. If we can have a code snippet to show how test by disabling the workflow cache, it'll definitely help out users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
client = await Client.connect(
"localhost:7233",
plugins=[
plugin,
],
)
async with Worker(
client,
task_queue="task-queue",
max_cached_workflows=0
):
Test some things.
| async def run(self, name: str) -> str: | ||
| return f"Hello, {name}!" | ||
|
|
||
| plugin = SimplePlugin( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tconley1428 my initial reaction was that this is unrealistic, since plugins are generally designed to be reused. I'd expect people to create a plugin class that declares the Workfows rather than make a global variable for people to use. If you agree, can you help update the code, here and in the other examples?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not really clear here where the different code lives. If the plugin is defined in some library, it likely will be created via constructor or function, but there's not really any inherent problem with a global. If it is just in the user's code, it's not really clear if this is a global or a local variable in the various main functions, given that there isn't really a scope in the code example at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tconley1428 I don't follow the objection or how this has anything to do with modules. What I mean is:
class MyLibraryPlugin(SimplePlugin):
...
Goal I'm aiming for these sample snippets is to be as much like user's realistic code as possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Misunderstood what you were asking then. The existing snippet makes sense for a plugin without arguments. You could do as you suggest if you think it will be more extensible, feel free.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, cool, if this is normal then 👍
| # Should consider interactions with other plugins, | ||
| # as this will override the data converter. | ||
| # This may mean failing, warning, or something else |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- This doesn't quite match what we did for OpenAI -- please double check that this is our best up-to-date advice.
- I can't find it, but I remember seeing a snippet where we pull the user's existing codecs into the payload. Should we do something like that here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def _data_converter(converter: Optional[DataConverter]) -> DataConverter:
if converter is None:
return DataConverter(payload_converter_class=OpenAIPayloadConverter)
elif converter.payload_converter_class is DefaultPayloadConverter:
return dataclasses.replace(
converter, payload_converter_class=OpenAIPayloadConverter
)
elif not isinstance(converter.payload_converter, OpenAIPayloadConverter):
raise ValueError(
"The payload converter must be of type OpenAIPayloadConverter."
)
return converter
Is our current implementation. We could do so to demonstrate it, but the desires of various payloads may vary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where we pull the user's existing codecs into the payload
That is the code you linked and I pasted.
docs/develop/plugins-guide.mdx
Outdated
|
|
||
| # Plugins | ||
|
|
||
| A **Plugin** is an abstraction that will let your users add your feature to their Workflows in one line. It makes it seamless for platform developers to add their own custom functionality to many Workflows. Using plugins, you can build reusable open source libraries or build add-ons for engineers at your company. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe re-word this to be more generic than adding stuff to workflows, since you can do more than that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you have in mind?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do y'all think about this?
A **Plugin** is an abstraction that will let your users add your feature to their Temporal primitives, like Workflows and Activities, in one line. It makes it seamless for platform engineers to add their own custom functionality to many primitives.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A Plugin is an abstraction that allows you to customize any aspect of Temporal Worker setup, including registering Workflow and Activity definitions, modifying worker and client options, and more. Using plugins, you can build reusable open source libraries or build add-ons for engineers at your company.
Would be my take
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fine with me.
| - Single write operations | ||
| - Batches of similar writes | ||
| - One or more read operations followed by a write operation | ||
| - A read that should be memoized, like an LLM call, a large download, or a slow-polling read |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads awkwardly to me. Each line sort of contradicts the last. Presumably we have advice elsewhere in the docs we can link to about what an activity should be scoped to?
This could be just "Activities should be scoped to a small, ideally idempotent, unit of work" or something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Presumably we have advice elsewhere in the docs we can link to about what an activity should be scoped to?
We actually don't AFAICT! I'd love to link out to the general activity docs, but I figured, let's get the content in here for starters and then refactor next.
Let me suggest a rewording.
re: idempotent, we have to be careful there since LLMs are not idempotent by some definitions.
| - Activity arguments and return values must be serializable. | ||
| - Activities that perform writes should be idempotent. | ||
| - Activities have [timeouts](https://docs.temporal.io/develop/python/failure-detection#heartbeat-timeout) and [retry policies](https://docs.temporal.io/encyclopedia/retry-policies). To be Activity-friendly, your operation should either complete within a few minutes or it should support the ability to heartbeat or poll for a result. This way it will be clear to the Workflow when the Activity is still making progress. | ||
| - You need to specify at least one timeout, typically the [start_to_close_timeout](https://docs.temporal.io/encyclopedia/detecting-activity-failures#start-to-close-timeout). Keep in mind that the shorter the timeout, the faster Temporal will retry upon failure. See the [timeouts and retry policies](#timeouts-and-retry-policies) to learn more. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely seems like we should link to general advice on activities somewhere in this section at least
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, but we don't actually have almost any of this advice anywhere that I could find. I've opened a task as a follow-up to figure out where to put it.
We can put it here: https://docs.temporal.io/activities and here https://docs.temporal.io/activity-definition in some combination. (But separate from this PR, I hope.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we can make that update to the activity definition in a separate PR. For now, let's leave it here and I can come back and change it to a link later.
docs/develop/plugins-guide.mdx
Outdated
| Temporal's Activity retry mechanism gives applications the benefits of durable execution. | ||
| For example, Temporal will keep track of the exponential backoff delay even if the Worker crashes. Since Temporal can’t tell when a Worker crashes, Workflows rely on the [start_to_close_timeout](https://docs.temporal.io/encyclopedia/detecting-activity-failures#start-to-close-timeout) to know how long to wait before assuming that an Activity is inactive. | ||
|
|
||
| Be cautious when doing retries within your Activity because it lengthens this overall timeout to be longer than the longest possible retry sequence. That means it takes too long to recover from other causes of failure. Such internal retries also prevent users from counting failure metrics and make it harder for users to debug in Temporal UI when something is wrong. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same deal as my above comments, not sure this whole section even needs to be here at all. We're just repeating advice we give elsewhere in the docs - none of this is specific to plugins
Co-authored-by: Spencer Judge <spencer@temporal.io>
Co-authored-by: Spencer Judge <spencer@temporal.io>
Co-authored-by: Spencer Judge <spencer@temporal.io>
Co-authored-by: Spencer Judge <spencer@temporal.io>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
docs/develop/plugins-guide.mdx
Outdated
|
|
||
| # Plugins | ||
|
|
||
| A **Plugin** is an abstraction that will let your users add your feature to their Workflows in one line. It makes it seamless for platform developers to add their own custom functionality to many Workflows. Using plugins, you can build reusable open source libraries or build add-ons for engineers at your company. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you have in mind?
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
Co-authored-by: Drew Hoskins <166441821+drewhoskins-temporal@users.noreply.github.com>
7e53964 to
4852d49
Compare
What does this PR do?
Adds documentation for using plugins with the Python SDK.
Notes to reviewers