User metadata #1657

THardy98 · 2025-03-21T19:51:16Z

What was changed

Added functionality for users to provide staticDetails and staticSummary metadata to workflow start commands.
These can be retrieved via describe on the workflow handler and are visibile via the UI/CLI.

Users can set currentDetails - a mutable details string within the workflow - via SetCurrentDetails and retrieve it via GetCurrentDetails. This is visible via the UI/CLI and the __temporal_workflow_metadata internal query.

Users can add a summary (short string description) when setting a timer and when starting an activity.

A notable part of the change is the addition of runWithOptions to activities. Activities can now override their activity options at call time, like so:

* // Setup Activities from module exports
 * const { httpGet, otherActivity } = proxyActivities<typeof activities>({
 *   startToCloseTimeout: '30 minutes',
 * });
 *
 * // Use activities with default options
 * const result1 = await httpGet('http://example.com');
 *
 * // Override options for specific activity calls
 * const result2 = await httpGet.runWithOptions({
 *   staticSummary: 'Fetches data from external API',
 *   scheduleToCloseTimeout: '5m'
 * }, ['http://api.example.com']);
 *
 * const result3 = await otherActivity.runWithOptions({
 *   staticSummary: 'Processes the fetched data',
 *   taskQueue: 'special-task-queue'
 * }, [data]);

Closes [Feature Request] Support user metadata #1544
How was this tested:
Couple integration tests
Any docs updates needed?
Maybe

…es field to add a summary to an activity

mjameswh · 2025-05-13T18:03:58Z

packages/common/src/workflow-options.ts

+   * General fixed details for this workflow execution that may appear in UI/CLI.
+   * This can be in Temporal markdown format and can span multiple lines.
+   *
+   * @experimental


Please add a short comment on all @experimental tags (see what we did elsewhere). Benefit isn't huge, but at least, it identifies which "feature" the experimental API is linked with.

I believe i've done so in all the relevant places

mjameswh · 2025-05-13T21:53:39Z

packages/client/src/schedule-helpers.ts

@@ -240,6 +245,7 @@ export async function encodeScheduleAction(
  action: CompiledScheduleAction,
  headers: Headers
 ): Promise<temporal.api.schedule.v1.IScheduleAction> {
+  const jsonConverter = new JsonPayloadConverter();


Use the data converters configured on the client (i.e. custom payload converter and codec).

mjameswh · 2025-05-13T21:53:46Z

packages/client/src/schedule-helpers.ts

@@ -312,6 +322,8 @@ export async function decodeScheduleAction(
  pb: temporal.api.schedule.v1.IScheduleAction
 ): Promise<ScheduleDescriptionAction> {
  if (pb.startWorkflow) {
+    const jsonConverter = new JsonPayloadConverter();


Use the data converters configured on the client (i.e. custom payload converter and codec).

mjameswh · 2025-05-13T21:55:43Z

packages/client/src/workflow-client.ts

@@ -1196,6 +1197,7 @@ export class WorkflowClient extends BaseClient {
  protected async _signalWithStartWorkflowHandler(input: WorkflowSignalWithStartInput): Promise<string> {
    const { identity } = this.options;
    const { options, workflowType, signalName, signalArgs, headers } = input;
+    const jsonConverter = new JsonPayloadConverter();


Use the data converters configured on the client (i.e. custom payload converter and codec).

mjameswh · 2025-05-13T22:20:31Z

packages/client/src/workflow-client.ts

@@ -1225,6 +1227,10 @@ export class WorkflowClient extends BaseClient {
          : undefined,
      cronSchedule: options.cronSchedule,
      header: { fields: headers },
+      userMetadata: {
+        summary: jsonConverter.toPayload(options?.staticSummary),


You will need to use await encodeToPayload(...) instead here and everything else in the client package. Not in Workflows and Worker, though.

mjameswh · 2025-05-13T22:28:27Z

packages/workflow/src/workflow.ts

+    return undefined;
+  }
+
+  const jsonConverter = new JsonPayloadConverter();


Don't create a JSON payload converter. Use the payload converter from the activator.

mjameswh · 2025-05-13T23:51:06Z

packages/workflow/src/interceptors.ts

@@ -80,6 +80,7 @@ export interface ActivityInput {
  readonly options: ActivityOptions;
  readonly headers: Headers;
  readonly seq: number;
+  readonly cmdOpts?: WorkflowCommandOptions;


I don't think we should have this WorkflowCommandOptions type. I even wonder if we should inline summary and details directly in ActivityInput (without the extra UserMetadata nesting). Or add these two to ActivityOptions, though that might pose some minor challenge regarding the type provided to the proxyActivities() function.

What did we do in other SDKs?

Typically they are inline, I've changed it as such (aside from using ActivityOptions for summary/details for an activity, and TimerOptions as you suggested below)

mjameswh · 2025-05-13T23:59:55Z

packages/workflow/src/workflow.ts

 */
-export function sleep(ms: Duration): Promise<void> {
+export function sleep(ms: Duration, summary?: string): Promise<void> {


I'm worried this will evolve badly. What if we then need to add description, and then other properties? Instead, I think I'd define a TimerOptions type, and take this as a second argument here. Then, we can easily add whatever properties we need.

Out of curiosity, what do we do in other SDKs?

typically we inline, but went with TimerOptions here given your reasoning

mjameswh · 2025-05-14T00:03:39Z

packages/workflow/src/workflow.ts

+  activityType: string,
+  args: any[],
+  options: ActivityOptions,
+  summary?: string


Yeah… So, looking at this one, I really feel this should be part of ActivityOptions.

mjameswh · 2025-05-14T01:01:28Z

packages/workflow/src/workflow.ts

+   * @param summaries Record mapping activity names to their summary descriptions
+   * @returns A new proxy with the provided summaries
+   */
+  withSummaries(summaries: Record<string, string>): ActivityInterfaceFor<A>;


🤔 That feels a bit awkward, and would prevent having an activity named withSummaries (ok, that's arguably very unlikely, but still means this approach would evolve badly).

I think we could get a better DX by using something similar to this, but at the activity-level rather than at the activities-level (mind the plural). That would also allow any extra option to be set at the call site, which is a recurring request.

For example, we could have:

const { echo } = proxyActivities<MyActivities>({ startToCloseTimeout: '5m' }); // No extra options await echo(myArg1, myArg2); // With extra options, higher order function style await echo.withOptions({ summary: "..." })(myArg1, myArg2); // Some other possible variants to be considered await echo.withOptions({ summary: "..." }, myArg1, myArg2); await echo.withOptions({ summary: "..." }, [myArg1, myArg2]);

I'll let you think you a bit about this, but we should probably include others in this discussion.

opted for:
await echo.runWithOptions(options, args);

to allow users to override their default activity options

THardy98 · 2025-06-03T00:00:27Z

also quickly added lazy decoding summary/details from workflow description

can't do so (or at least annoying to do so) for schedule description because the description type has an invariant where the description must be a superclass of schedule options (which necessitate summary/details being strings)

mjameswh · 2025-06-05T21:46:41Z

packages/workflow/src/interceptors.ts

+/**
+ * User metadata that can be attached to workflow commands.
+ *
+ * Current used for:


Suggested change

* Current used for:

* Currently used for:

mjameswh · 2025-06-05T21:47:24Z

packages/workflow/src/interceptors.ts

+ * User metadata that can be attached to workflow commands.
+ *
+ * Current used for:
+ * - startTimer, scheduleActivity/scheduleLocalActivity commands


Suggested change

* - startTimer, scheduleActivity/scheduleLocalActivity commands

* - `StartTimer`, `ScheduleActivity` and `ScheduleLocalActivity` commands

mjameswh · 2025-06-05T21:56:12Z

packages/workflow/src/workflow.ts

+        startToFireTimeout: msToTs(durationMs),
+      },
+      userMetadata: options && {
+        summary: options.summary ? activator.payloadConverter.toPayload(options.summary) : undefined,


What if summary is an empty string? As it is now, it will be replaced by undefined. Not sure what would be the desired result.

If we want to keep empty strings, then it might be cleaner to use some toOptionalPayload helper.

And if we don't care about empty strings, then should instead move the terniary out of this object:

userMetadata: options?.summary ? { summary: activator.payloadConverter.toPayload(options.summary), } : undefined,

mjameswh · 2025-06-05T22:03:11Z

packages/workflow/src/workflow.ts

@@ -130,8 +134,9 @@ function timerNextHandler(input: TimerInput) {
 *
 * @param ms sleep duration - number of milliseconds or {@link https://www.npmjs.com/package/ms | ms-formatted string}.
 * If given a negative number or 0, value will be set to 1.
+ * @param summary a short summary/description of the timer. Can serve as a timer ID.


@param summary is boggus.

mjameswh · 2025-06-05T22:08:19Z

packages/workflow/src/workflow.ts

@@ -330,6 +345,7 @@ export async function scheduleLocalActivity<R>(
        seq,
        attempt,
        originalScheduleTime,
+        ...(summary !== undefined && { cmdOpts: { userMetadata: { summary } } }),


Left over? I don't see any other traces of cmdOpts.

mjameswh · 2025-06-06T06:09:53Z

packages/workflow/src/workflow.ts

@@ -301,7 +315,8 @@ export function scheduleActivity<R>(activityType: string, args: any[], options:
 export async function scheduleLocalActivity<R>(
  activityType: string,
  args: any[],
-  options: LocalActivityOptions
+  options: LocalActivityOptions,
+  summary?: string


summary should be part of options, same as for regular activities

mjameswh · 2025-06-06T06:13:35Z

packages/workflow/src/workflow.ts

+/**
+ * Extends ActivityInterfaceFor to include the withOptions method
+ */
+export type ActivityInterfaceWithOptions<A> = ActivityInterfaceFor<A> & {


That type seems completely wrong. It has no reason to exist. You can't call runWithOptions on the activity proxy object itself.

mjameswh · 2025-06-06T06:14:01Z

packages/workflow/src/workflow.ts

+   *
+   * @param options ActivityOptions
+   * @param args: list of arguments
+   * @returns return value of the activity


@experimental

mjameswh · 2025-06-06T06:23:48Z

packages/workflow/src/workflow.ts

+          overrideOptions: ActivityOptions,
+          args: any[]
+        ): Promise<unknown> {
+          return scheduleActivity(activityType, args, { ...options, ...overrideOptions });


Not so fast. You will need to merge recursively on retry policy and priority.

mjameswh · 2025-06-06T06:26:11Z

packages/workflow/src/workflow.ts

-    {},
-    {
+
+  function createActivityProxy(options: ActivityOptions): ActivityInterfaceWithOptions<A> {


Why did you added this inner function, rather than create the proxy directly?

mjameswh · 2025-06-06T06:27:32Z

packages/workflow/src/workflow.ts

-        return function localActivityProxyFunction(...args: unknown[]) {
-          return scheduleLocalActivity(activityType, args, options);
+
+        function activityProxyFunction(...args: unknown[]): Promise<unknown> {


That function name may be visible in stack traces on the workflow side. Better keep the "localActivity" name

mjameswh · 2025-06-06T06:28:17Z

packages/workflow/src/workflow.ts

+          overrideOptions: ActivityOptions,
+          args: any[]
+        ): Promise<unknown> {
+          return scheduleLocalActivity(activityType, args, { ...options, ...overrideOptions });


Same as before, need to handle recursive options merging on retryOptions and priority.

mjameswh · 2025-06-06T06:30:54Z

packages/common/src/activity-options.ts

+   *
+   * @experimental User metadata is a new API and suspectible to change.
+   */
+  staticSummary?: string;


I think that should be only summary, no static. What's the terminology in other SDKs?

mjameswh · 2025-06-06T06:31:06Z

packages/common/src/activity-options.ts

+   *
+   * @experimental User metadata is a new API and suspectible to change.
+   */
+  staticSummary?: string;


mjameswh · 2025-06-06T06:55:23Z

I believe there should also be user metadata on:

child workflows
conditions
signals
update

THardy98 added 3 commits March 20, 2025 19:33

init impl of user metadata

7b3aa1c

add test, removed proxyActivities overloads instead added withSummari…

2dab40d

…es field to add a summary to an activity

schedule metadata test, fix schedule action decode

6af2e2c

THardy98 requested a review from a team as a code owner March 21, 2025 19:51

Merge branch 'main' into user-metadata

e0489cd

THardy98 force-pushed the user-metadata branch from 9bceb64 to e0489cd Compare March 24, 2025 18:20

test fix

031e295

mjameswh reviewed May 13, 2025

View reviewed changes

mjameswh reviewed May 14, 2025

View reviewed changes

THardy98 added 2 commits June 2, 2025 21:16

use supplied data converter

d496966

inline options, lazily decode summary/details on workflow description

1efb3f0

THardy98 force-pushed the user-metadata branch from 414967a to 1efb3f0 Compare June 2, 2025 23:42

THardy98 added 2 commits June 3, 2025 01:48

update proxyActivities comment to show runWithOptions usage

ebe745f

Merge branch 'main' into user-metadata

2c8f922

THardy98 added 2 commits June 3, 2025 02:45

add runWithOptions for local activities

67d8e06

allow for undefined userMetadata when scheduling activity

c491683

THardy98 force-pushed the user-metadata branch from d2bb388 to c491683 Compare June 3, 2025 00:48

THardy98 requested a review from mjameswh June 3, 2025 07:08

mjameswh reviewed Jun 5, 2025

View reviewed changes

mjameswh reviewed Jun 6, 2025

View reviewed changes

	* - startTimer, scheduleActivity/scheduleLocalActivity commands
	* - `StartTimer`, `ScheduleActivity` and `ScheduleLocalActivity` commands

User metadata #1657

Are you sure you want to change the base?

User metadata #1657

Uh oh!

Conversation

THardy98 commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was changed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

THardy98 Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

THardy98 commented Jun 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjameswh Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjameswh Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

THardy98 commented Mar 21, 2025 •

edited

Loading

THardy98 Jun 2, 2025 •

edited

Loading

mjameswh Jun 5, 2025 •

edited

Loading

mjameswh Jun 6, 2025 •

edited

Loading