Skip to content

Add system Nexus signalWithStartWorkflow API#2127

Draft
tconley1428 wants to merge 5 commits into
mainfrom
nexus-api-gen-signal-with-start
Draft

Add system Nexus signalWithStartWorkflow API#2127
tconley1428 wants to merge 5 commits into
mainfrom
nexus-api-gen-signal-with-start

Conversation

@tconley1428

Copy link
Copy Markdown
Contributor

Summary

  • add system Nexus TypeScript generation to build:protos using nex-gen
  • expose generated signalWithStartWorkflow workflow API and checked-in generated review output
  • serialize system Nexus envelopes with default JSON through the isolate, then protobuf-binary in the worker while applying codecs only to nested payloads
  • add unit and integration coverage for codec/payload converter behavior

Validation

  • NEX_GEN_BIN=../nexus-api-gen/target/debug/nex-gen pnpm run build:protos
  • pnpm --filter @temporalio/test run build:ts
  • pnpm -C packages/test exec ava ./lib/test-workflow-codec-runner.js
  • pnpm -C packages/test exec ava ./lib/test-system-nexus.js

Notes

  • Draft while the corresponding nex-gen TypeScript export/metadata changes are finalized and published.

Comment thread scripts/gen-system-nexus.ts Outdated
Comment thread packages/worker/src/system-nexus-payload-visitor.ts Outdated
Comment thread packages/worker/src/system-nexus-payload-visitor.ts Outdated
Comment thread packages/worker/src/system-nexus-payload-visitor.ts Outdated
Comment thread packages/worker/src/system-nexus-payload-converter.ts Outdated
Comment thread packages/worker/src/system-nexus-payload-converter.ts
Comment thread packages/worker/src/system-nexus-payload-converter.ts
Comment thread packages/worker/src/system-nexus-payload-converter.ts Outdated
Comment thread .github/workflows/ci.yml
function nexGenCommand(): string {
if (process.env.NEX_GEN_BIN) return process.env.NEX_GEN_BIN;
if (!commandExists('nex-gen')) {
run('cargo', ['install', '--locked', 'nex-gen', '--force']);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we don't have gen scripts mess with host systems. I think erroring in this case with how to install is just fine.

resolve(workflowOutputDir, 'operations/signal-with-start-workflow.ts'),
]) {
let source = readFileSync(path, 'utf8');
source = source.replace(/((?:from|export .* from) ['"]\.{1,2}[^'"]*)\.ts(['"])/g, '$1$2');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why nexus-gen creates these files with imports that specify .ts as the extension? I don't think that is correct unless we run TS files directly instead of building them to JS first.

"dependencies": {
"@temporalio/common": "workspace:*",
"@temporalio/proto": "workspace:*",
"long": "^5.2.3",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can see we only use this for a type reference. If so we can move it to devDependency

);
}

function postProcessGeneratedWorkflowFiles(): void {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would appreciate comment describing what/why post processing is necessary

import type Long from "long";

function int64ToNumber(
value: Long | number | string | object | null | undefined,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why this helper accepts so many types, but it looks like Long | null | undefined are the only ones ever passed?

});
}

function generateWorkerVisitor(operations: OperationInfo[], messages: Map<string, MessageInfo>): void {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure how I feel about this. Could we piggyback on protobufjs reflection to just directly implement this instead of generating it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants