Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: inline BeginTransaction with first statement #1692

Merged
merged 28 commits into from
Nov 7, 2022
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3d06c45
feat: inline BeginTransaction with first statement
ko3a4ok Aug 6, 2022
8d5c6d6
fix: prevent multiple begin transactions happen during a race conditi…
ko3a4ok Aug 12, 2022
15944f5
Merge branch 'googleapis:main' into inline-begin-txns
ko3a4ok Aug 24, 2022
dd9891e
fix: minor
ko3a4ok Aug 26, 2022
6b57710
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Sep 14, 2022
976ee59
Merge branch 'googleapis:main' into inline-begin-txns
ko3a4ok Sep 14, 2022
154872b
fix: call explicit begin transaction only after the first attempt.
ko3a4ok Sep 14, 2022
2caaa0f
feat: add inline begin transaction for batch DML requests
ko3a4ok Sep 20, 2022
9929be5
Merge branch 'main' into inline-begin-txns
ko3a4ok Sep 20, 2022
f65e2d1
fix: lint
ko3a4ok Sep 20, 2022
6d83a94
fix: explicit begin transaction for blind commit if a transaction run…
ko3a4ok Sep 23, 2022
7eb3831
Merge branch 'googleapis:main' into inline-begin-txns
ko3a4ok Sep 23, 2022
f9aa221
Merge branch 'googleapis:main' into inline-begin-txns
ko3a4ok Oct 12, 2022
eeef8cf
feat: adding more unit tests
ko3a4ok Oct 12, 2022
4fff842
fix: explicit begin transaction for unknown error
ko3a4ok Oct 17, 2022
cfa405e
Merge branch 'main' into inline-begin-txns
surbhigarg92 Oct 18, 2022
53f1e1c
fix: format
ko3a4ok Oct 17, 2022
e146e89
Merge remote-tracking branch 'origin/inline-begin-txns' into inline-b…
ko3a4ok Oct 21, 2022
1b82e78
Merge branch 'main' into inline-begin-txns
surbhigarg92 Oct 31, 2022
e5585ff
fix: tests after merge
ko3a4ok Oct 31, 2022
241394d
Lint fix
surbhigarg92 Nov 2, 2022
fa3c024
fix: lint
ko3a4ok Oct 31, 2022
c2b2ac6
Merge remote-tracking branch 'origin/inline-begin-txns' into inline-b…
ko3a4ok Nov 2, 2022
2f226e1
fix: samples test
ko3a4ok Nov 2, 2022
6d94a86
Merge remote-tracking branch 'origin/inline-begin-txns' into inline-b…
ko3a4ok Nov 4, 2022
b0bccf5
Merge branch 'main' into inline-begin-txns
surbhigarg92 Nov 4, 2022
346abea
fix: system test
ko3a4ok Nov 5, 2022
311a80f
fix: system emulator test
surbhigarg92 Nov 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: minor
  • Loading branch information
ko3a4ok committed Aug 26, 2022
commit dd9891ed4213ff805c5daa5a6d34c4dc7726a47e
14 changes: 8 additions & 6 deletions src/transaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -570,7 +570,7 @@ export class Snapshot extends EventEmitter {

if (this.id) {
transaction.id = this.id as Uint8Array;
} else if (typeof this._options.readWrite !== 'undefined') {
} else if (this._options.readWrite) {
transaction.begin = this._options;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind adding a test that verifies the following:

  1. A query is the first statement in a read/write transaction. A transaction ID is successfully returned by initial request.
  2. One or more PartialResultSets are returned by the stream, with (at least) one of them returning a resume token.
  3. The stream fails halfway with an UNAVAILABLE error and the stream is restarted with a resume token.

Step 3 should use the transaction ID that was returned by step 1, and not include a BeginTransaction option.

(https://github.com/googleapis/nodejs-spanner/pull/1253/files is a very old implementation for the same. Some of the tests there might be re-usable.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. thanks a lot for this comment: it helped to find a bug in the code

} else {
transaction.singleUse = this._options;
Expand Down Expand Up @@ -615,7 +615,7 @@ export class Snapshot extends EventEmitter {
json,
jsonOptions,
maxResumeRetries,
}).on('response', response => {
})?.on('response', response => {
if (response.metadata && response.metadata!.transaction && !this.id) {
this._update(response.metadata!.transaction);
}
Expand Down Expand Up @@ -1040,7 +1040,7 @@ export class Snapshot extends EventEmitter {
const transaction: spannerClient.spanner.v1.ITransactionSelector = {};
if (this.id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not take into account that a transaction could in theory have multiple requests in flight at the same time. If a transaction starts out with sending two SELECT statements to the backend, it might very well be that the first that is sent has not yet returned a transaction id before the second is being sent. That will cause both requests to include a BeginTransaction option.

Consider the following test case (the getRowCountFromStreamingSql function is defined in test/Spanner.ts):

    it('should only inline one begin transaction', async () => {
      const database = newTestDatabase();
      await database.runTransactionAsync(async tx => {
        const rowCount1 = getRowCountFromStreamingSql(tx, {sql: selectSql});
        const rowCount2 = getRowCountFromStreamingSql(tx, {sql: selectSql});
        await Promise.all([rowCount1, rowCount2]);
        await tx.commit();
      });
      await database.close();

      let request = spannerMock.getRequests().find(val => {
        return (val as v1.ExecuteSqlRequest).sql;
      }) as v1.ExecuteSqlRequest;
      assert.ok(request, 'no ExecuteSqlRequest found');
      assert.deepStrictEqual(request.transaction!.begin!.readWrite, {});
      assert.strictEqual(request.sql, selectSql);

      request = spannerMock
        .getRequests()
        .slice()
        .reverse()
        .find(val => {
          return (val as v1.ExecuteSqlRequest).sql;
        }) as v1.ExecuteSqlRequest;
      assert.ok(request, 'no ExecuteSqlRequest found');
      assert.strictEqual(request.sql, selectSql);
      assert.ok(request.transaction!.id, 'TransactionID is not set.');
      const beginTxnRequest = spannerMock.getRequests().find(val => {
        return (val as v1.BeginTransactionRequest).options?.readWrite;
      }) as v1.BeginTransactionRequest;
      assert.ok(!beginTxnRequest, 'beginTransaction was called');
    });

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right. Added a lock that prevents a multiple inline begin transaction at the same time. Thanks!

transaction.id = this.id as Uint8Array;
} else if (typeof this._options.readWrite !== 'undefined') {
} else if (this._options.readWrite) {
transaction.begin = this._options;
} else {
transaction.singleUse = this._options;
Expand Down Expand Up @@ -1269,16 +1269,18 @@ export class Snapshot extends EventEmitter {
private _wrapWithIdWaiter(
makeRequest: (resumeToken?: ResumeToken) => Readable):
(resumeToken?: ResumeToken) => Readable {
if (this.id || typeof this._options.readWrite === 'undefined') {
if (this.id || !this._options.readWrite) {
return makeRequest;
}
if (!this._inlineBeginStarted) {
this._inlineBeginStarted = true;
return makeRequest;
}
return (resumeToken?: ResumeToken): Readable =>
this._idWaiter.once('notify', () =>
this._idWaiter.wrap(makeRequest(resumeToken)));
this._idWaiter.once('notify', () => makeRequest(resumeToken)
.on('data', (chunk) => this._idWaiter.emit('data', chunk))
.once('end', () => this._idWaiter.emit('end'))
);
}
}

Expand Down