Group concurrent errors #105

armanio123 · 2023-03-14T23:38:02Z

Fixes #100

This changes how the error are displayed. Instead of showing an error per repository, it displays the "error message" as header and groups all of the repositories containing that error.

The group algorithm parses all of the stack lines containing "tsserver" and creates a hash based on that.

Important:

The header is now an error message and not the last stack frame. This might generate some noise when grouping if the error message contains dynamic content as file paths or names. Nonetheless, I'm not sure if we have that scenario.
Due to the app architecture is hard to sort the generated "summaries". That is because we generate files based on the hash instead of the error stack trace as it used.
The way we put the "old errors" on top of the summary is hacky. We use a ! character to make the sort to put it on top.

armanio123 · 2023-03-14T23:39:17Z

Example of a report:

@armanio123 Here are the results of running the user test suite comparing main and refs/pull/2/merge:

Unfortunately, something went wrong, but it probably wasn't caused by your change.

Details

⚠️ Old server errors ⚠️

Error: Old Error 1

Req #4 - getOutliningSpans
    at IOSession.getOutliningSpans (\typescript-main\built\local\tsserver.js:179250:13)
    at getOutliningSpans (\typescript-main\built\local\tsserver.js:177767:43)
    at \typescript-main\built\local\tsserver.js:180176:69
    at IOSession.executeWithRequestId (\typescript-main\built\local\tsserver.js:180168:14)
    at IOSession.executeCommand (\typescript-main\built\local\tsserver.js:180176:29)
    at IOSession.onMessage (\typescript-main\built\local\tsserver.js:180218:51)
    at Interface.<anonymous> (\typescript-main\built\local\tsserver.js:181762:14)

Repos no longer reporting the error

Error: Old Error 2

Req #4 - getOutliningSpans
    at IOSession.getOutliningSpans (\typescript-main\built\local\tsserver.js:179248:13)
    at getOutliningSpans (\typescript-main\built\local\tsserver.js:177767:43)
    at \typescript-main\built\local\tsserver.js:180176:69
    at IOSession.executeWithRequestId (\typescript-main\built\local\tsserver.js:180168:14)
    at IOSession.executeCommand (\typescript-main\built\local\tsserver.js:180176:29)
    at IOSession.onMessage (\typescript-main\built\local\tsserver.js:180218:51)
    at Interface.<anonymous> (\typescript-main\built\local\tsserver.js:181762:14)

Repos no longer reporting the error

Error: This is error 1

Req #5 - navto
    at IOSession.getNavigateToItems (\typescript-2\built\local\tsserver.js:179673:13)
    at navto (\typescript-2\built\local\tsserver.js:177937:43)
    at \typescript-2\built\local\tsserver.js:180179:69
    at IOSession.executeWithRequestId (\typescript-2\built\local\tsserver.js:180171:14)
    at IOSession.executeCommand (\typescript-2\built\local\tsserver.js:180179:29)
    at IOSession.onMessage (\typescript-2\built\local\tsserver.js:180221:51)
    at Interface.<anonymous> (\typescript-2\built\local\tsserver.js:181765:14)

Affected repos

acorn

Raw error text: out\acorn.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":false}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test acorn
Back in the initial folder, download out\acorn.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./acorn ./acorn.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

async

Raw error text: out\async.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test async
Back in the initial folder, download out\async.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./async ./async.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

clear-require

Raw error text: out\clear-require.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":false}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test clear-require
Back in the initial folder, download out\clear-require.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./clear-require ./clear-require.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

minimatch

Raw error text: out\minimatch.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test minimatch
Back in the initial folder, download out\minimatch.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./minimatch ./minimatch.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

url-search-params

Raw error text: out\url-search-params.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test url-search-params
Back in the initial folder, download out\url-search-params.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./url-search-params ./url-search-params.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

Error: Test error message.

Req #9 - completionInfo
    at IOSession.getCompletions (\typescript-2\built\local\tsserver.js:179400:13)
    at completionInfo (\typescript-2\built\local\tsserver.js:177831:43)
    at \typescript-2\built\local\tsserver.js:180179:69
    at IOSession.executeWithRequestId (\typescript-2\built\local\tsserver.js:180171:14)
    at IOSession.executeCommand (\typescript-2\built\local\tsserver.js:180179:29)
    at IOSession.onMessage (\typescript-2\built\local\tsserver.js:180221:51)
    at Interface.<anonymous> (\typescript-2\built\local\tsserver.js:181765:14)

Affected repos

npm

Raw error text: out\npm.rawError.txt in the artifact folder

Last few requests

{"seq":6,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/bin/npx-cli.js"}},"skipDestructiveCodeActions":false}}
{"seq":7,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/bin/npx-cli.js"}}
{"seq":8,"type":"request","command":"updateOpen","arguments":{"changedFiles":[{"fileName":"@PROJECT_ROOT@/bin/npx-cli.js","textChanges":[{"newText":" //comment","start":{"line":1,"offset":20},"end":{"line":1,"offset":20}}]}],"closedFiles":[],"openFiles":[]}}
{"seq":9,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/bin/npx-cli.js","line":36,"offset":17,"includeExternalModuleExports":false,"triggerKind":1}}

Repro steps

git clone https://github.com/npm/cli.git --recurse-submodules
In dir npm, run git reset --hard 434b461e4c15513817eaec6acfe82c7814789c85
Back in the initial folder, download out\npm.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./npm ./npm.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

TypeScript-React-Native-Starter

Raw error text: out\TypeScript-React-Native-Starter.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/ExampleProject/__tests__/App.tsx","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/ExampleProject/__tests__/App.tsx"}}
{"seq":4,"type":"request","command":"updateOpen","arguments":{"changedFiles":[{"fileName":"@PROJECT_ROOT@/ExampleProject/__tests__/App.tsx","textChanges":[{"newText":" //comment","start":{"line":1,"offset":23},"end":{"line":1,"offset":23}}]}],"closedFiles":[],"openFiles":[]}}
{"seq":5,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/ExampleProject/__tests__/App.tsx","line":5,"offset":59,"includeExternalModuleExports":false,"triggerKind":2,"triggerCharacter":"."}}

Repro steps

git clone https://github.com/Microsoft/TypeScript-React-Native-Starter.git --recurse-submodules
In dir TypeScript-React-Native-Starter, run git reset --hard 4395b5b4adf3bbc28d1f939ab44be4e6bc6d02ff
Back in the initial folder, download out\TypeScript-React-Native-Starter.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./TypeScript-React-Native-Starter ./TypeScript-React-Native-Starter.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

Error: This is error 0

Req #5 - navto
    at IOSession.getNavigateToItems (\typescript-2\built\local\tsserver.js:179671:13)
    at navto (\typescript-2\built\local\tsserver.js:177937:43)
    at \typescript-2\built\local\tsserver.js:180179:69
    at IOSession.executeWithRequestId (\typescript-2\built\local\tsserver.js:180171:14)
    at IOSession.executeCommand (\typescript-2\built\local\tsserver.js:180179:29)
    at IOSession.onMessage (\typescript-2\built\local\tsserver.js:180221:51)
    at Interface.<anonymous> (\typescript-2\built\local\tsserver.js:181765:14)

Affected repos

clone

Raw error text: out\clone.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":false}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test clone
Back in the initial folder, download out\clone.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./clone ./clone.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

content-disposition

Raw error text: out\content-disposition.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test content-disposition
Back in the initial folder, download out\content-disposition.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./content-disposition ./content-disposition.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

debug

Raw error text: out\debug.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":false}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test debug
Back in the initial folder, download out\debug.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./debug ./debug.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

enhanced-resolve

Raw error text: out\enhanced-resolve.rawError.txt in the artifact folder

Last few requests

{"seq":1,"type":"request","command":"configure","arguments":{"preferences":{"disableLineTextInReferences":true,"includePackageJsonAutoImports":"auto","includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"allowIncompleteCompletions":true,"includeCompletionsForModuleExports":false},"watchOptions":{"excludeDirectories":["**/node_modules"]}}}
{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":false}}
{"seq":4,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test enhanced-resolve
Back in the initial folder, download out\enhanced-resolve.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./enhanced-resolve ./enhanced-resolve.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

follow-redirects

Raw error text: out\follow-redirects.rawError.txt in the artifact folder

Last few requests

{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/index.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":5,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test follow-redirects
Back in the initial folder, download out\follow-redirects.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./follow-redirects ./follow-redirects.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

fp-ts

Raw error text: out\fp-ts.rawError.txt in the artifact folder

Last few requests

{"seq":1,"type":"request","command":"configure","arguments":{"preferences":{"disableLineTextInReferences":true,"includePackageJsonAutoImports":"auto","includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"allowIncompleteCompletions":true,"includeCompletionsForModuleExports":false},"watchOptions":{"excludeDirectories":["**/node_modules"]}}}
{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/dtslint/index.d.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"organizeImports","arguments":{"scope":{"type":"file","args":{"file":"@PROJECT_ROOT@/dtslint/index.d.ts"}},"skipDestructiveCodeActions":true}}
{"seq":4,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

git clone https://github.com/gcanti/fp-ts.git --recurse-submodules
In dir fp-ts, run git reset --hard 2ce18c53604ad772937f4d3281d3590a17d70a95
Back in the initial folder, download out\fp-ts.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./fp-ts ./fp-ts.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

util

Raw error text: out\util.rawError.txt in the artifact folder

Last few requests

{"seq":1,"type":"request","command":"configure","arguments":{"preferences":{"disableLineTextInReferences":true,"includePackageJsonAutoImports":"auto","includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"allowIncompleteCompletions":true,"includeCompletionsForModuleExports":false},"watchOptions":{"excludeDirectories":["**/node_modules"]}}}
{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/index.ts","projectRootPath":"@PROJECT_ROOT@"}]}}
{"seq":3,"type":"request","command":"getOutliningSpans","arguments":{"file":"@PROJECT_ROOT@/index.ts"}}
{"seq":4,"type":"request","command":"navto","arguments":{"searchValue":"a","maxResultCount":256}}

Repro steps

Download user test util
Back in the initial folder, download out\util.replay.txt from the artifact folder
npm install --no-save @typescript/server-replay
npx tsreplay ./util ./util.replay.txt path/to/tsserver.js
npx tsreplay --help to learn about helpful switches for debugging, logging, etc

src/main.ts

sandersn

Some minor comments, plus I agree with Daniel's question about equality of stdout.

Overall, though,

I'm suspicious that more de-duping will be needed.
I think it's worthwhile to ship this and have an example of the needed de-duping.

sandersn · 2023-03-15T16:02:38Z

src/utils/hashStackTrace.ts

@@ -0,0 +1,32 @@
+import { createHash } from "crypto";


just a comment for later: createHash is one of the functions from crypto that causes a security warning to be issued on a repo. We'll have to disable the warning since it's not used for secure purposes, but for identity purposes.

src/utils/hashStackTrace.ts

src/main.ts

Co-authored-by: Nathan Shively-Sanders <293473+sandersn@users.noreply.github.com>

src/utils/hashStackTrace.ts

src/postGithubComments.ts

src/main.ts

DanielRosenwasser

Approved, but I'd appreciate if you just made some of the minor simplifications I suggested.

Co-authored-by: Daniel Rosenwasser <DanielRosenwasser@users.noreply.github.com>

DanielRosenwasser · 2023-03-24T16:30:12Z

Last run of server tests had an issue:

Error: ENOENT: no such file or directory, open '/mnt/ts_downloads/apache.echarts.replay.txt'
at Object.openSync (node:fs:590:3)
at Object.readFileSync (node:fs:458:35)
at createNewErrorSummaryAsync (/home/vsts/work/1/s/dist/main.js:288:6)
at mainAsync (/home/vsts/work/1/s/dist/main.js:600:35)
at runMicrotasks ()

Let me try running off of a revert.

armanio123 · 2023-04-13T04:28:01Z

I have submitted a new PR for returning this feature. It also fixes the bug introduced.

@DanielRosenwasser @sandersn @jakebailey I appreciate if I can get a review.

Armando Aguirre added 2 commits March 13, 2023 15:17

Group errors. First iteration.

b00a2a0

Added old server errors group

72c6439

armanio123 requested review from sandersn, DanielRosenwasser and jakebailey March 14, 2023 23:38

DanielRosenwasser reviewed Mar 15, 2023

View reviewed changes

src/main.ts Outdated Show resolved Hide resolved

DanielRosenwasser reviewed Mar 15, 2023

View reviewed changes

src/main.ts Outdated Show resolved Hide resolved

sandersn approved these changes Mar 15, 2023

View reviewed changes

Armando Aguirre and others added 2 commits March 15, 2023 16:41

PR comments and fixes

82c05c9

Update src/utils/hashStackTrace.ts

9efb8e2

Co-authored-by: Nathan Shively-Sanders <293473+sandersn@users.noreply.github.com>

armanio123 requested a review from DanielRosenwasser March 15, 2023 23:45

Fix build

5b4cd68