StepByStepReport plugin fails on BeforeSuite

[damian-zygmuntowicz-siili]

What are you trying to achieve?

Run test with step by step plugin enabled

What do you get instead?

Error

Provide console output if related. Use --verbose mode for more details.

Error processing test.failed event:
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
    at validateString (internal/validators.js:107:11)
    at Object.join (path.js:1037:7)
    at persist /node_modules/codeceptjs/lib/plugin/stepByStepReport.js:192:24)
    at EventEmitter.<anonymous> (/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:112:5)
    at EventEmitter.emit (events.js:201:15)
    at EventEmitter.emit (domain.js:471:20)
    at Object.emit (/node_modules/codeceptjs/lib/event.js:143:28)
    at errHandler (/node_modules/codeceptjs/lib/scenario.js:114:13)
    at /node_modules/codeceptjs/lib/scenario.js:121:7
    at /node_modules/codeceptjs/lib/recorder.js:229:9

Provide test source code if related
in codeceptjs/lib/plugin/stepByStepReport.js line 140 -> step number is undefined

Details

• CodeceptJS version: 3.0.4
• NodeJS Version: 12.16.1
• Operating System: macOS

[shpyrkova]

StepByStepReport also generates 'record_' folder with screenshot for each passed test, that has BeforeSuite hook. Even when "deleteSuccessful": true.
@damian-zygmuntowicz-siili did you find desicion?(

@DavertMik that's so sad :(

[Naimadnap]

@shpyrkova what do you mean?

[shpyrkova]

@Naimadnap I have the same issue.
stepByStepReport plugin is very helpful, but this problem is inconvenient to use it.

[mirao]

Have the same issue in CodeceptJS 3.5.12, Ubuntu 22.04

    I get random string "ATPackage"
Error processing test.failed event:
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at new NodeError (node:internal/errors:405:5)
    at validateString (node:internal/validators:162:11)
    at Object.join (node:path:1171:7)
    at persist (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:201:24)
    at EventEmitter.<anonymous> (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:115:5)
    at EventEmitter.emit (node:events:529:35)
    at EventEmitter.emit (node:domain:489:12)
    at Object.emit (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/event.js:149:28)
    at errHandler (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/scenario.js:115:13)
    at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/scenario.js:162:11
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/s4w/output/EditTextIn_1707724682.failed.png
  ✖ FAILED in 122ms

...

1) EditTextInWidgetTemplate
       "before all" hook: BeforeSuite for "Login":
     Cannot read properties of undefined (reading 'toString')
      at persistStep (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:144:63)
      at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.12_6zevq3paj6aplkar2h6tolhzt4_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:105:59
      at runNextTicks (node:internal/process/task_queues:60:5)
  
  Scenario Steps:
  - I.getRandomString("ATPackage") at Context.<anonymous> (./EditTextInWidgetTemplate_test.ts:31:27)

[mirao]

I've just tried it with CodeceptJS 3.6.2 (where it should be fixed), but I'm still getting an error

$ codeceptjs run Login_test.ts  --profile personal --verbose
***************************************
nodeInfo:  18.20.3
osInfo:  Linux 6.5 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
cpuInfo:  (16) x64 AMD Ryzen 9 7940HS w/ Radeon 780M Graphics
chromeInfo:  125.0.6422.141
edgeInfo:  "N/A"
firefoxInfo:  undefined
safariInfo:  N/A
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v3.6.2 #StandWithUkraine
Using test root "/home/mirao/workspace/codeceptjs/tests/s4h-web"
Helpers: Common, Playwright, REST, JSONResponse, Mochawesome, ChaiWrapper, CmdHelper, DbHelper, S4H_Web, S4h_server, SPORK_Server, MySQL_DB
Plugins: screenshotOnFail, retryFailedStep, stepByStepReport, tryTo, customLocator

Login --
    [1]  Starting recording promises
    Timeouts: 600
 › [Session] Starting singleton browser session
 › Open login screen
 › [Browser:Info] %cDownload the React DevTools for a better development experience: https://reactjs.org/link/react-devtools font-weight:bold%cDownload the React DevTools for a better development experience: https://reactjs.org/link/react-devtools font-weight:bold
    --- STARTED "before all" hook: BeforeSuite for "Verify login screen @loginScreen" ---
    I get random string "ATProfile"
    [1]  Error (Non-Terminated) | TypeError: Cannot read properties of undefined (reading 'toString') | (err) => { step.status = 'failed'; step.endTime = ...
    [1] Error | TypeError: Cannot read properties of undefined (reading 'toString') (e) => { const err = (recorder.getAsyncErr() === n...
    [1] <teardown>  Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/s4h-web/output/Login_befo_1717674506.failed.png
  ✖ FAILED in 121ms

    --- STARTED "after all" hook: AfterSuite for "Login with an invalid user account" ---
    [2]  Starting recording promises
    I delete user 
    › Deletion of user failed: TypeError: Cannot read properties of undefined (reading 'toString')
    at QuoteString (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/quoteString.js:16:21)
    at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/sqlObject.js:24:17
    at Array.map (<anonymous>)
    at SqlObject.map (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/sqlObject.js:23:14)
    at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/statement.js:88:46
    at new Promise (<anonymous>)
    at PreparedStatement.execute (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/statement.js:85:16)
    at DbHelper.run (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs-dbhelper@1.2.2/node_modules/codeceptjs-dbhelper/index.js:114:21)
    at S4H_server.deleteUser (/home/mirao/workspace/codeceptjs/tests/s4h-web/s4h_server_helper.ts:92:33)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    [2]  Error (Non-Terminated) | TypeError: Cannot read properties of undefined (reading 'toString') | (err) => { step.status = 'failed'; step.endTime = ...
    [2]  Error (Non-Terminated) | TypeError: Cannot read properties of undefined (reading 'toString') | (err) => { step.status = 'failed'; step.endTime = ...
    [2]  Error (Non-Terminated) | TypeError: Cannot read properties of undefined (reading 'toString') | (err) => { step.status = 'failed'; step.endTime = ...
    [2] Error | TypeError: Cannot read properties of undefined (reading 'toString') (e) => { const err = (recorder.getAsyncErr() === n...
Error processing test.failed event:
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at new NodeError (node:internal/errors:405:5)
    at validateString (node:internal/validators:162:11)
    at Object.join (node:path:1171:7)
    at persist (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:203:24)
    at EventEmitter.<anonymous> (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:117:5)
    at EventEmitter.emit (node:events:529:35)
    at EventEmitter.emit (node:domain:489:12)
    at Object.emit (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/event.js:149:28)
    at errHandler (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/scenario.js:115:13)
    at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/scenario.js:162:11
    [2] <teardown>  Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/s4h-web/output/Login_afte_1717674506.failed.png
  ✖ FAILED in 218ms

  S Login with an empty username
  S Login with an empty password
  S Login with a correct username and an incorrect password
  S Login with an incorrect username and a correct password
  S Login with a correct username and a correct password
  S Create a profile
  S User (not a parent or a child) is redirected to view selection screen when accessing root route
  S Select parent view
  S Parent is redirected to profiles when accessing root route
  S Switch to child view
  S Child is redirected to lesson when accessing root route
  S Refresh page to later verify cookies are removed correctly after logging out
  S Move to Account tab from student view
  S Verify logout
  S Verify you are still logged out after reload of page
  S Verify you are still logged out after opening of a root URL

-- FAILURES:

  1) Login
       "before all" hook: BeforeSuite for "Verify login screen @loginScreen":
     Cannot read properties of undefined (reading 'toString')
      at persistStep (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:146:63)
      at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.6.2_@babel+core@7.24.7_@babel+preset-env@7.24.7_react@18.2.0_typescript@4.8.2/node_modules/codeceptjs/lib/plugin/stepByStepReport.js:105:59
  
  Scenario Steps:
  - I.getRandomString("ATProfile") at Context.<anonymous> (./Login_test.ts:38:31)
  

  2) Login
       "after all" hook: AfterSuite for "Login with an invalid user account":
     Cannot read properties of undefined (reading 'toString')
      at QuoteString (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/quoteString.js:16:21)
      at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/sqlObject.js:24:17
      at Array.map (<anonymous>)
      at SqlObject.map (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/sqlObject.js:23:14)
      at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/statement.js:88:46
      at new Promise (<anonymous>)
      at PreparedStatement.execute (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/database-js@3.0.11/node_modules/database-js/lib/statement.js:85:16)
      at DbHelper.run (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs-dbhelper@1.2.2/node_modules/codeceptjs-dbhelper/index.js:114:21)
      at S4H_server.deleteUser (s4h_server_helper.ts:92:33)
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
  
  Scenario Steps:
  - I.deleteUser() at Context.<anonymous> (./Login_test.ts:166:11)
  


  FAIL  | 0 passed, 2 failed, 16 skipped   // 2s
  -

If the test doesn't contain BeforeSuite() nor AfterSuite(), it works well.
Anybody else?

[mirao]

It seems to be fixed in 3.6.3: #4385
However we can't use the plugin because screenshots are not synchronized with steps, see #4405