E2E test - calendar-body.test.js

[htdat]

Expected/Desired Behavior

No error for E2E tests on described on this file tests/e2e/specs/calendar-body.test.js

Actual Behavior

For a while, this file did not create any failed test, then it keeps yelling the same error.

This can be seen here https://github.com/Automattic/Edit-Flow/actions/workflows/e2e-and-js-tests.yml
and this screenshot.

Errors

From https://github.com/Automattic/Edit-Flow/runs/2715084649?check_suite_focus=true:

FAIL tests/e2e/specs/calendar-body.test.js (22.893s)
  ● Calendar Body › expects a scheduled post can be dragged and dropped

    expect(received).not.toContain(expected) // indexOf

    Expected value: not "Scheduled Post"
    Received array:     ["Scheduled Post", "Published Post"]

      94 |         expect(await scheduledPostDay.evaluate((node) => {
      95 |             return Array.prototype.slice.call(node.querySelectorAll('.item-headline.post-title strong')).map((n) => n.innerText)
    > 96 |         })).not.toContain('Scheduled Post');
         |                 ^
      97 |     });
      98 | });

      at Object._callee3$ (specs/calendar-body.test.js:96:17)
      at asyncGeneratorStep (specs/calendar-body.test.js:7:103)
      at _next (specs/calendar-body.test.js:9:194)
          at runMicrotasks (<anonymous>)

This is the result of E2E tests on my local machine:

  ● Calendar Body › expects a scheduled post can be dragged and dropped

    expect(received).not.toContain(expected) // indexOf

    Expected value: not "Scheduled Post"
    Received array:     ["Scheduled Post", "Published Post", "Unpublished Post"]

      94 |         expect(await scheduledPostDay.evaluate((node) => {
      95 |             return Array.prototype.slice.call(node.querySelectorAll('.item-headline.post-title strong')).map((n) => n.innerText)
    > 96 |         })).not.toContain('Scheduled Post');
         |                 ^
      97 |     });
      98 | });

      at _callee3$ (specs/calendar-body.test.js:96:17)
      at asyncGeneratorStep (specs/calendar-body.test.js:7:103)
      at _next (specs/calendar-body.test.js:9:194)

 PASS  tests/e2e/specs/editorial-comments.test.js (5.575s)
 PASS  tests/e2e/specs/calendar-header.test.js
 PASS  tests/e2e/specs/simple.test.js

(Optional) Additional notes

[htdat]

FAIL tests/e2e/specs/calendar-body.test.js (22.893s)
● Calendar Body › expects a scheduled post can be dragged and dropped

tl;dr: The main reason is that our schedulePost() function is not working as expected for all months. See more below!

Troubleshooting

I am using this debugging code to take screenshots. Basically, it takes advantage of Puppeteer's screenshot.

Click to see the diff

Index: tests/e2e/utils/index.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/tests/e2e/utils/index.js b/tests/e2e/utils/index.js
--- a/tests/e2e/utils/index.js	(revision 6ade7349426e439e255cb961b4cea167840446d6)
+++ b/tests/e2e/utils/index.js	(date 1623228378856)
@@ -58,11 +58,14 @@
 
     // wait for popout animation
     await page.waitFor(200);
+    await page.screenshot( { path: '/tmp/schedulePost_before-choosing-date.png'} );
 
     await page.click( 'div[aria-label="Move forward to switch to the next month."]' );
 
     await page.click( '.CalendarDay_1' );
 
+    await page.screenshot( { path: '/tmp/schedulePost_after-choosing-date.png'} );
+
     await publishPost();
 
 }
Index: tests/e2e/specs/calendar-body.test.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/tests/e2e/specs/calendar-body.test.js b/tests/e2e/specs/calendar-body.test.js
--- a/tests/e2e/specs/calendar-body.test.js	(revision 6ade7349426e439e255cb961b4cea167840446d6)
+++ b/tests/e2e/specs/calendar-body.test.js	(date 1623225160182)
@@ -71,6 +71,7 @@
         await visitAdminPage("index.php", "page=calendar");
 
         await page.waitForSelector('.ef-calendar-header');
+        await page.screenshot( { path: '/tmp/calendar_before-drag-drop.png'} );
 
         const scheduledPost = (await page.$x('//strong[text()="Scheduled Post"]'))[0];
         const scheduledPostParent = await scheduledPost.evaluateHandle((node) => node.closest('.day-item'));
@@ -91,6 +92,8 @@
 
         await page.waitFor(200);
 
+        await page.screenshot( { path: '/tmp/calendar_after-drag-drop.png'} );
+
         expect(await scheduledPostDay.evaluate((node) => {
             return Array.prototype.slice.call(node.querySelectorAll('.item-headline.post-title strong')).map((n) => n.innerText)
         })).not.toContain('Scheduled Post');

Screenshots captured by the code are below. You can see the date and time of the post. It's expected to someday in the future but it's always today June 09, 2021.

• schedulePost_before-choosing-date
• schedulePost_after-choosing-date
• calendar_before-drag-drop
• calendar_after-drag-drop

The fun part is that it does not work for the current month June 2021, however, it works for May 2021.

Here is the recording video for this whole process - https://d.pr/v/EZeddZ
I could do this by adding --puppeteer-interactive to test-e2e in file package.json:

    "test-e2e": "wp-scripts test-e2e --config tests/e2e/jest.config.js --puppeteer-interactive",

Root Cause

Code
http://github.com/Automattic/Edit-Flow/blob/6ade7349426e439e255cb961b4cea167840446d6/tests/e2e/utils/index.js#L62-L64

As I understand this code means:

• Click the right arrow to choose a date for the next month.
• Select the first day of this month.

However, the selector for the first day of the next month is not always expected.