Feature: Enhanced handling of GDPR banner acceptance in CLI

[amovar18]

Description

This PR allows the user to pass a selector.json file with flag -b or --button-selectors which will be used when any custom selector is used for GDPR banner acceptance.
It also adds support for major CMP's like OneTrust, Ibuenda etc.

Relevant Technical Choices

• Use constant array for selector CSS.
• Use constant array for Text selector.
• Add a custom function for using selector.json provided by the user.

Testing Instructions

• Clone this branch.
• In the terminal run npm i && npm run build:cli:dashboard.
• Then run the CLI on site doctolib.fr using npm run cli https://doctolib.fr -- -b selector.json.
• Add the following in selector.json.

{
    "cssSelectors":["didomi-notice-agree-button"],
    "textSelectors": [],
    "xPath": []
}

Additional Information:

• If the user provided selectors aren't found in the page then the CLI falls back to using major selector, if that fails then we use the text selectors in the language.

Screenshot/Screencast

---

Checklist

• I have thoroughly tested this code to the best of my abilities.
• I have reviewed the code myself before requesting a review.
• This code is covered by unit tests to verify that it works as intended.
• The QA of this PR is done by a member of the QA team (to be checked by QA).

Fixes #810

[fellyph]

1. When the user provides a selector that doesn't exist, we should give an error message:

[fellyph]

2. When the JSON has invalid properties, the CLI should handle it and trigger a error message:

for example:

{
  "cssselectors": ["#didomi-notice-agree-button"],
  "textSelectors": [],
  "xpath": []
}

[fellyph]

3. When selecting an item with xPath an error is triggered during the audit:

{
  "cssSelectors": [],
  "textSelectors": [],
  "xPath": ["//*[@id=\"didomi-notice-agree-button\"]"]
}

[fellyph]

Three items listed were fixed on the latest commit:

• When the user provides a selector that doesn't exist, we should give an error message
• When the JSON has invalid properties, the CLI should handle it and trigger an error message
• When selecting an item with xPath an error is triggered during the audit

[fellyph]

When it runs a selector.json for a sitemap, the CLI can't find the URLs on the sitemap:

Steps to reproduce:

1. creates a json file

{
  "cssSelectors": ["#didomi-notice-agree-button"],
  "textSelectors": [],
  "xPath": []
}

2. run the command: npm run cli -- -s https://www.doctolib.fr/sitemap.xml -b selector.json

[amovar18]

@fellyph not facing this issue can you check with another sitemap please?
I debugged this got the error as 403 forbidden.

[pavanpatil1]

Hi @amovar18, Verified the PR, Observed few things added below:

• Below is a list of few sites that display a GDPR banner but have not yet accepted GDPR consent through the CLI. include these in the array.

• https://www.assurances.decathlon.fr/

• https://playstack.com/

---

• If the file is not provided, it currently only shows the file name. Instead, we could display the file path.

---

[amovar18]

@pavanpatil1 Added a commit for the 3rd feedback, but the first 2 cannot be worked on because in those sites the button is in having an anchor tag with button styling. This breaks the accessibility rules. Our tool finds GDPR banners with acceptance buttons, not anchor tags.

[fellyph]

@ amovar18 anchors are common; can we consider clicking on them? I think block elements like divs and other elements are a no-go.

[fellyph]

I have pulled the latest changes and am still getting a Zero URL counter

Command:
npm run cli -- -s https://www.doctolib.fr/sitemap.xml -b selector.json

Selector.json:

{
  "cssSelectors": ["#didomi-notice-agree-button"],
  "textSelectors": [],
  "xPath": []
}

[fellyph]

The issue with zero URLs is the bot prevention system:

It is not the scope for this task but we can open an issue to handle those cases

[amovar18]

@ amovar18 anchors are common; can we consider clicking on them? I think block elements like divs and other elements are a no-go.

They are but if the detection is off then it might redirect to another page and mess up the whole analysis.

[mohdsayed]

Looks good

[mohdsayed]

@amovar18 Please resolve conflicts and we are good to merge.

[amovar18]

@mohdsayed resolved conflicts.