Deduplicate existing findings method

[tvims81]

Is your feature request related to a problem? Please describe
We would like to have a feature for deduplication of existing vulnerabilities. For instance, in the System Settings, the "Deduplicate Findings" feature was initially disabled, and I uploaded test reports to DefectDojo. Later, the "Deduplicate Findings" setting was enabled. While the deduplication mechanism works for new reports, existing reports containing duplicates cannot be reprocessed.

Describe the solution you'd like
As a user, I want the ability to deduplicate existing vulnerabilities without re-uploading the report. It might be possible to achieve this through the reimport mechanism, but by utilizing the already existing report within the Test.

Describe alternatives you've considered
As an alternative, all Tests with duplicates, and all Engagements with those Tests, could be deleted. However, this process is time-consuming if there are many such cases. Additionally, reports could be manually reimported, but this may be problematic when CI/CD report import is configured, as the reports might not have been saved.

Additional context
I had to delete the Test data directly from the DefectDojo database, as there is no bulk delete functionality for Tests or Engagements via the DefectDojo API. There may be an easier way to analyze existing vulnerabilities for duplicates. Please let me know if such a method exists.

[valentijnscholten]

Did you also try the global System Settings (you need both) that deletes duplicates across the whole portfolio?

If you set the maximum number of duplicates to 0, it will delete all of them (So be careful).

[tvims81]

@valentijnscholten Thanks for reply!

Yes, I tried it following your suggestion, but the "Delete duplicates + Max duplicates" setting only works for newly uploaded reports. For already existing tests (and vulnerabilities), this approach does not work.

[valentijnscholten]

The delete duplicates job works across all findings, including previously created findings.
The deduplicate findings feature only works on future findings or edited findings after being enabled. It sounds like this applies to you?
You can manually trigger dedupe for existing findings via the dedupe management command:

docker compose exec uwsgi /bin/bash -c "python manage.py migrate"

This is assuming you're using the Docker Compose deployment option.

More info:

• https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/management/commands/dedupe.py
• https://docs.defectdojo.com/en/open_source/upgrading/upgrading_guide/
• https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/DOCKER.md