Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Integration Test to scan the top 1k domains and validate that ZDNS's result is correct #370

Merged
merged 25 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
92412fd
testing if github runner can scan 1k domains without crazy timeouts
phillip-stephens May 24, 2024
bb83377
seeing results
phillip-stephens May 24, 2024
75e5fdc
able to request X domain to Y IP
phillip-stephens May 24, 2024
ab3c87c
working a and alookup tests
phillip-stephens May 24, 2024
2a789ce
working validators
phillip-stephens May 28, 2024
dec7f03
cleaning up unneeded files and added some logs
phillip-stephens May 28, 2024
f48992e
cleanup and add as github action
phillip-stephens May 28, 2024
202d766
updated zdns exe path
phillip-stephens May 28, 2024
f706612
updated test name in ci.yml
phillip-stephens May 28, 2024
48a16c6
made python test a script
phillip-stephens May 28, 2024
2eeeae9
updated script permissions
phillip-stephens May 28, 2024
1855325
removed dead import
phillip-stephens May 28, 2024
f5cc9e3
updated file name
phillip-stephens May 28, 2024
e0ca48b
made error msg less scary
phillip-stephens May 28, 2024
fb07c87
only run CI on commits to main branch and all PR's, otherwise we're d…
phillip-stephens May 28, 2024
b7005b1
remove limit to top 100, run against all 1k domains
phillip-stephens May 28, 2024
e9e7ee2
moved to only scanning 100 domains
phillip-stephens May 28, 2024
581fa70
cleanup
phillip-stephens May 28, 2024
4b04b9f
fixed name of CI action
phillip-stephens May 28, 2024
fc889fd
added rest of domains
phillip-stephens May 28, 2024
1b87244
moved to 500 domains
phillip-stephens May 28, 2024
ac820d1
moved back to 100 domains
phillip-stephens May 28, 2024
6399896
Merge branch 'main' into phillip/large-scale-integration
phillip-stephens May 28, 2024
bed56ec
add comments explaining integration tests
phillip-stephens May 28, 2024
cdb6659
Merge branch 'phillip/large-scale-integration' of github.com:zmap/zdn…
phillip-stephens May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup and add as github action
  • Loading branch information
phillip-stephens committed May 28, 2024
commit f48992e5ff6fb2ea91ab860eaa77c98dad75f792
17 changes: 15 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,21 @@ jobs:
sudo rm /etc/resolv.conf
sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
python --version
./integration_tests.py
./testing/integration_tests.py
build-and-test-top-1k:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.20'
- name: Build
run: |
go version
make
- name: Large-Scale Integration Test
run: |
python --version
cat largescaleintegration/top-1k.csv | ./zdns A --iterative
./testing/large_scale_integration_test.py
65 changes: 0 additions & 65 deletions largescaleintegration/large_scan_validator.py

This file was deleted.

2 changes: 1 addition & 1 deletion integration_tests.py → testing/integration_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def dictSort(d):
class Tests(unittest.TestCase):

maxDiff = None
ZDNS_EXECUTABLE = "./zdns"
ZDNS_EXECUTABLE = "../zdns"

def run_zdns_check_failure(self, flags, name, expected_err, executable=ZDNS_EXECUTABLE):
flags = flags + " --threads=10"
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import unittest
import json

ZDNS_EXECUTABLE = "./zdns"
ZDNS_EXECUTABLE = "../zdns"

def can_request_successfully(domain: str, ip: str) -> bool:
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15"
Expand Down
Loading