The web crawler
python crawler/main.py [URL]pip install -r requirements.txtScreenshots are saved in the screenshots directory.
The log is saved in the crawl_log.md file.
- Add AI decision making
- Add more tests
- Add more error handling
-
Couldn't load subscription status.
- Fork 0
knowledge-graph-base/web-crawler
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
The web crawler
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published