- Node 18
- yt-dlp (https://github.com/yt-dlp/yt-dlp)
npm i -g tcdown
npx tcdown
- pull the project
- install dependencies
- run next command:
node lib/cli.js
if there are issues with cloudflare run with '-h' or headless option so solve captcha manually
node lib/cli.js -h no
Usage
$ tcdown [CourseUrl]
Options
--all, -a Get all courses from particular school or provider.
--login, -l Your login url with login form.
--email, -e Your email.
--password, -p Your password.
--directory, -d Directory to save.
--file, -f Location of the file where are the courses
--headless, -h Enable headless (values: 'yes' or 'no'), default value is 'yes'
--html, -t Enable html download (values: 'yes' or 'no'), default value is 'yes'
--screenshot, -s Enable screenshot (values: 'yes' or 'no'), default value is 'yes'
--concurrency, -c
Examples
$ tcdown
$ tcdown -a
$ [DEBUG=scraper*] tcdown [url] [-l url...] [-e user@gmail.com] [-p password] [-d dirname] [-f path-to-file] [-h yes/no] [-t yes/no] [-s yes/no] [-c number]
This module uses debug to log events. To enable logs you should use environment variable DEBUG
.
Next command will log everything from scraper
export DEBUG=scraper*; tcdown
Module has different loggers for levels: scraper:error
, scraper:warn
, scraper:info
, scraper:debug
, scraper:log
. Please read debug documentation to find how to include/exclude specific loggers.
MIT
If you require assistance with an existing scraper/crawler, need to develop a new one, or have any other programming-related inquiries, please don't hesitate to reach out to me: muhamed.didovic@gmail.com