Fix various issues #188
Merged
Fix various issues #188
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This avoids problems related to URL encoding. Obsoletes: hartator#116
This avoids the messages breaking JSON parsing when the output is being redirected to a file and parsed.
This avoids producing JSON that is not parsable.
|
Works great. Used this branch when I needed to use this project. 👍 Hope this can be merged soon to keep the original repo relevant. |
hartator
approved these changes
Jun 7, 2021
| @@ -42,7 +42,7 @@ It will download the last version of every file present on Wayback Machine to `. | |||
| -x, --exclude EXCLUDE_FILTER Skip downloading of urls that match this filter | |||
| (use // notation for the filter to be treated as a regex) | |||
| -a, --all Expand downloading to error files (40x and 50x) and redirections (30x) | |||
| -c, --concurrency NUMBER Number of multiple files to dowload at a time | |||
| -c, --concurrency NUMBER Number of multiple files to download at a time | |||
| @@ -62,7 +62,7 @@ Example: | |||
|
|
|||
| -s, --all-timestamps | |||
|
|
|||
| Optional. This option will download all timestamps/snapshots for a given website. It will uses the timepstamp of each snapshot as directory. | |||
| Optional. This option will download all timestamps/snapshots for a given website. It will uses the timestamp of each snapshot as directory. | |||
| @@ -169,7 +169,7 @@ Example: | |||
|
|
|||
| -c, --concurrency NUMBER | |||
|
|
|||
| Optional. Specify the number of multiple files you want to download at the same time. Allows to speed up the download of a website significantly. Default is to download one file at a time. | |||
| Optional. Specify the number of multiple files you want to download at the same time. Allows one to speed up the download of a website significantly. Default is to download one file at a time. | |||
There was a problem hiding this comment.
👍 Thank you for all the typo corrections.
| @@ -46,7 +46,7 @@ option_parser = OptionParser.new do |opts| | |||
| options[:all] = true | |||
| end | |||
|
|
|||
| opts.on("-c", "--concurrency NUMBER", Integer, "Number of multiple files to dowload at a time", "Default is one file at a time (ie. 20)") do |t| | |||
| opts.on("-c", "--concurrency NUMBER", Integer, "Number of multiple files to download at a time", "Default is one file at a time (ie. 20)") do |t| | |||
| file_timestamp = line[0..13].to_i | ||
| file_url = line[15..-2] | ||
| get_all_snapshots_to_consider.each do |file_timestamp, file_url| | ||
| next unless file_url.include?('/') |
| end | ||
| if page_index | ||
| parameters += "&page=#{page_index}" | ||
| parameters.push(["page", page_index]) |
| @@ -70,7 +70,7 @@ def tidy_bytes(force = false) | |||
| if is_unused || is_restricted | |||
| bytes[i] = tidy_byte(byte) | |||
| elsif is_cont | |||
| # Not expecting contination byte? Clean up. Otherwise, now expect one less. | |||
| # Not expecting continuation byte? Clean up. Otherwise, now expect one less. | |||
|
Excellent, thanks for that. I was thinking of packaging the project for
to Debian at some point. Would you be OK with that?
…--
bye,
pabs
https://bonedaddy.net/pabs3/
|
Sure, go for it. 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If nessecary I can split this up into multiple pull requests,
but then there will be some conflicts between them.