Skip to content

Commit

Permalink
fetch & fetchpost: --http-header -H shortcut (dathere#596)
Browse files Browse the repository at this point in the history
* cargo update bump reqwest from 0.11.12 to 0.11.13

* `fetch` & `fetchpost`: create --http-header -H shortcut option

also move report column descriptions from usage text to source code as the usage text was overly verbose

* update test for --http-header -H shortcut

* rustfmt
  • Loading branch information
jqnatividad authored Nov 17, 2022
1 parent 79cce33 commit b9aa8f8
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 16 deletions.
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 5 additions & 4 deletions src/cmd/fetch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ The --http-header option allows you to append arbitrary key value pairs (a valid
to the HTTP header (to authenticate against an API, pass custom header fields, etc.). Note that you can
pass as many key-value pairs by using --http-header option repeatedly. For example:
$ qsv fetch URL data.csv --http-header "X-Api-Key:TEST_KEY" --http-header "X-Api-Secret:ABC123XYZ" --http-header "Accept-Language: fr-FR"
$ qsv fetch URL data.csv --http-header "X-Api-Key:TEST_KEY" -H "X-Api-Secret:ABC123XYZ" -H "Accept-Language: fr-FR"
For more extensive examples, see https://github.com/jqnatividad/qsv/blob/master/tests/test_fetch.rs.
Expand Down Expand Up @@ -124,7 +124,7 @@ Fetch options:
[default: 0 ]
--timeout <seconds> Timeout for each URL request.
[default: 15 ]
--http-header <key:value> Append custom header(s) to the HTTP header. Pass multiple key-value pairs
-H, --http-header <k:v> Append custom header(s) to the HTTP header. Pass multiple key-value pairs
by adding this option multiple times, once for each pair. The key and value
should be separated by a colon.
--max-retries <count> Maximum number of retries per record before an error is raised.
Expand All @@ -145,8 +145,6 @@ Fetch options:
report has the same columns as the input CSV with six additional columns -
qsv_fetch_url, qsv_fetch_status, qsv_fetch_cache_hit, qsv_fetch_retries,
qsv_fetch_elapsed_ms & qsv_fetch_response.
fetch_url - URL used, fetch_status - HTTP code, fetch_cache_hit - cache hit flag,
fetch_retries - retry attempts, fetch_elapsed - elapsed time & fetch_response.
The short report only has the six columns without the "qsv_fetch_" prefix.
[default: none]
--redis Use Redis to cache responses. It connects to "redis://127.0.0.1:6379/1"
Expand Down Expand Up @@ -543,6 +541,9 @@ pub fn run(argv: &[&str]) -> CliResult<()> {
} else {
""
};
// the fetch report has the following columns:
// url - URL used, status - HTTP status code, cache_hit - cache hit flag,
// retries - retry attempts, elapsed - elapsed time (milliseconds) & response.
report_headers.push_field(format!("{rptcol_prefix}url").as_bytes());
report_headers.push_field(format!("{rptcol_prefix}status").as_bytes());
report_headers.push_field(format!("{rptcol_prefix}cache_hit").as_bytes());
Expand Down
11 changes: 6 additions & 5 deletions src/cmd/fetchpost.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ The --http-header option allows you to append arbitrary key value pairs (a valid
to the HTTP header (to authenticate against an API, pass custom header fields, etc.). Note that you can
pass as many key-value pairs by using --http-header option repeatedly. For example:
$ qsv fetchpost https://httpbin.org/post col1-col3 data.csv --http-header "X-Api-Key:TEST_KEY" --http-header "X-Api-Secret:ABC123XYZ"
$ qsv fetchpost https://httpbin.org/post col1-col3 data.csv -H "X-Api-Key:TEST_KEY" -H "X-Api-Secret:ABC123XYZ"
For more extensive examples, see https://github.com/jqnatividad/qsv/blob/master/tests/test_fetch.rs.
Expand Down Expand Up @@ -105,7 +105,7 @@ Fetch options:
[default: 0 ]
--timeout <seconds> Timeout for each URL request.
[default: 15 ]
--http-header <key:value> Append custom header(s) to the HTTP header. Pass multiple key-value pairs
-H, --http-header <k:v> Append custom header(s) to the HTTP header. Pass multiple key-value pairs
by adding this option multiple times, once for each pair. The key and value
should be separated by a colon.
--max-retries <count> Maximum number of retries per record before an error is raised.
Expand All @@ -126,9 +126,6 @@ Fetch options:
report has the same columns as the input CSV with seven additional columns -
qsv_fetchp_url, qsv_fetchp_form, qsv_fetchp_status, qsv_fetchp_cache_hit,
qsv_fetchp_retries, qsv_fetchp_elapsed_ms & qsv_fetchp_response.
fetchp_url - URL used, qsv_fetchp_form - form data sent, fetchp_status - HTTP code,
fetchp_cache_hit - cached hit flag, fetchp_retries - retry attempts,
fetchp_elapsed - elapsed time & fetchp_response.
The short report only has the seven columns without the "qsv_fetchp_" prefix.
[default: none]
--redis Use Redis to cache responses. It connects to "redis://127.0.0.1:6379/2"
Expand Down Expand Up @@ -511,6 +508,10 @@ pub fn run(argv: &[&str]) -> CliResult<()> {
} else {
""
};
// the fetchpost report has the following columns:
// url - URL used, form - form data sent, status - HTTP status code,
// cache_hit - cache hit flag, retries - retry attempts,
// elapsed - elapsed time (milliseconds) & response.
report_headers.push_field(format!("{rptcol_prefix}url").as_bytes());
report_headers.push_field(format!("{rptcol_prefix}form").as_bytes());
report_headers.push_field(format!("{rptcol_prefix}status").as_bytes());
Expand Down
6 changes: 3 additions & 3 deletions tests/test_fetch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -356,9 +356,9 @@ fn fetch_custom_header() {
);
let mut cmd = wrk.command("fetch");
cmd.arg("URL")
.arg("--http-header")
.arg("-H")
.arg(" X-Api-Key : DEMO_KEY")
.arg("--http-header")
.arg("-H")
.arg("X-Api-Secret :ABC123XYZ")
.arg("--jql")
.arg(r#""headers"."X-Api-Key","headers"."X-Api-Secret""#)
Expand Down Expand Up @@ -460,7 +460,7 @@ fn fetchpost_custom_invalid_header_error() {
let mut cmd = wrk.command("fetchpost");
cmd.arg("URL")
.arg("bool_col,col1,number col")
.arg("--http-header")
.arg("-H")
.arg("X-Api-\tSecret :ABC123XYZ") // non-visible ascii not valid
.arg("data.csv");

Expand Down

0 comments on commit b9aa8f8

Please sign in to comment.