Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): add support for stdin based tools #755

Merged
merged 3 commits into from
Feb 26, 2025
Merged

feat(cli): add support for stdin based tools #755

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f63a99e
feat(cli): add support for stdin based tools
hougesen Feb 26, 2025
fe06ef1
feat(tools): add support for xq
hougesen Feb 26, 2025
0fbf1b6
feat(tools): add support for jqfmt
hougesen Feb 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ jobs:
run: ( ( which sqlfluff ) || ( ( which pipx && pipx install sqlfluff ) ) || ( echo "Unable to install tool" ) )
- name: swiftformat
run: ( ( which swiftformat ) || ( ( which brew && brew install swiftformat ) ) || ( echo "Unable to install tool" ) )
- name: jqfmt
run: ( ( which jqfmt ) || ( ( which go && go install github.com/noperator/jqfmt/cmd/jqfmt@latest ) ) || ( echo "Unable to install tool" ) )
- name: auto-optional
run: ( ( which auto-optional ) || ( ( which pipx && pipx install auto-optional ) ) || ( echo "Unable to install tool" ) )
- name: taplo
Expand All @@ -164,6 +166,8 @@ jobs:
run: ( ( which rescript ) || ( ( which npm && npm i -g rescript ) ) || ( echo "Unable to install tool" ) )
- name: yapf
run: ( ( which yapf ) || ( ( which pipx && pipx install yapf ) ) || ( echo "Unable to install tool" ) )
- name: jq
run: ( ( which jq ) || ( ( which apt-get && sudo apt-get install -y jq ) ) || ( echo "Unable to install tool" ) )
- name: autopep8
run: ( ( which autopep8 ) || ( ( which pipx && pipx install autopep8 ) ) || ( echo "Unable to install tool" ) )
- name: usort
Expand Down Expand Up @@ -196,6 +200,8 @@ jobs:
run: ( ( which stylefmt ) || ( ( which npm && npm i -g stylefmt ) ) || ( echo "Unable to install tool" ) )
- name: rubocop
run: ( ( which rubocop ) || ( ( which gem && gem install rubocop ) ) || ( echo "Unable to install tool" ) )
- name: xq
run: ( ( which xq ) || ( ( which brew && brew install xq ) || ( which apt-get && sudo apt-get install -y xq ) ) || ( echo "Unable to install tool" ) )
- name: htmlbeautifier
run: ( ( which htmlbeautifier ) || ( ( which gem && gem install htmlbeautifier ) ) || ( echo "Unable to install tool" ) )
- name: Run tests
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).

#### [Unreleased](https://github.com/hougesen/mdsf/compare/v0.5.3...HEAD)

- feat(cli): add support for stdin based tools [`#755`](https://github.com/hougesen/mdsf/pull/755)
- refactor(cli): error if no input is provided [`#754`](https://github.com/hougesen/mdsf/pull/754)
- docs: conda install script [`#753`](https://github.com/hougesen/mdsf/pull/753)
- docs: add treefmt usage example [`#752`](https://github.com/hougesen/mdsf/pull/752)
Expand Down
13 changes: 11 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ mdsf init

<!-- START_SECTION:supported-tools -->

`mdsf` currently supports 267 tools. Feel free to open an issue/pull-request if your favorite tool/command is missing! 😃
`mdsf` currently supports 271 tools. Feel free to open an issue/pull-request if your favorite tool/command is missing! 😃

| Name | Description | Categories | Languages |
| ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | ------------------------------------------------------------------------- |
Expand Down Expand Up @@ -503,6 +503,8 @@ mdsf init
| [inko](https://github.com/inko-lang/inko) | Code formatter for the inko programming language | `formatter` | `inko` |
| [isort](https://github.com/timothycrosley/isort) | A Python utility to sort imports | `formatter` | `python` |
| [joker](https://github.com/candid82/joker) | Small Clojure interpreter, linter and formatter | `formatter`, `linter` | `clojure` |
| [jq](https://github.com/jqlang/jq) | JSON processor | `formatter` | `json` |
| [jqfmt](https://github.com/noperator/jqfmt) | like gofmt, but for jq | `formatter` | `jq` |
| [js-beautify](https://github.com/beautifier/js-beautify) | A JavaScript formatter | `formatter` | `javascript` |
| [json5format](https://github.com/google/json5format) | JSON5 (a.k.a., JSON for Humans) formatter that preserves contextual comments | `formatter` | `json5`, `json` |
| [jsona](https://github.com/jsona/jsona) | JSONA linter and formatter | `formatter`, `linter` | `jsona` |
Expand Down Expand Up @@ -653,10 +655,12 @@ mdsf init
| [xmlformat](https://github.com/pamoller/xmlformatter) | Format and compress XML documents | `formatter` | `xml` |
| [xmllint](https://gnome.pages.gitlab.gnome.org/libxml2/xmllint.html) | XML linter | `linter` | `xml` |
| [xo](https://github.com/xojs/xo) | JavaScript/TypeScript linter (ESLint wrapper) with great defaults | `linter` | `javascript`, `typescript` |
| [xq](https://github.com/sibprogrammer/xq) | Command-line XML and HTML beautifier and content extractor | `formatter` | `html`, `xml` |
| [yamlfix](https://github.com/lyz-code/yamlfix) | A simple opinionated yaml formatter that keeps your comments | `formatter` | `yaml` |
| [yamlfmt](https://github.com/google/yamlfmt) | An extensible command line tool or library to format yaml files | `formatter` | `yaml` |
| [yapf](https://github.com/google/yapf) | A formatter for Python files | `formatter` | `python` |
| [yew-fmt](https://github.com/its-the-shrimp/yew-fmt) | Code formatter for the Yew framework | `formatter` | `rust` |
| [yq](https://github.com/mikefarah/yq) | yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor | `formatter` | `yaml` |
| [zig](https://ziglang.org/) | Reformat Zig source into canonical form | `formatter` | `zig` |
| [ziggy](https://ziggy-lang.io/) | Formats Ziggy documents and Ziggy schemas | `formatter` | `ziggy` |
| [zprint](https://github.com/kkinnear/zprint) | Executables, uberjar, and library to beautifully format Clojure and Clojurescript source code and s-expressions | `formatter` | `clojure`, `clojurescript` |
Expand All @@ -667,7 +671,7 @@ mdsf init

<!-- START_SECTION:supported-commands -->

`mdsf` currently supports 288 commands. Feel free to open an issue/pull-request if your favorite tool/command is missing! 😃
`mdsf` currently supports 293 commands. Feel free to open an issue/pull-request if your favorite tool/command is missing! 😃

| Name | Command |
| ------------------------ | --------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -789,6 +793,8 @@ mdsf init
| `inko:fmt` | `inko fmt $PATH` |
| `isort` | `isort --quiet $PATH` |
| `joker` | `joker --format --write $PATH` |
| `jq` | `jq ` |
| `jqfmt` | `jqfmt ` |
| `js-beautify` | `js-beautify -r --type js -f $PATH` |
| `json5format` | `json5format -r $PATH` |
| `jsona:format` | `jsona format $PATH` |
Expand Down Expand Up @@ -952,10 +958,13 @@ mdsf init
| `xmlformat` | `xmlformat --overwrite $PATH` |
| `xmllint` | `xmllint --format $PATH --output $PATH` |
| `xo` | `xo --fix $PATH` |
| `xq:html` | `xq --html` |
| `xq` | `xq ` |
| `yamlfix` | `yamlfix $PATH` |
| `yamlfmt` | `yamlfmt -quiet $PATH` |
| `yapf` | `yapf --in-place $PATH` |
| `yew-fmt` | `yew-fmt --edition 2021 --quiet $PATH` |
| `yq` | `yq --inplace $PATH` |
| `zig:fmt` | `zig fmt $PATH` |
| `ziggy:fmt` | `ziggy fmt $PATH` |
| `zprint` | `zprint -w $PATH` |
Expand Down
33 changes: 25 additions & 8 deletions codegen/src/tools.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@
#[schemars(url)]
pub homepage: Option<String>,

#[serde(default, skip_serializing_if = "is_false")]
pub stdin: bool,

#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub tests: Vec<ToolCommandTest>,
}
Expand Down Expand Up @@ -173,6 +176,8 @@
pub homepage: String,

pub deprecated: bool,

pub stdin: bool,
}

const DEPRECATED_ATTRIBUTE: &str = "\n #[deprecated]";
Expand All @@ -194,7 +199,12 @@
)
}

fn generate_test(&self, command: &str, test: &ToolCommandTest) -> (String, String) {
fn generate_test(
&self,
command: &str,
test: &ToolCommandTest,
is_stdin: bool,
) -> (String, String) {

Check warning on line 207 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L202-L207 

Added lines #L202 - L207 were not covered by tests
let mut hasher = DefaultHasher::new();

test.hash(&mut hasher);
Expand Down Expand Up @@ -222,7 +232,7 @@
{INDENT}{INDENT}{INDENT}crate::execution::setup_snippet(input, &file_ext).expect(\"it to create a snippet file\");

{INDENT}{INDENT}let result =
{INDENT}{INDENT}{INDENT}crate::execution::run_tools(&super::COMMANDS, snippet.path(), super::set_args, 0)
{INDENT}{INDENT}{INDENT}crate::execution::run_tools(&super::COMMANDS, snippet.path(), super::set_args, 0, {is_stdin})

Check warning on line 235 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L235 

Added line #L235 was not covered by tests
{INDENT}{INDENT}{INDENT}{INDENT}.expect(\"it to be successful\")
{INDENT}{INDENT}{INDENT}{INDENT}.1
{INDENT}{INDENT}{INDENT}{INDENT}.expect(\"it to be some\");
Expand Down Expand Up @@ -304,14 +314,16 @@
.collect::<Vec<_>>()
.join("\n");

assert!(args_includes_path);
if !options.stdin {
assert!(args_includes_path);
}

Check warning on line 319 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L317-L319 

Added lines #L317 - L319 were not covered by tests

let module_name = command_name.to_case(Case::Snake);

let tests = options
.tests
.iter()
.map(|test| self.generate_test(cmd, test).1)
.map(|test| self.generate_test(cmd, test, options.stdin).1)

Check warning on line 326 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L326 

Added line #L326 was not covered by tests
.collect::<Vec<_>>();

let tests = if tests.is_empty() {
Expand All @@ -329,7 +341,7 @@
#[inline]
pub fn set_args(
{INDENT}mut cmd: std::process::Command,
{INDENT}file_path: &std::path::Path,
{INDENT}{unused_prefix}file_path: &std::path::Path,

Check warning on line 344 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L344 

Added line #L344 was not covered by tests
) -> std::process::Command {{
{string_args}
{INDENT}cmd
Expand All @@ -340,6 +352,7 @@
#[cfg(test)]
mod test_{module_name} {{{tests}}}
",
unused_prefix = if options.stdin { "_" } else { "" },

Check warning on line 355 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L355 

Added line #L355 was not covered by tests
);

let homepage = options.homepage.clone().unwrap_or_default();
Expand Down Expand Up @@ -369,6 +382,7 @@
description
},
deprecated: options.deprecated || self.deprecated,
stdin: options.stdin,

Check warning on line 385 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L385 

Added line #L385 was not covered by tests
});
}

Expand Down Expand Up @@ -470,8 +484,10 @@
args = command.args.join(" ")
));

let is_stdin = command.stdin;

Check warning on line 488 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L487-L488 

Added lines #L487 - L488 were not covered by tests
format_snippet_values.insert(format!(
"{INDENT}{INDENT}{INDENT}Self::{enum_value} => (&{module_name}::COMMANDS, {module_name}::set_args),"
"{INDENT}{INDENT}{INDENT}Self::{enum_value} => (&{module_name}::COMMANDS, {module_name}::set_args, {is_stdin}),",

Check warning on line 490 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L490 

Added line #L490 was not covered by tests
));

as_ref_values.insert(format!(
Expand Down Expand Up @@ -517,14 +533,15 @@
{INDENT}{INDENT}snippet_path: &std::path::Path,
{INDENT}{INDENT}timeout: u64,
{INDENT}) -> Result<(bool, Option<String>), crate::error::MdsfError> {{
{INDENT}{INDENT}let (commands, set_args_fn): (
{INDENT}{INDENT}let (commands, set_args_fn, is_stdin): (

Check warning on line 536 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L536 

Added line #L536 was not covered by tests
{INDENT}{INDENT}{INDENT}&[crate::runners::CommandType],
{INDENT}{INDENT}{INDENT}fn(std::process::Command, &std::path::Path) -> std::process::Command,
{INDENT}{INDENT}{INDENT}bool,

Check warning on line 539 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L539 

Added line #L539 was not covered by tests
{INDENT}{INDENT}) = match self {{
{}
{INDENT}{INDENT}}};

{INDENT}{INDENT}crate::execution::run_tools(commands, snippet_path, set_args_fn, timeout)
{INDENT}{INDENT}crate::execution::run_tools(commands, snippet_path, set_args_fn, timeout, is_stdin)

Check warning on line 544 in codegen/src/tools.rs

View check run for this annotation

Codecov / codecov/patch

codegen/src/tools.rs#L544 

Added line #L544 was not covered by tests
{INDENT}}}
}}

Expand Down
2 changes: 2 additions & 0 deletions mdsf/src/error.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
LanguageAliasMissingTools(String),
ReadStdinError(std::io::Error),
MissingInput,
StdinWriteError,
}

impl std::error::Error for MdsfError {}
Expand Down Expand Up @@ -67,6 +68,7 @@
"'{alias}' cannot be used as an alias since it has no tools specified"
),
Self::MissingInput => write!(f, "No input was provided to mdsf"),
Self::StdinWriteError => write!(f, "Error writing to stdin"),

Check warning on line 71 in mdsf/src/error.rs

View check run for this annotation

Codecov / codecov/patch

mdsf/src/error.rs#L71 

Added line #L71 was not covered by tests
}
}
}
Expand Down
Loading
Loading