Skip to content

Better EditorConfig Settings & Code Styles Validation via Travis CI #6523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 67 additions & 4 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -1,16 +1,79 @@
root = true

[*]

## Repository Configurations
############################
[.{git*,editorconfig,*.yml}]
charset = utf-8
end_of_line = lf
indent_style = space
indent_size = unset
trim_trailing_whitespace = true
insert_final_newline = true

[.gitmodules]
indent_style = tab


## Shell Scripts
################
[*.sh]
charset = utf-8
end_of_line = lf
indent_style = tab
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are using 4 spaces, at least that's what I've been using. But I might be wrong and I am ok with either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To the best of my knowledge, shell scripts should use tabs only for indentation, because spaces indents could lead to problems in some contexts — the matter is subject of dispute (as everything relating to the tabs-vs-space flamewars) but this seems to be the prevalent opinion, at least on long StackOverflow discussions.

But I'll set it to any value you suggest, the most important thing is to keep the code consistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why tabs are preferable is in case shell scripts are used to dynamically generate other scripts via pipes and redirections, where tabs are a safer option because common used tools will auto-strip leading tabs (but not spaces).

An example:

https://www.howtobuildsoftware.com/index.php/how-do/ctWc/shell-sh-perforce-error-in-creating-p4-label-non-interactively-as-part-of-shell-script

Beside that, it boils down to a matter of personal choice (and the many flamewars on the subject didn't really bring any solution to the issue). I personally prefer using spaces in version controlled projects, for they create more readable diffs. But since in Make files tabs are mandatory, and shell scripts are sometimes used to manipulate Make files, I prefer to adopt tabs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some interesting trivia on the subject:

So, yes, I'm for spaces-indentation, but make an exception for shell scripts.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I didn't know. Thanks!

indent_size = unset
trim_trailing_whitespace = true
insert_final_newline = true


## Make Files
#############
[Makefile]
charset = utf-8
end_of_line = lf
indent_style = tab
indent_size = 2
trim_trailing_whitespace = true
insert_final_newline = true


## Haskell Sources
##################
[*.hs]
charset = utf-8
end_of_line = unset
indent_style = space
indent_size = unset
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general coding style uses two spaces, but Haskell indentation is weird sometimes. Does verification fail if we set this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it would fail because the validator tool expects all indentation to be a multiple of the base value. Unfortunately the EditorConfig project doesn't offer separate settings for editor/IDEs configurations and validators right now.

Since most coders add extra spaces to obtain elegant alignments on code that spans multiple lines, it often becomes necessary to either unset indent_size or set it to 1 — the former is better because it doesn't override the editor's defaults, whereas the latter would actually enforce 1-space tabs.

As for markdown (and other lightweight markup syntaxes) the problem affect code blocks, which could be using even or odd indentation, depending on the language snippet being included.

Hopefully in future editions of EditorConfig we'll see new settings to address validation aspects only. Right now, one is forced to compromise on some features to obtain both editor settings and CI validation — e.g. EditorConfig doesn't (and will never) support end_of_line = native, so to allow cross-platform EOL normalization end_of_line has to be unset and the project has to rely on .gitattributes instead.

I'll take a further look at EClint's documentation, there are some ways to customize validation settings and overriding the .editorconfig values via a separate config file for EClint; if this is the case, we could enforce 2-spaces on the editor and override it on EClint by unsetting indent_size for some file types. It might require some time for me to dig into it and test it, but in the meantime we could use these settings for they are fairly good in practical use, from my own experience (you get a sound validation and don't interfere with the editor/IDE).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've checked EClint documentation, and there's no easy way to override .editorconfig settings on a per-file-type manner (you can override options globally for each run). The only alternative would be to run multiple EClint checks, each targetting a specific file extension, but it seems an overkill solution.

trim_trailing_whitespace = true
insert_final_newline = true


## Markdown
###########
[*.{markdown,md}]
charset = utf-8
end_of_line = unset
indent_style = space
indent_size = unset
trim_trailing_whitespace = false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're not using trailing spaces anywhere, might be ok to set this to true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally never use trailing spaces for hard line-breaks, but we need to check with @jgm for this setting, after all GFM does support that feature so potentially it could be used in the GFM documentation or templates, and if I remember correctly pandoc also supports the feature via extensions.

I always set trim_trailing_whitespace to true in my projects, because trailing space are the main cause of commits "noise", leading to hard-to-read diffs and (often) spurious contents changes.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure there are tests that include trailing whitespace, even if we don't use it in documentation (and for all I recall, we might).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've revised the rules for test/ folder in commit 6080aeb, so they apply to markdown files only — basically the only checks now are for UTF-8 encoding and EOL consistency, all other rules are dropped to allow testing edge-cases (including trailing spaces).

even if we don't use it in documentation (and for all I recall, we might).

I've checked. the only markdown file with trailing spaces is COPYING.md, which contains some trailing spaces that are just spurious and could actually be removed.

So, @jgm, should we set the rules for markdown files outside the test/ folder to forbid trailing WS? (and cleanup the COPYING.md doc)

Unless we need to use them in the documentation, it would prevent unclean commits.

insert_final_newline = true

[test/*]
insert_final_newline = false
trim_trailing_whitespace = false

## Lua and Perl
###############
[*.{lua,pl}]
charset = utf-8
end_of_line = unset
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
insert_final_newline = true


## Test Files
#############
[test/**]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that we still want the general rules to apply for Haskell and Lua files in test. Should we move those rules further down, or is there a better way?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should apply the above pattern only to markdown files?

## Test Files
#############
[test/**.{md,markdown}]

After all, I believe that is only the markdown files that need relaxed rules to allow testing edge-cases and malformed documents.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6080aeb!

indent_style = unset
indent_size = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
15 changes: 15 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,16 @@
* text=auto

## Repository Configuration
###########################
.*.yml text eol=lf
.editorconfig text eol=lf
.gitattributes text eol=lf
.gitconfig text eol=lf
.gitignore text eol=lf
.gitmodules text eol=lf


*.sh text eol=lf
Makefile text eol=lf

test/fb2/reader/* -text
15 changes: 15 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
dist: trusty
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, then this is a temporary solution and would be converted into a GitHub Action workflow after merging?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I leave that choice to you, as you're surely more qualified to decide on this.

Personally, I always use Travis for code validation, to avoid bumping into the free GH Actions limits (since validation is done for every commit on every branch, having many repositories could quickly reach the limit).

Another reason to keep it on Travis (instead of Circle or GH Actions) might be performance, since it would run in a parallel job.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to avoid using multiple CI providers; if we can do it all in GH actions that seems better to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a ready availabe EClint action:

https://github.com/marketplace/actions/eclint

I haven't use GH Actions before, some I'm not sure how to set it up. Can someone else add the action?


git:
depth: false

install:
- npm install -g eclint

script:
# ==============================================
# EditorConfig Code Styles Validation via EClint
# ==============================================
# https://editorconfig.org
# https://www.npmjs.com/package/eclint
- bash ./validate.sh
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,10 @@ a ghci REPL for working with pandoc. With [stack], use
We recommend using the following `.ghci` file (which can be
placed in the source directory):

:set -fobject-code
:set -XTypeSynonymInstances
:set -XScopedTypeVariables
:set -XOverloadedStrings
:set -fobject-code
:set -XTypeSynonymInstances
:set -XScopedTypeVariables
:set -XOverloadedStrings

Profiling
---------
Expand Down
2 changes: 1 addition & 1 deletion COPYING.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,4 +358,4 @@ you may consider it more useful to permit linking proprietary
applications with the library. If this is what you want to do, use the
[GNU Lesser General Public
License](http://www.gnu.org/licenses/lgpl.html) instead of this
License.
License.
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,8 @@ man/pandoc.1: MANUAL.txt man/pandoc.1.before man/pandoc.1.after
-o $@

README.md: README.template MANUAL.txt tools/update-readme.lua
pandoc --lua-filter tools/update-readme.lua --reference-links \
--reference-location=section -t gfm $< -o $@
pandoc --lua-filter tools/update-readme.lua --reference-links \
--reference-location=section -t gfm $< -o $@

download_stats:
curl https://api.github.com/repos/jgm/pandoc/releases | \
Expand Down
16 changes: 8 additions & 8 deletions data/sample.lua
Original file line number Diff line number Diff line change
Expand Up @@ -143,12 +143,12 @@ end

function Link(s, src, tit, attr)
return "<a href='" .. escape(src,true) .. "' title='" ..
escape(tit,true) .. "'>" .. s .. "</a>"
escape(tit,true) .. "'>" .. s .. "</a>"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand most of the changes in this file. Does validation fail without them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it fails because in this case the setting is 2 spaces, so all indentation needs to be a multiple of 2 (i.e. odd spacing).

All the (cosmetic) changes I've done to the sources were due to validations errors.

end

function Image(s, src, tit, attr)
return "<img src='" .. escape(src,true) .. "' title='" ..
escape(tit,true) .. "'/>"
escape(tit,true) .. "'/>"
end

function Code(s, attr)
Expand Down Expand Up @@ -226,8 +226,8 @@ function HorizontalRule()
end

function LineBlock(ls)
return '<div style="white-space: pre-line;">' .. table.concat(ls, '\n') ..
'</div>'
return '<div style="white-space: pre-line;">' .. table.concat(ls, '\n') ..
'</div>'
end

function CodeBlock(s, attr)
Expand All @@ -238,8 +238,8 @@ function CodeBlock(s, attr)
return '<img src="data:' .. image_mime_type .. ';base64,' .. img .. '"/>'
-- otherwise treat as code (one could pipe through a highlighter)
else
return "<pre><code" .. attributes(attr) .. ">" .. escape(s) ..
"</code></pre>"
return "<pre><code" .. attributes(attr) .. ">" .. escape(s) ..
"</code></pre>"
end
end

Expand All @@ -264,7 +264,7 @@ function DefinitionList(items)
for _,item in pairs(items) do
local k, v = next(item)
table.insert(buffer, "<dt>" .. k .. "</dt>\n<dd>" ..
table.concat(v, "</dd>\n<dd>") .. "</dd>")
table.concat(v, "</dd>\n<dd>") .. "</dd>")
end
return "<dl>\n" .. table.concat(buffer, "\n") .. "\n</dl>"
end
Expand All @@ -284,7 +284,7 @@ function html_align(align)
end

function CaptionedImage(src, tit, caption, attr)
return '<div class="figure">\n<img src="' .. escape(src,true) ..
return '<div class="figure">\n<img src="' .. escape(src,true) ..
'" title="' .. escape(tit,true) .. '"/>\n' ..
'<p class="caption">' .. caption .. '</p>\n</div>'
end
Expand Down
2 changes: 1 addition & 1 deletion linux/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build:
mkdir -p $(ARTIFACTS)
docker build -t alpine-pandoc .
docker run --env TREE=$(TREE) --env REVISION=$(REVISION) \
-v $(ARTIFACTS):/artifacts alpine-pandoc
-v $(ARTIFACTS):/artifacts alpine-pandoc

interact:
docker run --env TREE=$(TREE) --env REVISION=$(REVISION) \
Expand Down
12 changes: 6 additions & 6 deletions linux/make_deb.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ set -e

MACHINE=$(uname -m)
case "$MACHINE" in
x86_64) ARCHITECTURE=amd64;;
i686) ARCHITECTURE=i386;;
i386) ARCHITECTURE=i386;;
x86_64) ARCHITECTURE=amd64;;
i686) ARCHITECTURE=i386;;
i386) ARCHITECTURE=i386;;
esac

ARTIFACTS="${ARTIFACTS:-/artifacts}"
Expand Down Expand Up @@ -42,9 +42,9 @@ $ARTIFACTS/pandoc-citeproc --license >> $COPYRIGHT
INSTALLED_SIZE=$(du -k -s $DEST | awk '{print $1}')
mkdir $DIST/DEBIAN
perl -pe "s/VERSION/$DEBVER/" linux/control.in | \
perl -pe "s/ARCHITECTURE/$ARCHITECTURE/" | \
perl -pe "s/INSTALLED_SIZE/$INSTALLED_SIZE/" \
> $DIST/DEBIAN/control
perl -pe "s/ARCHITECTURE/$ARCHITECTURE/" | \
perl -pe "s/INSTALLED_SIZE/$INSTALLED_SIZE/" \
> $DIST/DEBIAN/control

fakeroot dpkg-deb --build $DIST
rm -rf $DIST
Expand Down
Empty file modified linux/make_tarball.sh
100644 → 100755
Empty file.
4 changes: 2 additions & 2 deletions macos/uninstall-pandoc.pl
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,8 @@

} else {

print "OK, aborting uninstall.\n";
exit;
print "OK, aborting uninstall.\n";
exit;
}

print "Pandoc has been successfully uninstalled.\n";
Expand Down
8 changes: 4 additions & 4 deletions src/Text/Pandoc/Writers/LaTeX.hs
Original file line number Diff line number Diff line change
Expand Up @@ -1037,7 +1037,7 @@ mapAlignment a = case a of
"top-baseline" -> "t"
"bottom" -> "b"
"center" -> "c"
_ -> a
_ -> a

wrapDiv :: PandocMonad m => Attr -> Doc Text -> LW m (Doc Text)
wrapDiv (_,classes,kvs) t = do
Expand All @@ -1051,7 +1051,7 @@ wrapDiv (_,classes,kvs) t = do
(lookup "totalwidth" kvs)
onlytextwidth = filter ((==) "onlytextwidth") classes
options = text $ T.unpack $ T.intercalate "," $
valign : totalwidth ++ onlytextwidth
valign : totalwidth ++ onlytextwidth
in inCmd "begin" "columns" <> brackets options
$$ contents
$$ inCmd "end" "columns"
Expand All @@ -1062,8 +1062,8 @@ wrapDiv (_,classes,kvs) t = do
maybe ""
(brackets . text . T.unpack . mapAlignment)
(lookup "align" kvs)
w = maybe "0.48" fromPct (lookup "width" kvs)
in inCmd "begin" "column" <>
w = maybe "0.48" fromPct (lookup "width" kvs)
in inCmd "begin" "column" <>
valign <>
braces (literal w <> "\\textwidth")
$$ contents
Expand Down
12 changes: 6 additions & 6 deletions test/grofftest.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@
# is the directory, and pandoc is used from path.

if [ $# -eq 2 ]; then
PANDOC=$1
DIR=$2
PANDOC=$1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

DIR=$2
elif [ $# -eq 1 ]; then
PANDOC=pandoc
DIR=$1
PANDOC=pandoc
DIR=$1
else
echo "Not enough arguments"
exit 1
Expand All @@ -21,6 +21,6 @@ fi
$PANDOC --version > /dev/null || { echo "pandoc executable error" >&2 ; exit 1 ; }

for f in `find "$DIR" -name '*.[0-9]'`; do
( iconv -f utf8 -t utf8 $f 2>/dev/null || iconv -f latin1 -t utf8 $f ) | \
$PANDOC --resource-path "$DIR":"$(dirname $f)" -f man -o /dev/null || echo "Failed to convert $f"
( iconv -f utf8 -t utf8 $f 2>/dev/null || iconv -f latin1 -t utf8 $f ) | \
$PANDOC --resource-path "$DIR":"$(dirname $f)" -f man -o /dev/null || echo "Failed to convert $f"
done
4 changes: 2 additions & 2 deletions test/lua/inlines-filter.lua
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
function isWorldAfterSpace (fst, snd)
return fst and fst.t == 'LineBreak'
and snd and snd.t == 'Str' and snd.text == 'World!'
return fst and fst.t == 'LineBreak'
and snd and snd.t == 'Str' and snd.text == 'World!'
end

function Inlines (inlns)
Expand Down
2 changes: 1 addition & 1 deletion test/lua/module/pandoc-list.lua
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ return {
)
end),
test('return empty list if no argument is given', function ()
assert.are_same({}, List:new())
assert.are_same({}, List:new())
end),
test('metatable of result is pandoc.List', function ()
local test = List:new{5}
Expand Down
28 changes: 14 additions & 14 deletions test/lua/module/pandoc.lua
Original file line number Diff line number Diff line change
Expand Up @@ -215,20 +215,20 @@ return {

group 'walk_block' {
test('block walking order', function ()
local acc = {}
local nested_nums = pandoc.Div {
pandoc.Para{pandoc.Str'1'},
pandoc.Div{
pandoc.Para{pandoc.Str'2'},
pandoc.Para{pandoc.Str'3'}
},
pandoc.Para{pandoc.Str'4'}
}
pandoc.walk_block(
nested_nums,
{Para = function (p) table.insert(acc, p.content[1].text) end}
)
assert.are_equal('1234', table.concat(acc))
local acc = {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be indented by two additional spaces.

Copy link
Contributor Author

@tajmone tajmone Jul 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit 5bdf167 should fix this (hopefully).

local nested_nums = pandoc.Div {
pandoc.Para{pandoc.Str'1'},
pandoc.Div{
pandoc.Para{pandoc.Str'2'},
pandoc.Para{pandoc.Str'3'}
},
pandoc.Para{pandoc.Str'4'}
}
pandoc.walk_block(
nested_nums,
{Para = function (p) table.insert(acc, p.content[1].text) end}
)
assert.are_equal('1234', table.concat(acc))
end)
},

Expand Down
25 changes: 12 additions & 13 deletions tools/changelog-helper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,16 @@ lastmod=`git log -n2 --format=oneline changelog | awk '{print $1;}'`
files=`git ls-tree -r master --name-only`
for x in $files
do
commits=`git log -n1 $lastmod..HEAD $x`
if [ ! -z "$commits" ]
then
if echo $x | grep -q "src\/.*\.hs"
then
file=`echo $x | sed -e 's/src\///' | sed -e 's/\//\./g' | sed -e 's/\.hs$//'`
else
file=$x
fi
echo " * $file"
GIT_PAGER=cat git log --pretty=format:'%n%w(78,4,6)+ %s (%aN)%n%n%w(78,6,6)%b%n' -- "$lastmod..HEAD" "$x"
fi
commits=`git log -n1 $lastmod..HEAD $x`
if [ ! -z "$commits" ]
then
if echo $x | grep -q "src\/.*\.hs"
then
file=`echo $x | sed -e 's/src\///' | sed -e 's/\//\./g' | sed -e 's/\.hs$//'`
else
file=$x
fi
echo " * $file"
GIT_PAGER=cat git log --pretty=format:'%n%w(78,4,6)+ %s (%aN)%n%n%w(78,6,6)%b%n' -- "$lastmod..HEAD" "$x"
fi
done

Loading