smart squotes and apostrophe

README.md

@@ -1,12 +1,11 @@
-# vim-pandoc-syntax
+# Vim Syntax for Simplified Pandoc
 
-[![Vint](https://github.com/vim-pandoc/vim-pandoc-syntax/workflows/Vint/badge.svg)](https://github.com/vim-pandoc/vim-pandoc-syntax/actions?workflow=Vint)
+The `vim-pandoc-simple-syntax` standalone Pandoc syntax module is a fork from the [vim-pandoc/vim-pandoc-syntax](https://github.com/vim-pandoc/vim-pandoc-syntax):
 
-Standalone pandoc syntax module, to be used alongside
-[vim-pandoc](http://github.com/vim-pandoc/vim-pandoc).
-
-Forked from the version provided by `fmoralesc/vim-pantondoc`, in turn taken
-from `vim-pandoc/vim-pandoc`.
+* Corrects faults in highlighting and smart punctuation
+* Adds opinionated color settings
+* Removes many confusing default conceals
+* Removes (and flags) common departures from Simplified Pandoc
 
 ## Requirements
 
@@ -16,44 +15,44 @@ from `vim-pandoc/vim-pandoc`.
 
 ## Installation
 
-The repository follows the usual bundle structure, so it's easy to install it
-using [pathogen](https://github.com/tpope/vim-pathogen),
-[Vundle](https://github.com/gmarik/vundle) or NeoBundle.
-
-For Vundle users, it should be enough to add
-
-    Plugin 'vim-pandoc/vim-pandoc-syntax'
+Add something like the following to your `.vimrc` after installing the [`Plug`](https://github.com/junegunn/vim-plug) Vim package manager (all the others are rather dated and don't allow storing plugins separated):
 
-to `.vimrc`, and then run `:PluginInstall`.
 
-For those who need it, a tarball is available from
-[here](https://github.com/vim-pandoc/vim-pandoc-syntax/archive/master.zip).
+```vim
+call plug#begin('~/.vimplugins')
+Plug 'vim-pandoc/vim-pandoc'
+Plug 'rwxrob/vim-pandoc-syntax-simple'
+call plug#end()
+```
 
 ### Standalone
 
-If you want to use `vim-pandoc-syntax` without vim-pandoc, you'll need to tell
-Vim to load it for certain files. Just add something like this to your vimrc:
+If you want to use `vim-pandoc-syntax` without vim-pandoc, you'll need to tell Vim to load it for certain files. Just add something like this to your vimrc:
 
-~~~ vim
-    augroup pandoc_syntax
-        au! BufNewFile,BufFilePre,BufRead *.md set filetype=markdown.pandoc
-    augroup END
-~~~
+```vim
+augroup pandoc_syntax
+  au! BufNewFile,BufFilePre,BufRead *.md set filetype=markdown.pandoc
+augroup END
+```
 
 ## Features
 
-* Supports most (if not all) pandoc's markdown features, including tables,
+* Supports most (if not all) Pandoc's markdown features, including tables,
   delimited codeblocks, references, etc.
-* Can handle multiple embedded languages (LaTeX, YAML headers, many languages
-  in delimited codeblocks). Some commands are provided to help with this (see
-  `:help pandoc-syntax-commands`)
+
+* Can handle multiple embedded languages (LaTeX, YAML headers, many languages in delimited codeblocks). Some commands are provided to help with this (see `:help pandoc-syntax-commands`)
+
 * Pretty display using `conceal` (optional).
-* Configurable (see `:help pandoc-syntax-configuration` for an overview of the
-  options).
 
-## Screenshots
+* Configurable (see `:help pandoc-syntax-configuration` for an overview of the options).
+
+## TODO
 
-![img1](http://i.imgur.com/UKXbG2V.png)
-![img2](http://i.imgur.com/z8FpxRP.png)
-![img3](http://i.imgur.com/ziNjQiE.png)
-![img4](http://i.imgur.com/UKoOxzP.png)
+* ***Remove*** conceal ligatures for section and lists.
+* Add back screenshots (from imgur to keep plugin light).
+* Correct syntax highlighting of Pandoc Tables.
+* Integrated better defaults colors.
+* Consider adding themes for better color customization.
+* Add ellipsis smart punctuation from list context. 
+* Correct en-dash and em-dash missing contexts.
+* Other stuff ...

doc/pandoc-syntax.txt

@@ -37,6 +37,8 @@ CONFIGURATION				       *vim-pandoc-syntax-configuration*
   - dashes
   - ellipses
   - quotes
+  - squotes
+  - apostrophe
   - inlinecode
 
   To review what are the rules for, look for the call to |s:WithConceal| in

syntax/pandoc.vim

@@ -3,13 +3,14 @@ scriptencoding utf-8
 "
 " Vim syntax file
 "
-" Language: Pandoc (superset of Markdown)
-" Maintainer: Felipe Morales <hel.sheep@gmail.com>
-" Maintainer: Caleb Maclennan <caleb@alerque.com>
+" Language: Simplified Pandoc (superset of Markdown)
+" Maintainer: Rob Muhlestein (twitch.tv/rwxrob) <rwx@robs.io>
+" Contributor: Felipe Morales <hel.sheep@gmail.com>
+" Contributor: Caleb Maclennan <caleb@alerque.com>
 " Contributor: David Sanson <dsanson@gmail.com>
 " Contributor: Jorge Israel Peña <jorge.israel.p@gmail.com>
 " OriginalAuthor: Jeremy Schultz <taozhyn@gmail.com>
-" Version: 5.0
+" Version: 5.1
 
 " Configuration: {{{1
 "
@@ -223,6 +224,14 @@ endif
 
 " Syntax Rules: {{{1
 
+" corrects distracting conceal (ligature) background colors, blue makes 
+" it obvious that the character is a ligature instead of an actual Unicode
+" character since both can exist in the same file
+hi Conceal ctermbg=none ctermfg=Blue
+
+" preferable link color to underlining
+hi link pandocDelimitedCodeBlock pandocNoFormatted
+
 " Embeds: {{{2
 
 " HTML: {{{3
@@ -264,7 +273,7 @@ syn match pandocTitleBlockTitle /\%^%.*\n/ contained containedin=pandocTitleBloc
 " }}}2
 
 " Blockquotes: {{{2
-syn match pandocBlockQuote /^\s\{,3}>.*\n\(.*\n\@1<!\n\)*/ contains=@Spell,pandocEmphasis,pandocStrong,pandocPCite,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocUListItem,pandocNoFormatted,pandocAmpersandEscape,pandocLaTeXInlineMath,pandocEscapedDollar,pandocLaTeXCommand,pandocLaTeXMathBlock,pandocLaTeXRegion skipnl
+syn match pandocBlockQuote /^\s\{,3}>.*\n\(.*\n\@1<!\n\)*/ contains=@Spell,pandocEmphasis,pandocStrong,pandocPCite,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocUListItem,pandocNoFormatted,pandocAmpersandEscape,pandocLaTeXInlineMath,pandocEscapedDollar,pandocLaTeXCommand,pandocLaTeXMathBlock,pandocLaTeXRegion,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostrophe  skipnl
 syn match pandocBlockQuoteMark /\_^\s\{,3}>/ contained containedin=pandocEmphasis,pandocStrong,pandocPCite,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocUListItem,pandocNoFormatted
 " }}}2
 
@@ -368,11 +377,11 @@ call s:WithConceal('strikeout', 'syn match pandocStrikeoutMark /\~\~/ contained
 " }}}2
 
 " Headers: {{{2
-syn match pandocAtxHeader /\(\%^\|<.\+>.*\n\|^\s*\n\)\@<=#\{1,6}.*\n/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,@Spell,pandocAmpersandEscape,pandocReferenceLabel,pandocReferenceURL display
+syn match pandocAtxHeader /\(\%^\|<.\+>.*\n\|^\s*\n\)\@<=#\{1,6}.*\n/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostrophe,pandocEllipses,@Spell,pandocAmpersandEscape,pandocReferenceLabel,pandocReferenceURL display
 syn match pandocAtxHeaderMark /\(^#\{1,6}\|\\\@<!#\+\(\s*.*$\)\@=\)/ contained containedin=pandocAtxHeader
 call s:WithConceal('atx', 'syn match pandocAtxStart /#/ contained containedin=pandocAtxHeaderMark', 'conceal cchar='.s:cchars['atx'])
-syn match pandocSetexHeader /^.\+\n[=]\+$/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,@Spell,pandocAmpersandEscape
-syn match pandocSetexHeader /^.\+\n[-]\+$/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,@Spell,pandocAmpersandEscape
+syn match pandocSetexHeader /^.\+\n[=]\+$/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostrophe,pandocEllipses,@Spell,pandocAmpersandEscape
+syn match pandocSetexHeader /^.\+\n[-]\+$/ contains=pandocEmphasis,pandocStrong,pandocNoFormatted,pandocLaTeXInlineMath,pandocEscapedDollar,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostrophe,pandocEllipses,@Spell,pandocAmpersandEscape
 syn match pandocHeaderAttr /{.*}/ contained containedin=pandocAtxHeader,pandocSetexHeader
 syn match pandocHeaderID /#[-_:.[:lower:][:upper:]]*/ contained containedin=pandocHeaderAttr
 " }}}2
@@ -452,12 +461,12 @@ call s:WithConceal('abbrev', 'syn match pandocAbbreviationTail /\]/ contained co
 syn match pandocFootnoteID /\[\^[^\]]\+\]/ nextgroup=pandocFootnoteDef
 
 "   Inline footnotes
-syn region pandocFootnoteDef start=/\^\[/ skip=/\[.\{-}]/ end=/\]/ contains=pandocReferenceLabel,pandocReferenceURL,pandocLatex,pandocPCite,pandocCiteKey,pandocStrong,pandocEmphasis,pandocStrongEmphasis,pandocNoFormatted,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocEnDash,pandocEmDash,pandocEllipses,pandocBeginQuote,pandocEndQuote,@Spell,pandocAmpersandEscape skipnl keepend
+syn region pandocFootnoteDef start=/\^\[/ skip=/\[.\{-}]/ end=/\]/ contains=pandocReferenceLabel,pandocReferenceURL,pandocLatex,pandocPCite,pandocCiteKey,pandocStrong,pandocEmphasis,pandocStrongEmphasis,pandocNoFormatted,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocEnDash,pandocEmDash,pandocEllipses,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostrophe,@Spell,pandocAmpersandEscape skipnl keepend
 call s:WithConceal('footnote', 'syn match pandocFootnoteDefHead /\^\[/ contained containedin=pandocFootnoteDef', 'conceal cchar='.s:cchars['footnote'])
 call s:WithConceal('footnote', 'syn match pandocFootnoteDefTail /\]/ contained containedin=pandocFootnoteDef', 'conceal')
 
 " regular footnotes
-syn region pandocFootnoteBlock start=/\[\^.\{-}\]:\s*\n*/ end=/^\n^\s\@!/ contains=pandocReferenceLabel,pandocReferenceURL,pandocLatex,pandocPCite,pandocCiteKey,pandocStrong,pandocEmphasis,pandocNoFormatted,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocEnDash,pandocEmDash,pandocNewLine,pandocStrongEmphasis,pandocEllipses,pandocBeginQuote,pandocEndQuote,pandocLaTeXInlineMath,pandocEscapedDollar,pandocLaTeXCommand,pandocLaTeXMathBlock,pandocLaTeXRegion,pandocAmpersandEscape,@Spell skipnl
+syn region pandocFootnoteBlock start=/\[\^.\{-}\]:\s*\n*/ end=/^\n^\s\@!/ contains=pandocReferenceLabel,pandocReferenceURL,pandocLatex,pandocPCite,pandocCiteKey,pandocStrong,pandocEmphasis,pandocNoFormatted,pandocSuperscript,pandocSubscript,pandocStrikeout,pandocEnDash,pandocEmDash,pandocNewLine,pandocStrongEmphasis,pandocEllipses,pandocBeginQuote,pandocEndQuote,pandocBeginSQuote,pandocEndSQuote,pandocApostophe,pandocLaTeXInlineMath,pandocEscapedDollar,pandocLaTeXCommand,pandocLaTeXMathBlock,pandocLaTeXRegion,pandocAmpersandEscape,@Spell skipnl
 syn match pandocFootnoteBlockSeparator /:/ contained containedin=pandocFootnoteBlock
 syn match pandocFootnoteID /\[\^.\{-}\]/ contained containedin=pandocFootnoteBlock
 call s:WithConceal('footnote', 'syn match pandocFootnoteIDHead /\[\^/ contained containedin=pandocFootnoteID', 'conceal cchar='.s:cchars['footnote'])
@@ -500,19 +509,19 @@ endif
 
 " Emdashes: {{{3
 if &encoding ==# 'utf-8'
-  call s:WithConceal('emdashes', 'syn match pandocEllipses /\([^-]\)\@<=---\([^-]\)\@=/ display', 'conceal cchar=—')
+  call s:WithConceal('emdashes', 'syn match pandocEmdashes /---/ containedin=pandocSetexHeader,pandocEmphasis,pandocStrong,pandocListItem,pandocListItemContinuation,pandocUListItem display', 'conceal cchar=—')
 endif
 " }}}3
 
 " Endashes: {{{3
 if &encoding ==# 'utf-8'
-  call s:WithConceal('endashes', 'syn match pandocEllipses /\([^-]\)\@<=--\([^-]\)\@=/ display', 'conceal cchar=–')
+  call s:WithConceal('endashes', 'syn match pandocEndashes /\([^-]\)\@<=--\([^-]\)\@=/ display', 'conceal cchar=–')
 endif
 " }}}3
 
 " Ellipses: {{{3
 if &encoding ==# 'utf-8'
-    call s:WithConceal('ellipses', 'syn match pandocEllipses /\.\.\./ display', 'conceal cchar=…')
+    call s:WithConceal('ellipses', 'syn match pandocEllipses /\.\.\./ containedin=pandocEmphasis,pandocStrong,pandocListItem,pandocListItemContinuation,pandocUListItem  display', 'conceal cchar=…')
 endif
 " }}}3
 
@@ -522,6 +531,19 @@ if &encoding ==# 'utf-8'
     call s:WithConceal('quotes', 'syn match pandocEndQuote /\(\>[[:punct:]]*\)\@<="[[:blank:][:punct:]\n]\@=/  containedin=pandocEmphasis,pandocStrong,pandocUListItem,pandocListItem,pandocListItemContinuation display', 'conceal cchar=”')
 endif
 " }}}3
+"
+" Apostrophes: {{{3
+if &encoding ==# 'utf-8'
+  call s:WithConceal('apostrophes', 'syn match pandocApostrophe /[*~_\n[:space:]]\@<!''/  containedin=pandocEmphasis,pandocStrong,pandocUListItem,pandocListItem,pandocListItemContinuation display', 'conceal cchar=’')
+endif
+" }}}3
+
+" Single Quotes: {{{3
+if &encoding ==# 'utf-8'
+    call s:WithConceal('squotes', 'syn match pandocBeginSQuote /[_*\n[:space:]]\@<=''\</  containedin=pandocEmphasis,pandocStrong,pandocListItem,pandocListItemContinuation,pandocUListItem display', 'conceal cchar=‘')
+    call s:WithConceal('squotes', 'syn match pandocEndSQuote /\(\>[[:punct:]]*\)\@<=''[[:blank:][:punct:]\n]\@=/  containedin=pandocEmphasis,pandocStrong,pandocUListItem,pandocListItem,pandocListItemContinuation display', 'conceal cchar=’')
+endif
+" }}}3
 
 " Hrule: {{{3
 syn match pandocHRule /^\s*\([*\-_]\)\s*\%(\1\s*\)\{2,}$/ display

tests/smart.pdc

@@ -0,0 +1,95 @@
+# Smart Punctuation
+
+# Ellipses ...
+
+Ellipses ...
+----------
+
+Ellipses ...
+==========
+
+And then there was ...
+
+Blah ... blah.
+
+... and yet.
+...and yet.
+
+* I'm in a bullet...
+
+Endash --
+----------
+
+Endash --
+==========
+
+And then there was --
+
+Blah--blah.
+Blah -- blah.
+
+-- and yet.
+--and yet.
+
+* I'm in a bullet...
+
+Emdash ---
+---------
+
+Emdash ---
+==========
+
+And then there was ---
+
+Blah---blah.
+Blah --- blah.
+
+--- and yet.
+---and yet.
+
+# "Foo" is a thing
+
+A "Foo" is a thing
+------------------
+
+A "Foo" is a thing
+==================
+
+Just some "foo" here.
+Are you a "foo"?
+
+"Foo"
+*"Foo"*
+**"Foo"**
+_"Foo"_
+__"Foo"__
+~~"Foo"~~
+
+# 'Bar'
+
+'Bar'
+-----
+
+'Bar'
+=====
+
+Just some 'bar' here.
+Are you a 'bar'?
+
+'Bar'
+*'Bar'*
+**'Bar'**
+_'Bar'_
+__'Bar'__
+~~'Bar'~~
+
+# I'd Really Like to Know
+
+I'd Really Like to Know
+-----------------------
+
+I'd Really Like to Know
+=======================
+
+I'd
+'cuz