Merge branch 'pchaigno-rmarkdown' into release-1.3.0

* pchaigno-rmarkdown:
  Case-insensitive on just markdown
  Perform case-insensitive test for file extensions
  Escape the filepath to rest2html so paths containing spaces will be passed to CommandImplementation#execute correctly.
  Update README.md
  Tables are cooler
  Organize the README's whitelisted elements a bit better
  Render RMarkdown (.rmd) as Markdown
  Force encoding of posix_spawn output
  Force encoding of posix_spawn output
  Fix a spelling mistake
  rst2html: Extend the field_name_limit to 50

Author: bkeepers

README.md

@@ -2,7 +2,7 @@ GitHub Markup
 =============
 
 We use this library on GitHub when rendering your README or any other
-rich text file. The generated HTML is then run through filters in the [html-pipeline](https://github.com/jch/html-pipeline) to perform things like [santization](#html-sanitization) and [syntax highlighting](https://github.com/jch/html-pipeline/blob/master/lib/html/pipeline/syntax_highlight_filter.rb).
+rich text file. The generated HTML is then run through filters in the [html-pipeline](https://github.com/jch/html-pipeline) to perform things like [sanitization](#html-sanitization) and [syntax highlighting](https://github.com/jch/html-pipeline/blob/master/lib/html/pipeline/syntax_highlight_filter.rb).
 
 Markups
 -------
@@ -37,32 +37,36 @@ Or, more realistically:
     require 'github/markup'
     GitHub::Markup.render(file, File.read(file))
 
+Contributing
+------------
+
+See [Contributing](CONTRIBUTING.md)
+
 HTML sanitization
 -----------------
 
 HTML rendered by the various markup language processors gets passed through an [HTML sanitization filter](https://github.com/jch/html-pipeline/blob/master/lib/html/pipeline/sanitization_filter.rb) for security reasons. HTML elements not in the whitelist are removed. HTML attributes not in the whitelist are removed from the preserved elements.
 
 The following HTML elements, organized by category, are whitelisted:
 
-* Headings: h1, h2, h3, h4, h5, h6, h7, h8
-* Prose: p, div, blockquote
-* Preformatted: pre
-* Inline: b, i, strong, em, tt, code, ins, del, sup, sub, kbd, samp, q, var
-* Lists: ol, ul, li, dl, dt, dd
-* Tables: table, thead, tbody, tfoot, tr, td, th
-* Breaks: br, hr
-* Ruby (East Asian): ruby, rt, rp
+|Type | Elements
+|------|----------
+|Headings | `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `h7`, `h8`
+|Prose |  `p`, `div`, `blockquote`
+|Formatted | `pre`
+| Inline | `b`, `i`, `strong`, `em`, `tt`, `code`, `ins`, `del`, `sup`, `sub`, `kbd`, `samp`, `q`, `var`
+| Lists | `ol`, `ul`, `li`, `dl`, `dt`, `dd`
+| Tables | `table`, `thead`, `tbody`, `tfoot`, `tr`, `td`, `th`
+| Breaks | `br`, `hr`
+| Ruby (East Asian) | `ruby`, `rt`, `rp`
 
 The following attributes, organized by element, are whitelisted:
 
-* a: href (http://, https://, mailto://, github-windows:// and github-mac:// URI schemes and relative paths only)
-* img: src (http:// and https:// URI schemes and relative paths only)
-* div: itemscope, itemtype
-* all: abbr, accept, accept-charset, accesskey, action, align, alt, axis, border, cellpadding, cellspacing, char, charoff, charset, checked, cite, clear, cols, colspan, color, compact, coords, datetime, dir, disabled, enctype, for, frame, headers, height, hreflang, hspace, ismap, label, lang, longdesc, maxlength, media, method, multiple, name, nohref, noshade, nowrap, prompt, readonly, rel, rev, rows, rowspan, rules, scope, selected, shape, size, span, start, summary, tabindex, target, title, type, usemap, valign, value, vspace, width, itemprop
+|Element | Attributes
+|------|----------
+| `a` | `href` (`http://`, `https://`, `mailto://`, `github-windows://`, and `github-mac://` URI schemes and relative paths only)
+| `img` | `src` (`http://` and `https://` URI schemes and relative paths only)
+| `div` | `itemscope`, `itemtype`
+| All | `abbr`, `accept`, `accept-charset`, `accesskey`, `action`, `align`, `alt`, `axis`, `border`, `cellpadding`, `cellspacing`, `char`, `charoff`, `charset`, `checked`, `cite`, `clear`, `cols`, `colspan`, `color`, `compact`, `coords`, `datetime`, `dir`, `disabled`, `enctype`, `for`, `frame`, `headers`, `height`, `hreflang`, `hspace`, `ismap`, `label`, `lang`, `longdesc`, `maxlength`, `media`, `method`, `multiple`, `name`, `nohref`, `noshade`, `nowrap`, `prompt`, `readonly`, `rel`, `rev`, `rows`, `rowspan`, `rules`, `scope`, `selected`, `shape`, `size`, `span`, `start`, `summary`, `tabindex`, `target`, `title`, `type`, `usemap`, `valign`, `value`, `vspace`, `width`, `itemprop`
 
-Note that the id attribute is *not* whitelisted.
-
-Contributing
-------------
-
-See [Contributing](CONTRIBUTING.md)
+Note that the `id` attribute is *not* whitelisted.

lib/github/commands/rest2html

@@ -57,7 +57,8 @@ SETTINGS = {
     'initial_header_level': 2,
     'report_level': 5,
     'syntax_highlight' : 'none',
-    'math_output' : 'latex'
+    'math_output' : 'latex',
+    'field_name_limit': 50,
 }
 
 class GitHubHTMLTranslator(HTMLTranslator):

lib/github/markup/command_implementation.rb

@@ -35,7 +35,7 @@ def call_block(rendered, content)
       def execute(command, target)
         spawn = POSIX::Spawn::Child.new(*command, :input => target)
         if spawn.status.success?
-          spawn.out.gsub("\r", '')
+          spawn.out.gsub("\r", '').force_encoding(target.encoding)
         else
           raise CommandError.new(spawn.err.strip)
         end

lib/github/markup/markdown.rb

@@ -25,7 +25,7 @@ class Markdown < Implementation
       }
 
       def initialize
-        super(/md|mkdn?|mdwn|mdown|markdown|litcoffee/)
+        super(/md|rmd|mkdn?|mdwn|mdown|markdown|litcoffee/i)
       end
 
       def load

lib/github/markups.rb

@@ -1,4 +1,5 @@
 require "github/markup/markdown"
+require "shellwords"
 
 markups << GitHub::Markup::Markdown.new
 
@@ -29,7 +30,7 @@
   Asciidoctor.render(content, :safe => :secure, :attributes => %w(showtitle idprefix idseparator=- env=github env-github source-highlighter=html-pipeline))
 end
 
-command("python2 -S #{File.dirname(__FILE__)}/commands/rest2html", /re?st(\.txt)?/)
+command("python2 -S #{Shellwords.escape(File.dirname(__FILE__))}/commands/rest2html", /re?st(\.txt)?/)
 
 # pod2html is nice enough to generate a full-on HTML document for us,
 # so we return the favor by ripping out the good parts.

test/markup_test.rb

@@ -1,3 +1,5 @@
+# encoding: UTF-8
+
 $LOAD_PATH.unshift File.dirname(__FILE__) + "/../lib"
 
 require 'github/markup'
@@ -35,6 +37,8 @@ class MarkupTest < Test::Unit::TestCase
   def test_knows_what_it_can_and_cannot_render
     assert_equal false, GitHub::Markup.can_render?('README.html')
     assert_equal true, GitHub::Markup.can_render?('README.markdown')
+    assert_equal true, GitHub::Markup.can_render?('README.rmd')
+    assert_equal true, GitHub::Markup.can_render?('README.Rmd')
     assert_equal false, GitHub::Markup.can_render?('README.cmd')
     assert_equal true, GitHub::Markup.can_render?('README.litcoffee')
   end
@@ -50,4 +54,9 @@ def test_raises_error_if_command_exits_non_zero
       fail "an exception was expected but was not raised"
     end
   end
+
+  def test_preserve_markup
+    content = "Noël"
+    assert_equal content.encoding.name, GitHub::Markup.render('Foo.rst', content).encoding.name
+  end
 end

test/markups/README.rmd

@@ -0,0 +1,3 @@
+# Title
+* One
+* Two

test/markups/README.rmd.html

@@ -0,0 +1,6 @@
+<h1>Title</h1>
+
+<ul>
+<li>One</li>
+<li>Two</li>
+</ul>

test/markups/README.rst

@@ -46,3 +46,12 @@ Source          https://github.com/tony/pullv
 
 .. image:: https://scan.coverity.com/projects/621/badge.svg
 	:alt: Coverity Scan Build Status
+
+Field list
+----------
+
+:123456789 123456789 123456789 123456789 123456789 1: Uh-oh! This name is too long!
+:123456789 123456789 123456789 123456789 1234567890: this is a long name,
+	but no problem!
+:123456789 12345: this is not so long, but long enough for the default!
+:123456789 1234: this should work even with the default :)

test/markups/README.rst.html

@@ -5,6 +5,7 @@ <h2 class="subtitle" id="subtitle">Subtitle</h2>
 <p class="topic-title first">Table of Contents</p>
 <ul class="simple">
 <li><a class="reference internal" href="#header-2" id="id1">Header 2</a></li>
+<li><a class="reference internal" href="#field-list" id="id2">Field list</a></li>
 </ul>
 </div>
 <h2><a class="toc-backref" href="#id1">Header 2</a></h2>
@@ -40,3 +41,21 @@ <h2><a class="toc-backref" href="#id1">Header 2</a></h2>
 <a class="reference external image-reference" href="https://scan.coverity.com/projects/621"><img alt="Coverity Scan Build Status" src="https://scan.coverity.com/projects/621/badge.svg">
 </a>
 <img alt="Coverity Scan Build Status" src="https://scan.coverity.com/projects/621/badge.svg">
+<h2><a class="toc-backref" href="#id2">Field list</a></h2>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field"><th class="field-name" colspan="2">123456789 123456789 123456789 123456789 123456789 1:</th></tr>
+<tr class="field"><td>&nbsp;</td><td class="field-body">Uh-oh! This name is too long!</td>
+</tr>
+<tr class="field"><th class="field-name">123456789 123456789 123456789 123456789 1234567890:</th><td class="field-body">this is a long name,
+but no problem!</td>
+</tr>
+<tr class="field"><th class="field-name">123456789 12345:</th><td class="field-body">this is not so long, but long enough for the default!</td>
+</tr>
+<tr class="field"><th class="field-name">123456789 1234:</th><td class="field-body">this should work even with the default :)</td>
+</tr>
+</tbody>
+</table>
+