Skip to content

Commit

Permalink
Restore support for opts to Lexer.lex (#1178)
Browse files Browse the repository at this point in the history
Version 3.4.0 of Rouge moved from passing `opts[:continue]` to the class
method `.lex` to calling the new class method `.continue_lex`. As
implemented, the change was breaking and could cause Rouge to crash. 
This was not intentional and this commit reverts the change.
Internally, Rouge still uses `.continue_lex` but `.lex` will function as it
did in version 3.3.0. A warning is displayed for uses with `$VERBOSE` set
to true.
  • Loading branch information
pyrmont authored Jun 12, 2019
1 parent 6d62ba1 commit 6835352
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 6 deletions.
17 changes: 11 additions & 6 deletions lib/rouge/lexer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -417,16 +417,21 @@ def reset!

# Given a string, yield [token, chunk] pairs. If no block is given,
# an enumerator is returned.
def lex(string, opts=nil, &b)
if opts
warn 'the :continue option to Formatter#lex is deprecated, use #continue_lex instead.'
return continue_lex(string, &b)
#
# @option opts :continue
# Continue the lex from the previous state (i.e. don't call #reset!)
#
# @note The use of `opts` has been deprecated. A warning is issued if run
# with `$VERBOSE` set to true.
def lex(string, opts={}, &b)
unless opts.nil?
warn 'The use of opts with Lexer.lex is deprecated' if $VERBOSE
end

return enum_for(:lex, string) unless block_given?
return enum_for(:lex, string, opts) unless block_given?

Lexer.assert_utf8!(string)
reset!
reset! unless opts[:continue]

continue_lex(string, &b)
end
Expand Down
12 changes: 12 additions & 0 deletions spec/formatters/html_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,18 @@ class InlineTheme < Rouge::CSSTheme
end
end

describe 'format using lexer instance called with options' do
let(:text) { %(<meta name="description" content="foo">\n<script>alert("bar")</script>) }
let(:subject) { Rouge::Formatters::HTML.new }
let(:tokens) { Rouge::Lexers::HTML.new.lex text, {} }
let(:output) { subject.format(tokens) }

it 'should format token stream' do
assert { output == '<span class="nt">&lt;meta</span> <span class="na">name=</span><span class="s">"description"</span> <span class="na">content=</span><span class="s">"foo"</span><span class="nt">&gt;</span>
<span class="nt">&lt;script&gt;</span><span class="nx">alert</span><span class="p">(</span><span class="dl">"</span><span class="s2">bar</span><span class="dl">"</span><span class="p">)</span><span class="nt">&lt;/script&gt;</span>' }
end
end

describe 'tableized line numbers' do
let(:options) { { :line_numbers => true } }

Expand Down

0 comments on commit 6835352

Please sign in to comment.