Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for rich display of Markdown cells #2346

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
273bbb3
Adding support for rich display of Markdown cells
NicholasWMRitchie Aug 3, 2020
e72d033
Fixing Markdown as "text/csv" and "text/plain"
NicholasWMRitchie Aug 4, 2020
a1d9619
Fix test error in Markdown to HTML in 32-bit systems
NicholasWMRitchie Aug 4, 2020
57a3b58
Fixing show and adding multi-codepoint tests
NicholasWMRitchie Aug 6, 2020
531e089
32 it is...
NicholasWMRitchie Aug 6, 2020
ea571e6
Use chomp to remove trailing '\n
NicholasWMRitchie Aug 7, 2020
c53598e
Fixing conflicts
NicholasWMRitchie Aug 24, 2020
5df96a6
Merge branch 'master' into master
bkamins Aug 24, 2020
a391912
Don't match on AbstractVector type parameter due to compiler crash (#…
quinnj Aug 24, 2020
5f5f121
Pulling df[i,j] out and optimizing ourshow.
NicholasWMRitchie Aug 25, 2020
73a9010
Merge branch 'master' of https://github.com/NicholasWMRitchie/DataFra…
NicholasWMRitchie Aug 25, 2020
3a85fbf
Formatting improvements
NicholasWMRitchie Aug 25, 2020
8928695
Additional formatting improvements
NicholasWMRitchie Aug 25, 2020
1a0fd3b
Update CONTRIBUTING.md
bkamins Aug 27, 2020
c1e38d4
Update CONTRIBUTING.md
bkamins Aug 27, 2020
be35852
Fixing triple-quoted strings in 1.0.X
NicholasWMRitchie Aug 27, 2020
7d4e5d0
Strip space in test case
NicholasWMRitchie Aug 27, 2020
2f5c2d3
Using @nalimilan suggestion
NicholasWMRitchie Aug 27, 2020
c05e3ea
Merge branch 'master' of https://github.com/NicholasWMRitchie/DataFra…
NicholasWMRitchie Aug 27, 2020
b90696d
Improving LaTeX output appearance, NEWS.md entry
NicholasWMRitchie Aug 28, 2020
606f3eb
Moved NEWS.md item to "Other relevant changes" section.
NicholasWMRitchie Aug 28, 2020
174d632
Correct NEWS.md item on rich display support
NicholasWMRitchie Aug 28, 2020
5414031
Adding support for rich display of Markdown cells
NicholasWMRitchie Aug 3, 2020
17b6e1b
Fixing Markdown as "text/csv" and "text/plain"
NicholasWMRitchie Aug 4, 2020
7a8c0de
Fix test error in Markdown to HTML in 32-bit systems
NicholasWMRitchie Aug 4, 2020
c112ef4
Fixing show and adding multi-codepoint tests
NicholasWMRitchie Aug 6, 2020
8a04964
32 it is...
NicholasWMRitchie Aug 6, 2020
9fbe264
Use chomp to remove trailing '\n
NicholasWMRitchie Aug 7, 2020
43a047c
Fixing conflicts
NicholasWMRitchie Aug 24, 2020
8ff0ff0
Pulling df[i,j] out and optimizing ourshow.
NicholasWMRitchie Aug 25, 2020
e67308e
Formatting improvements
NicholasWMRitchie Aug 25, 2020
ab4209c
Additional formatting improvements
NicholasWMRitchie Aug 25, 2020
4993be3
Fixing triple-quoted strings in 1.0.X
NicholasWMRitchie Aug 27, 2020
feb0f5d
Strip space in test case
NicholasWMRitchie Aug 27, 2020
bbd5143
Using @nalimilan suggestion
NicholasWMRitchie Aug 27, 2020
088c894
Improving LaTeX output appearance, NEWS.md entry
NicholasWMRitchie Aug 28, 2020
6a2ed0d
Moved NEWS.md item to "Other relevant changes" section.
NicholasWMRitchie Aug 28, 2020
b3866e5
Correct NEWS.md item on rich display support
NicholasWMRitchie Aug 28, 2020
0f6fb38
Merge remote-tracking branch 'origin/master'
NicholasWMRitchie Aug 28, 2020
1342b4b
Removing duplicate using lines
NicholasWMRitchie Aug 28, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ DataAPI = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
Future = "9fa8497b-333b-5362-9e8d-4d0656e87820"
InvertedIndices = "41ab1584-1d38-5bbf-9106-f11c6c58b48f"
IteratorInterfaceExtensions = "82899510-4779-5014-852e-03e436cf321d"
Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
Missings = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
PooledArrays = "2dfb63ee-cc39-5dd5-95bd-886bf059d720"
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
Expand Down
1 change: 1 addition & 0 deletions src/DataFrames.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ using Reexport, SortingAlgorithms, Compat, Unicode, PooledArrays
@reexport using CategoricalArrays, Missings, InvertedIndices
using Base.Sort, Base.Order, Base.Iterators
using TableTraits, IteratorInterfaceExtensions
using Markdown

import DataAPI,
DataAPI.All,
Expand Down
13 changes: 12 additions & 1 deletion src/abstractdataframe/io.jl
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ function _show(io::IO, ::MIME"text/html", df::AbstractDataFrame;
cell_val = df[row, column_name]
if ismissing(cell_val)
write(io, "<td><em>missing</em></td>")
elseif cell_val isa Markdown.MD
write(io, "<td>")
show(io, "text/html", cell_val)
write(io, "</td>")
elseif cell_val isa SHOW_TABULAR_TYPES
write(io, "<td><em>")
cell = sprint(ourshow, cell_val)
Expand Down Expand Up @@ -303,6 +307,8 @@ function _show(io::IO, ::MIME"text/latex", df::AbstractDataFrame;
cell = df[row,col]
if ismissing(cell)
print(io, "\\emph{missing}")
elseif cell isa Markdown.MD
show(io, "text/latex", cell)
elseif cell isa SHOW_TABULAR_TYPES
print(io, "\\emph{")
print(io, latex_escape(sprint(ourshow, cell, context=io)))
Expand Down Expand Up @@ -417,7 +423,12 @@ function printtable(io::IO,
elseif isnothing(df[i, j])
print(io, nothingstring)
else
if ! (etypes[j] <: Real)
if df[i,j] isa Markdown.MD
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since accessing df[i, j] is relatively slow, better store it in cell = df[i, j] and use that afterwards as in the HTML method (not sure why we didn't do this here).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it, nor the other changes you mention below. :-)

print(io, quotemark)
r=repr(df[i,j])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
r=repr(df[i,j])
r = repr(df[i,j])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

escapedprint(io, first(r,max(0,length(r)-1)), quotestr)
NicholasWMRitchie marked this conversation as resolved.
Show resolved Hide resolved
print(io, quotemark)
elseif ! (etypes[j] <: Real)
print(io, quotemark)
escapedprint(io, df[i, j], quotestr)
print(io, quotemark)
Expand Down
5 changes: 5 additions & 0 deletions src/abstractdataframe/show.jl
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ ourshow(io::IO, x::Symbol) = ourshow(io, string(x))
ourshow(io::IO, x::Nothing; styled::Bool=false) = ourshow(io, "", styled=styled)
ourshow(io::IO, x::SHOW_TABULAR_TYPES; styled::Bool=false) =
ourshow(io, summary(x), styled=styled)
function ourshow(io::IO, x::Markdown.MD)
r = repr(x)
len = min(length(r, 1, something(findfirst(c->c=='\n', r), lastindex(r)+1)-1), 50)
bkamins marked this conversation as resolved.
Show resolved Hide resolved
return print(io, len < length(r) - 1 ? first(r, len)*"…" : first(r, len))
bkamins marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using chars should give a slightly more efficient code:

Suggested change
return print(io, len < length(r) - 1 ? first(r, len)*"" : first(r, len))
return print(io, len < length(r) - 1 ? first(r, len)*'' : first(r, len))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

end

# AbstractChar: https://github.com/JuliaLang/julia/pull/34730 (1.5.0-DEV.261)
# Irrational: https://github.com/JuliaLang/julia/pull/34741 (1.5.0-DEV.266)
Expand Down
145 changes: 131 additions & 14 deletions test/io.jl
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module TestIO

using Test, DataFrames, CategoricalArrays, Dates
using Test, DataFrames, CategoricalArrays, Dates, Markdown

# Test LaTeX export
@testset "LaTeX export" begin
Expand All @@ -9,20 +9,21 @@ using Test, DataFrames, CategoricalArrays, Dates
C = ["A", "B", "C", "S"],
D = [1.0, 2.0, missing, 3.0],
E = CategoricalArray(["a", missing, "c", "d"]),
F = Vector{String}(undef, 4)
F = Vector{String}(undef, 4),
G = [ md"[DataFrames.jl](http://juliadata.github.io/DataFrames.jl)", md"###A", md"``\frac{A}{B}``", md"*A*b**A**"]
)
str = """
\\begin{tabular}{r|cccccc}
\t& A & B & C & D & E & F\\\\
\t\\hline
\t& $(Int) & String & String & Float64? & Cat…? & String\\\\
\t\\hline
\t1 & 1 & \\\$10.0 & A & 1.0 & a & \\emph{\\#undef} \\\\
\t2 & 2 & M\\&F & B & 2.0 & \\emph{missing} & \\emph{\\#undef} \\\\
\t3 & 3 & A\\textasciitilde{}B & C & \\emph{missing} & c & \\emph{\\#undef} \\\\
\t4 & 4 & \\textbackslash{}\\textbackslash{}alpha & S & 3.0 & d & \\emph{\\#undef} \\\\
\\end{tabular}
"""
str =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you keep the triple quoted form? It's more readable. Same below.

"\\begin{tabular}{r|ccccccc}\n" *
"\t& A & B & C & D & E & F & G\\\\\n" *
"\t\\hline\n\t& "*repr(Int)*" & String & String & Float64? & Cat…? & String & MD…\\\\\n" *
"\t\\hline\n" *
"\t1 & 1 & \\\$10.0 & A & 1.0 & a & \\emph{\\#undef} & \\href{http://juliadata.github.io/DataFrames.jl}{DataFrames.jl}\n\n \\\\\n" *
"\t2 & 2 & M\\&F & B & 2.0 & \\emph{missing} & \\emph{\\#undef} & \\#\\#\\#A\n\n \\\\\n" *
"\t3 & 3 & A\\textasciitilde{}B & C & \\emph{missing} & c & \\emph{\\#undef} & \$\\frac{A}{B}\$\n\n \\\\\n" *
"\t4 & 4 & \\textbackslash{}\\textbackslash{}alpha & S & 3.0 & d & \\emph{\\#undef} & \\emph{A}b\\textbf{A}\n\n \\\\\n" *
"\\end{tabular}\n"


@test repr(MIME("text/latex"), df) == str
@test repr(MIME("text/latex"), eachcol(df)) == str
@test repr(MIME("text/latex"), eachrow(df)) == str
Expand Down Expand Up @@ -130,6 +131,27 @@ end

@test_throws ArgumentError DataFrames._show(stdout, MIME("text/html"),
DataFrame(ones(2,2)), rowid=10)

df = DataFrame(
A=Int64[1,4,9,16],
B = [
md"[DataFrames.jl](http://juliadata.github.io/DataFrames.jl)",
md"###A",
md"``\frac{A}{B}``",
md"*A*b**A**" ]
)

@test repr(MIME("text/html"), df) ==
NicholasWMRitchie marked this conversation as resolved.
Show resolved Hide resolved
"<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th></tr><tr><th></th>" *
"<th>Int64</th><th>MD…</th></tr></thead><tbody><p>4 rows × 2 columns</p><tr><th>1</th>" *
"<td>1</td><td><div class=\"markdown\">" *
"<p><a href=\"http://juliadata.github.io/DataFrames.jl\">DataFrames.jl</a>" *
"</p>\n</div></td></tr><tr><th>2</th><td>4</td><td><div class=\"markdown\">" *
"<p>###A</p>\n</div></td></tr><tr><th>3</th><td>9</td><td><div class=\"markdown\">" *
"<p>&#36;\\frac&#123;A&#125;&#123;B&#125;&#36;</p>\n</div></td></tr><tr><th>4</th>" *
"<td>16</td><td><div class=\"markdown\"><p><em>A</em>b<strong>A</strong></p>"*
"\n</div></td></tr></tbody></table>"

end

# test limit attribute of IOContext is used
Expand Down Expand Up @@ -180,6 +202,101 @@ end
end
end

@testset "Markdown as text/plain and as text/csv" begin
df = DataFrame(
A=Int64[1,4,9,16,25,36,49,64],
B = [
md"[DataFrames.jl](http://juliadata.github.io/DataFrames.jl)",
md"``\frac{x^2}{x^2+y^2}``",
md"# Header",
md"This is *very*, **very**, very, very, very, very, very, very, very long line" ,
md"",
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0" *
"∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ\n"*
" * ∞7∫αγ\n"*
" * ∞8∫αγ\n"*
" * ∞9∫αγ∞0∫α\nγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫α\n"*
" * γ∞1∫α\n"*
" * γ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
]
)
@test sprint(show, "text/plain", df) == """
8×2 DataFrame
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add 8-characters indentation. Same below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one still applies.

│ Row │ A │ B │
│ │ Int64 │ Markdown.MD │
├─────┼───────┼─────────────────────────────────────────────────────┤
│ 1 │ 1 │ [DataFrames.jl](http://juliadata.github.io/DataFra… │
│ 2 │ 4 │ \$\\frac{x^2}{x^2+y^2}\$ │
│ 3 │ 9 │ # Header │
│ 4 │ 16 │ This is *very*, **very**, very, very, very, very, … │
│ 5 │ 25 │ │
│ 6 │ 36 │ ∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0… │
│ 7 │ 49 │ ∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ… │
│ 8 │ 64 │ ∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0… │"""

@test sprint(show, "text/csv", df) == """
\"A\",\"B\"
1,\"[DataFrames.jl](http://juliadata.github.io/DataFrames.jl)\"
4,\"\$\\\\frac{x^2}{x^2+y^2}\$\"
9,\"# Header\"
16,\"This is *very*, **very**, very, very, very, very, very, very, very long line\"
25,\"\"
36,\"∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0\"
49,\"∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ\\n\\n * ∞7∫αγ\\n * ∞8∫αγ\\n * ∞9∫αγ∞0∫α\\n\\nγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0\"
64,\"∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫α\\n\\n * γ∞1∫α\\n * γ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0\"
"""
end

@testset "Markdown as HTML" begin
df = DataFrame(
A=Int64[1,4,9,16,25,36,49,64],
B = [
md"[DataFrames.jl](http://juliadata.github.io/DataFrames.jl)",
md"``\frac{x^2}{x^2+y^2}``",
md"# Header",
md"This is *very*, **very**, very, very, very, very, very, very, very long line" ,
md"",
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0" *
"∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ\n"*
" * ∞7∫αγ\n"*
" * ∞8∫αγ\n"*
" * ∞9∫αγ∞0∫α\nγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
Markdown.parse("∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫α\n"*
" * γ∞1∫α\n"*
" * γ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0"),
]
)
@test sprint(show,"text/html",df) ==
"<table class=\"data-frame\"><thead>" *
"<tr><th></th><th>A</th><th>B</th></tr>" *
"<tr><th></th><th>Int64</th><th>MD…</th></tr>" *
"</thead>" *
"<tbody>" * "<p>8 rows × 2 columns</p>" *
"<tr><th>1</th><td>1</td><td><div class=\"markdown\">" *
"<p><a href=\"http://juliadata.github.io/DataFrames.jl\">DataFrames.jl</a></p>\n</div></td></tr>" *
"<tr><th>2</th><td>4</td><td><div class=\"markdown\"><p>&#36;\\frac&#123;x^2&#125;&#123;x^2&#43;y^2&#125;&#36;</p>\n</div></td></tr>" *
"<tr><th>3</th><td>9</td><td><div class=\"markdown\"><h1>Header</h1>\n</div></td></tr>" *
"<tr><th>4</th><td>16</td><td><div class=\"markdown\">" *
"<p>This is <em>very</em>, <strong>very</strong>, very, very, very, very, very, very, very long line</p>\n" *
"</div></td></tr>" *
"<tr><th>5</th><td>25</td><td><div class=\"markdown\"></div></td></tr>" *
"<tr><th>6</th><td>36</td><td><div class=\"markdown\">" *
"<p>∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0</p>\n" *
"</div></td></tr>" *
"<tr><th>7</th><td>49</td><td><div class=\"markdown\">" *
"<p>∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ</p>\n<ul>\n<li><p>∞7∫αγ</p>\n</li>\n<li><p>∞8∫αγ</p>\n</li>\n<li><p>∞9∫αγ∞0∫α</p>\n</li>\n</ul>\n<p>γ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0</p>\n" *
"</div></td></tr>" *
"<tr><th>8</th><td>64</td><td><div class=\"markdown\">" *
"<p>∫αγ∞1∫αγ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0∫α</p>" *
"\n<ul>\n" *
"<li><p>γ∞1∫α</p>\n</li>\n" *
"<li><p>γ∞2∫αγ∞3∫αγ∞4∫αγ∞5∫αγ∞6∫αγ∞7∫αγ∞8∫αγ∞9∫αγ∞0</p>\n</li>\n" *
"</ul>\n" * "</div></td></tr></tbody></table>"
end

@testset "empty data frame and DataFrameRow" begin
df = DataFrame(a = [1,2], b = [1.0, 2.0])

Expand Down