DataFrames.jl roadmap

[bkamins]

TO-DO for DataFrames.jl 1.0:

• crucial split-apply-combine decisions, by with arrays of inputs and broadcasting #1615, standardized return values for groupby #1554, check for preserved index column during combine #1460
• row-oriented API, Review row vs. column orientation of API #1514, Functions for column- and row-wise processing #956
• setindex! finish deprecation
• consistent broadcasting, broadcast in DataFrames.jl #1643
• eachcol cleanup, make eachcol default to false #1613, End deprecation period of eachcol #1752
• consistency of construction from DataFrameRow and NamedTuple, Construction from a named tuple documentation #1580, DataFrameRow conversion to DataFrame broken #1675, Converting to DataFrame fails for NamedTuple[] #1670, DataFrame constructors #1599, Support for construction from a named tuple? #1573, Creating an empty DataFrame is unwieldy and has unexpected behavior #1569, Implement a from_records constructor #1191
• decide on API provided, Deprecate length, nrow, and ncol on DataFrames in favor of size. Fixe… #1224, minor fixes #916, Decision: remove _names and make names not perform a copy #1655, Future of eltypes #1596, Future of colwise #1595, Remove nrow/ncol #406
• cartesian index for DataFrame, DataFrames should be indexable by CartesianIndex{2} #1610

TO-DO past DataFrames.jl 1.0 (but may go in before if ready)

• a better describe, Add custom functions to describe #1664, Add describe for GroupedDataFrame #1539, Make describe work for a grouped dataframe #1443, allow describe function to take arbitrary functions #1436
• vcat review, Flexible vcat #1659
• better benchmarks, WIP: add benchmarks #1626
• improvements in split-apply-combine, Add more splatting and appending of arguments in by #1620, sanitized by function #1555, Split-apply-combine todo #1616
• metadata, Continue adding Metadata to dataframes #1458, Metadata for columns and/or DataFrames #35
• pivot function, Added pivot() funtion (for updated pull request #1175) #1181
• sample function decision, ENH: Add sample function for DataFrames #997
• documentation improvements, New contents about handing missing values in DataFrame #1662
• change default show parameters, Use limit=false by default in show? #1631, Writing to latex: omitting column with row index #1381, Dataframes with wide columns have poor plain/text representations #1272, Markdown display #1167, Stop printing row numbers in show(io, df)? #864, Allow to hide row numbers #592
• get rid of deprecated code, Remove deprecated readtable/writetable #1602
• delete! improvements, functional interface for deleting columns #1378
• replace improvements, define Missings.replace method #1377
• add indexing, Provide indexing feature to allow for fast sort, join, and group-by operations #1346
• type-stable DataFrame, Base DataFrame on NamedTuple #1335, DataFrames replacement that has no type-uncertainty #744
• handling duplicate column names in join, Handling of duplicate column names by join #1333
• better categorical with Char, Categorical! to include Char #1282
• JSON interop, JSON to dataframe input and output #873
• lead/lag, Question: Lead and lag operators? #791
• better vcat: Add a function for combine/vcat with source names #659
• better reshaping, Output format of reshape functions #645
• cor for DataFrame, Implement Base.cor for DataFrame #583
• col names slicing, ENH: Allow use of colon operator to slice ranges by column names #393
• add lookup, Add a lookup function #120

CC @nalimilan @oxinabox

[oxinabox]

Update checklist re:

• eachcol cleanup, make eachcol default to false #1613

being done

[bkamins]

@nalimilan - apart from some recent PRs/Issues pending, there are two big outstanding things:

• crucial split-apply-combine decisions
• row-oriented API

before we can go 1.0 I think.

Both topics are your pet things, if I may say 😄. Do you have any opinion how we should move forward with it?

[bkamins]

OK - I am closing this as all things mentioned here are done (except for one small issue that is tracked).

Now we will track 1.0 milestone with milestone labels. You can find the list here https://github.com/JuliaData/DataFrames.jl/milestone/5.

In general we are really close to 0.20 release (probably a few days), and then final design decisions will be made before we fix the functionality for 1.0 release (the decisions are mainly in three areas: split-apply-combine, row-vs-column oriented API, and column transformation API, i.e. :oldcol => f => :newcol pattern).