Made a tool to rename columns, want to integrate it

[culebron]

Example: I often have to export data from MongoDB where columns are extracted from an hierarchy, and have weird names.

mongoexport -d my_db -c my_collection --csv
       -f id,geopos.coordinates.0,geopos.coordinates.1,address > data.csv

There's no direct way in csvkit to rename those columns, only remove them completely or write a pipe, something like echo "column1,column2" > my_file.csv && head +2 >> my_file.csv.

I suggest to make a tool or an option to rename columns.

• Option 1: a parameter to csvcut:

   csvcut source.csv --rename-columns geopos.coordinates.0:x,geopos.coornidates.1:y > dest.csv
  

• Option 2: a separate tool:

   csvrename source.csv -c geopos.coordinates.0:x,geopos.coornidates.1:y > dest.csv
  

I have a working version of the latter in my repo. Please write what you think of this.

[jpmckinney]

Adding link to review: master...culebron:master

[jpmckinney]

How do you handle a column name containing a colon? (or whatever separator character you might switch to)

[jpmckinney]

For reference, earlier issues include #530. The above implementation is streaming (good). Just noting that the implementation of agate.Table.rename may be relevant.

[culebron]

Good question about the colon. Let me look into agate.Table.rename first. I looked at the tools and think we may just extend csvcut a bit, otherwise it's just too much pipelining and name repetition in the scripts.

My idea now is to add another parameter: --rename-columns old1:new1,old2:new2. I'll probably use backslash to quote colons and commas in the names.

[halloleo]

I'm in favour of a rename tool in csvkit! BTW, renaming columns is very close to adding columns - any ideas to integrate this functionality in the new tool? (I know about the adding via csvjoin, but it doesn't work that well.)

[jpmckinney]

@halloleo In what way does it not work well with csvjoin (as described in the docs)?

[halloleo]

I have to clarify: It does work as advertised, but it is a bit cumbersome: First you create a new column at the end,then yiou can move the column via a 2nd csvcut call into the order where you want it...

[jpmckinney]

Closing in favor of #396