Request for DataFrame.to_tsv() for reading tab delimited text

[dhimmel]

I propose a function, which can be called on a DataFrame, named to_tsv or to_table. The function is the equivalent of to_csv() with the argument sep='\t'. While to_tsv() contains the functionality to write tsv files, I find it annoying to always have to specify an additional argument. I prefer tsv files to csv files because tabs more rarely occur and therefore decrease the need for escaping. I also find the plain-text rendering more readable. I worry that the lack of a dedicated to_tsv() function encourages the use of csv over tsv. Currently read_table() defaults to tab separators, but there is no equivalent function for writing.

[dhimmel]

In addition to being just to_csv(sep='\t'), a to_tsv function should consider changing the default quoting, since quoting is less necessary for tsv files.

[shoyer]

The pandas API is already cluttered with an excess of rarely used convenience methods. I really don't think adding another one is a good idea.

[jorisvandenbossche]

I agree with @shoyer here. All functionality is there to do this within to_csv, and given we already have many methods, I think the reason to add a new one should be stronger than being able to provide other defaults.

I am closing this (we have too many open issues ..), but discussion can certainly continue if needed.

[DSLituiev]

+1. As practitioner, I would highly appreciate to_tab sugar.

[dhimmel]

I think the reason to add a new one should be stronger than being able to provide other defaults.

IMO convenience is worthy justification (people tend to write many text files, so to_csv has to constantly be supplemented with parameters).

However, my main motivation is a disdain for the CSV format. It pains me to see people still using CSV over TSV. Obviously excel/database support has a role to play. But a project like pandas should strive to make the best practices the easiest to implement.

[DSLituiev]

though this not a major issue for me currently, csv is US/Commonwealth representation-centric and internationally unaware. With all Pythonic philosophy of acceptance of UTF and internationalization, tab-separated must be preferred over csv / semicolon-sv.

[stevekm]

While I can understand the sentiment put forward by @shoyer, I agree with @dhimmel. It is my experience that TSV is much more of a standard format for data analysis than CSV. There are many use cases where the TSV format is a requirement, whereas I am not familiar with any for CSV format (there a couple examples of common usages here). TSV also has an advantage in that the raw text is easily readable, and avoids the issues with quoting as mentioned by @dhimmel.

[shoyer]

I am only slightly opposed to adding to_tsv. In my experience (in the US) CSV is more common than TSV (at least at the name for the file format), but only slightly. The main virtue to_tsv has going for it is that the name makes it instantly clear what it does.

[dhimmel]

CSV and TSV are both well supported and widely used in data science. CSV is more of a legacy format, thus many backwards-focused projects default to CSV. However, I think forwards-focused projects should default to TSV, as it's better for data science. Since there is no default to_text_delimited_file output function in pandas, to_csv is the de facto default. Since most users don't care enough to manually specify sep='\t', pandas is contributing to the prevalence of CSVs over TSVs and delaying the rise of the superior format.

[Starkiller4011]

Please excuse my ignorance on the matter but apart from being easier to read as a human, if and only if the column headers have roughly the same characters as their corresponding data which is not always the case, what advantages does TSV provide over CSV? Honestly curious if there is a performance difference between the two, I use TSV right now but honestly only because the data files I am working with came in that format so I left them in the same format.

[dhimmel]

what advantages does TSV provide over CSV?

@Starkiller4011 tabs are a more natural separator for columnar data. They require less quoting, since values rarely contain tabs but often contain commas.

Honestly curious if there is a performance difference between the two

I'd expect the performance difference is trivial. However, like most things in data science, the real type of performance that matters is programmer efficiency. And I think TSVs are nicer to work with than CSVs.

[dsm054]

Not everyone agrees that tab separation is superior to csv -- I don't, for example.

As Python programmers, we know that whitespace isn't always preserved across different operations, like copying and pasting. Those of us who answer a lot of questions on SO, for example, regularly have to use sep="\s\s+" to parse text which people have dumped in a whitespace-separated format, and we have to hope they've put enough spaces between columns for that to work. If they were using commas, or semicolons, or pipes, or something, this wouldn't be a problem. (And I just thought of carats, which used to be used pretty widely in some fields.)

If we want to add a to_tsv alias to make some people happier, okay. But let's not pretend that TSV doesn't have its own headaches when you're working with it, and the only advantage I can think of is less quoting.