Styling console/terminal output

[s-celles]

Hello,

Since v0.17.1, Pandas have an interesting API for styling Jupyter HTML outputs for Series / DataFrames...
https://pandas.pydata.org/pandas-docs/stable/style.html

It will be a nice feature to be able to have styling for console/terminal output
I wonder if such feature should be part of Pandas core or be inside an other package (like tabulate from @astanin which was mentioned in #11052

There is also PrettyPandas from @HHammond

Kind regards

[gfyoung]

@scls19fr : Thanks for the report! I think in the interest of modularity, we would probably want to move away from incorporating giant chunks of code into our codebase.

That being said, we could add functions that call those function from something like tabulate or PrettyPandas (assuming they're installed). However, we would probably need to choose one or the other (or make a uniform API to be able to call both).

[cbrnr]

RStudio adds color to the output of a data frame in the console:

Since IPython now supports syntax highlighting in a normal console, I was wondering if something similar was possible in pandas. This would likely not require any additional packages, but instead adding color codes to the __str__ or __repr__ methods could be all there needs to be done.

[jreback]

yes this certainly could be done. would likely take a separate formatting method (just to avoid having string / html / color strings code) in one place.

[cbrnr]

Great! What do you mean with separate method? Ideally, color strings should be used when I type df in IPython, so this has to be in __repr__, right?

[cbrnr]

ANSI escape color codes work, so it is pretty straightforward to get colorized output, e.g.:

The bigger questions are:

• Where do we want to add this functionality? Currently, I'm modifying the string representation in pandas.io.formats.format.to_string (before strcols get combined into text).
• What kind of implementation would be suitable? Do we add a helper function (e.g. colorize)?
• What are the defaults and how do you change them? E.g. there are white on black terminals, black on white terminals, and everything in between.

[TomAugspurger]

Where do we want to add this functionality?

Is this going to be doable with how to_string is currently implemented? From what I recall, that's a bit of a minefield.

Styler used jinja, but I'm not sure what our appetite is for adopting that as a full dependency.

[cbrnr]

I guess it's doable in to_string, but carrying around defaults and arguments might require some thinking. I wouldn't call it minefield, but the way the final string is constructed forced me to apply colors in the previous step (in the list of str_cols).

If there is an external tool that can do this coloring for us, we should use it if it means less work implementing it. It could be an optional dependency, in the sense that if it's not installed there will only be plain non-colored output.

[rswgnu]

@cbrnr wrote:

ANSI escape color codes work, so it is pretty straightforward to get colorized output

Say I wanted to colorize just specific table cells upon output to a terminal to highlight them, based on a list of iloc-compatible cell locations. How would I do that? I just need some pointers on what Pandas functions to tweak to allow this. Thanks.

[texxronn]

I could do somehting like

from colorama import Fore, Back, Style
df[c] = Fore.RED + Style.BRIGHT + df[c].astype(str) + Style.RESET_ALL
print (df)

[cbrnr]

Interesting. This was already suggested in #459, but never implemented.

[rswgnu]

Thanks for pointing me to this. It would be nice to have this as a core Pandas capability but at least terminal styling is possible with this package.

…

-- Bob

On Jun 12, 2019, at 10:55 PM, Rohan Machado ***@***.***> wrote: I could do somehting like from colorama import Fore, Back, Style df[c] = Fore.RED + Style.BRIGHT + df[c].astype(str) + Style.RESET_ALL print (df) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[ghost711]

I was looking for a way to use ANSI color codes in the terminal or qtconsole for a long time, and finally put this together. The main problem was that ANSI codes were being incorporated into the print width calculation, which messed up registration of the columns.

This is too hacky for a pull request, but it solves my problem, so I thought I'd post it in case it helps anyone else.

You can replace the "TextAdjustment" class with the version below in this file:
site-packages/pandas/io/formats/format.py

class TextAdjustment(object): 
    def __init__(self):
        import re
        self.ansi_regx = re.compile(r'\x1B[@-_][0-?]*[ -/]*[@-~]')
        self.encoding  = get_option("display.encoding")
    
    def len(self, text):  
        return compat.strlen(self.ansi_regx.sub('', text), 
                             encoding=self.encoding) 
            
    def justify(self, texts, max_len, mode='right'):       
        jfunc = str.ljust if (mode == 'left')  else \
                str.rjust if (mode == 'right') else str.center     
        out = [];  
        for s in texts:
            escapes = self.ansi_regx.findall(s)    
            if len(escapes) == 2:
                out.append(escapes[0].strip() + 
                           jfunc(self.ansi_regx.sub('', s), max_len) + 
                           escapes[1].strip()) 
            else:
                out.append(jfunc(s, max_len)) 
        return out;  
      
    def _join_unicode(self, lines, sep=''):
        try:
            return sep.join(lines)
        except UnicodeDecodeError:
            sep = compat.text_type(sep)
            return sep.join([x.decode('utf-8') if isinstance(x, str) else x
                                                            for x in lines])
    
    def adjoin(self, space, *lists, **kwargs): 
        # Add space for all but the last column: 
        pads = ([space] * (len(lists) - 1)) + [0] 
        max_col_len = max([len(col) for col in lists])
        new_cols = []
        for col, pad in zip(lists, pads): 
            width = max([self.len(s) for s in col]) + pad
            c     = self.justify(col, width, mode='left')
            # Add blank cells to end of col if needed for different col lens: 
            if len(col) < max_col_len:
                c.extend([' ' * width] * (max_col_len - len(col)))
            new_cols.append(c)
             
        rows = [self._join_unicode(row_tup) for row_tup in zip(*new_cols)] 
        return self._join_unicode(rows, sep='\n')