automatic color diagnostics breaks terminfo configuration

[llvmbot]

Bugzilla Link	25007
Version	3.7
OS	Linux
Reporter	LLVM Bugzilla Contributor
CC	@foutrelis,@jsonn

Extended Description

When there is no -f[no-]color-diagnostics in the command line, the clang driver will try to auto-detect if the terminal supports color text. This auto-detection resets any terminfo configuration that may be currently configured.

In the case of /usr/bin/clang that is not a problem, but when it is used as a library very bad things will happen, for example with Vim and the Clang-Complete plugin 1.

I've traced the problem to the function terminalHasColors(int fd) in lib/Support/Unix/Process.inc: it calls setupterm() and then set_curterm() and del_curterm(), thus disabling any terminal configuration.

My suggestion is to call first set_curterm(): if it returns null then terminfo is not initialized and it can be done as now. But if it returns not-null then terminfo is already initialized and you do not need to call set_curterm() or del_curterm().

[llvmbot]

[PATCH] Save and restore current term setting in terminalHasColors().
Proposed fix for the reported terminal issue.

• Save the current term settings before doing the check for color support.
• Restore the term settings after the tests (or if the test fails).

This fixes the reported issue for me (with vim-clang-complete), and keeps the original terminal color detection working (-f[no-]color-diagnostics).

I chose to duplicate the cleanup logic (set_curterm(OrigTerm)) twice (once on the error path and once on the normal path) instead of restructuring the function to keep the patch simpler.

Another option would be to use the already set up term in case set_curterm returns non-NULL term, but I felt this approach is safer (and closer to the original logic).

[llvmbot]

phabricator link: http://reviews.llvm.org/D15077

[AaronBallman]

Closing; I believe this issue is now resolved as terminalHasColors() no longer tries to save and restore terminal information:

llvm-project/llvm/lib/Support/Unix/Process.inc

Line 334 in 5e101de

static bool terminalHasColors() {