Add '-o posix' shell option

[McDutchie]

Recent email correspondence with @kdudka has made me think that maybe, in making POSIX compliance front and centre to development here, I've been giving insufficient consideration to the fact that there is an installed base of set-and-forget ksh93 scripts still running, which may somehow depend on old non-compliant behaviour.

In principle, I'm still thinking that anyone who needs 100% bug compatibility should stick with exactly the version they are running. However, we can restore the documented or otherwise clearly intended non-compliant behaviour as part of an option that can be turned on and off.

Other shells such as bash, mksh and yash can do set -o posix to enable POSIX compliance mode. It is worth considering adding a similar option to ksh93. ksh should probably default to POSIX mode when invoked as sh, and to "legacy" otherwise. Having the posix option off would restore behaviour such as what has been fixed in 6a49720 (edit: nah, that's just pure breakage, and completely undocumented), eeee77e (edit: done in c607c48), and maybe 36da314 (edit: nah, that was just a bug).

It would also keep the Bourne shell compatibility hack for a literal [ -t ] or test -t being interpreted as [ -t 1 ]/test -t 1. Korn hacked the parser so that only a literal [ -t ] is changed and not something like x=-t; [ "$x" ]. Still, that hack violates POSIX and should be deactivated in any new POSIX mode. (edit: done in 55f0f8c)

Let's think about this for a while before doing anything. We can talk about it in this issue's thread, and figure out a list of things that need to be switched by such an option.

[McDutchie]

One other thing that a posix option should control is the interpretation of numbers with initial zeros as octal in the let builtin, as currently controlled by the letoctal option. The posix option should enable the behaviour of letoctal regardless of the setting of the latter.

It is a little ambiguous because let is not actually specified by POSIX. However, shell arithmetic is specified, and let processes exactly that: it is a regular shell grammar version of the ((…)) arithmetic command. So it's actually quite bizarre that let and ((...)) behave differently by default.

(edit: done in b301d41)

[JohnoKing]

The Bash compatibility mode has a posix option:

ksh/src/cmd/ksh93/data/options.c

Line 93 in 8b51d12

bashextra("posix", SH_POSIX)

ksh/src/cmd/ksh93/include/defs.h

Line 310 in 8b51d12

#define SH_POSIX 46

It was originally intended by the KSH authors to replicate the quirks present in the posix option Bash has, although this could also be used for implementing a backward compatibility mode with set +o posix.

[McDutchie]

$ ksh -c 'readonly foo=bar; unset -v foo; echo $?'
ksh: unset: warning: foo: is read only
1

POSIXly, unsetting a readonly variable is an error, not a warning (edit: at least I think so; the spec doesn't seem all that clear on it). So it should cause the special built-in command unset to exit, and never get to the echo command.

Fixing this unconditionally might cause incompatibilities with old scripts, so this is another one for the POSIX mode.

(edit: Never mind, this is nontrivial. I don't understand why, but changing the relevant ERROR_warning(0) into ERROR_exit(1) does not cause unset to exit the shell on error, even though it's defined as a special builtin. But mksh and bash don't exit either, and this is not important enough to spend more time on.)

[JohnoKing]

POSIX mode should be enabled if $POSIXLY_CORRECT is set (like in bash). This would make the following commands equivalent:

$ ksh -o posix
$ POSIXLY_CORRECT=1 ksh

[McDutchie]

I'm not really convinced making POSIXLY_CORRECT equivalent to set -o posix is a good idea; that variable is part of the GNU ecosystem, and ksh is not. Other systems like AIX and HP-UX have different variables for that, and Solaris has a different strategy altogether (POSIX compliant utilities are stored in /usr/xpg[764]/bin). Why should ksh prefer one over another?

As far as I know, bash is the only shell that honours any such variable, and it does it in a way that causes side effects and corner case bugs. The posix option on pdksh/mksh is just that: a shell option. I think that's a better way.

However, the posix option should probably be enabled by default if ksh is invoked as sh.

[McDutchie]

ksh closes globally open file descriptors > 2 when executing external commands. That behaviour violates POSIX, which states:

Utilities other than the special built-ins […] shall be invoked in a separate environment that consists of the following. The initial value of these objects shall be the same as that for the parent shell, except as noted below.

• Open files inherited on invocation of the shell, open files controlled by the exec special built-in plus any modifications, and additions specified by any redirections to the utility
• […]

Also, no other shell acts like this, not even mksh except mksh, and it disables that behaviour in its POSIX mode. So here's another one to fix for the POSIX mode.

This fix should probably be added to sh_redirect(); flag==2 should act like flag==1 when in POSIX mode.

(edit: done in fd97738)

[McDutchie]

POSIX mode should be enabled if $POSIXLY_CORRECT is set (like in bash).

I'm changing my opinion on this. Turns out that libast already supports this variable. If you use the getconf builtin and do getconf CONFORMANCE, then the output is normally ast, but if POSIXLY_CORRECT was set on invoking ksh, the output is standard. The bash emulation mode used this to initialise POSIX mode on init, so ksh 93u+m should probably do so as well.

What I'm not going to do is change the option dynamically along with the variable (and vice versa) as bash does; there be dragons.

(edit: done in 921bbca)

(edit 2: undone in 6affd23; see commit message)