tilde expansion in path names

[StefanKarpinski]

@timholy implemented tilde_expand (see file.jl) some time ago and it is now used automatically by a few file-system commands, notably cd(). This concerns me, however, since while it's convenient for some things, it's inconsistently used, and it violates the treatment of strings as just data. What if the name of a file is just ~? We're now in a position where that name needs to be escaped and handled specially in code. On the other hand, tilde expansion is awfully handy.

So here's a possibility: bake tilde expansion into the backtick syntax instead. For example, we could make this work:

julia> run(`tail ~/.julia_history`)
           error("~user tilde expansion not yet implemented")
       end
tilde_expand("~")
tilde_expand("~/foo")
tilde_expand("~stefan/foo")
tilde_expand("~stefan")
`cat ~/.julia`
run(`cat ~/.julia`)
run(`tail ~/.julia_history`)
run(`tail $(ENV["HOME"])/.julia_history`)

Currently this just looks for a file literally named ~/.julia_history. Obviously, this would be handy for entering commands, but you could also use it in situations where you want shell-like treatment of any string. For example cd(~) would expand to cd(ENV["HOME"]) whereas cd("~") would treat its argument literally. Very importantly, tilde expansion would be done at compile time, not based on the run-time data — this is essential to avoiding nasty surprises and corner cases. In particular, interpolated values are not tilde expanded:

julia> `~`
`/Users/stefan`

julia> file = "~";

julia> `$file`
`~`

One corollary of this approach is that file and directory operations have to work with Cmd objects, which maybe ought to be renamed in light of their broadened use-cases under this proposal. Maybe they should be called ShellString objects since backtick strings are basically strings with shell-like behavior. Except that they're also not strings, so maybe that's not a great name either.

[timholy]

Just to make sure I understand, you're proposing that cd(/juliafunc) would take me to /home/tim/juliafunc, but that cd("~/juliafunc") would be a literal string and look for a subdirectory of the current directory named ""?

I kind of like it, it seems to offer control and flexibility. Especially since Windows temporary files seem to start with ~, I had to put an exception in tilde_expand for Windows. It would be good for the Windows folks to comment here, of course. @loladiro and @vtjnash ?

[StefanKarpinski]

Some interesting data points to consider about the shell's tilde expansion...

It only applies at the beginning of words:

bash-3.2$ echo ~
/Users/stefan
bash-3.2$ echo foo~
foo~
bash-3.2$ echo ~stefan
/Users/stefan
bash-3.2$ echo foo~stefan
foo~stefan

If the username does not exist, no expansion occurs:

bash-3.2$ echo ~foo
~foo

If the user name is interpolated, no expansion occurs:

bash-3.2$ var=stefan
bash-3.2$ echo ~$var
~stefan

I'm not sure we should mimic all of that, but it's worth considering.

[StefanKarpinski]

Just to make sure I understand, you're proposing that cd(~/juliafunc) would take me to /home/tim/juliafunc, but that cd("/juliafunc") would be a literal string and look for a subdirectory of the current directory named ""?

Yep, that's exactly what I'm proposing.

[Keno]

On windows ~ can either denote a short DOS name if used in the middle of a path segment and (as you mentioned) is also sometimes used to denote temporary files. I am a bit torn as to whether to allow tilde expansion to happen on Windows though I am tending towards allowing it.

[pao]

PowerShell allows ~ as an alias for $env:HOME (it's also able to do some fancier things).

[Keno]

Oh, I've actually never noticed that. In that case we should definitely allow it on windows.

[JeffBezanson]

Doing tilde expansion at compile time sounds like a terrible idea! To me that is absolutely wrong.

[StefanKarpinski]

The tilde expansion doesn't have to be done at compile time, but determining what is tilde expanded and what isn't would be done at compile time. So ~ could be expanded to be the equivalent of $(ENV["HOME"]). The point is that $var where var happens to be "~" should not cause tilde expansion.

[JeffBezanson]

Oh, carry on then :)

[timholy]

Stefan, I'm sure you already noticed this, but for the record: of your three points, the first matches the current behavior of tilde_expand, the second does not (and it might be better if we changed that, although it does lead to system-dependent results), and the third is essentially "no" but kindof orthogonal since it would be up to the caller.

But I am presuming that a new implementation would be needed with backticks, anyway.

[vtjnash]

bump. now that the file API cleanup happened [#1782], what is the status of this?

[diegozea]

Close related to this: #2761 (comment)

[sje30]

Bump. I'd really like to see ~ recognised within strings as the home directory;

cd(expanduser("~"))

just seems too clunky.