Description
I love that Julia uses FemtoLisp for parsing code, but I think it could be a real boon if the parser were to be rewritten in pure Julia. We could use a cut down version of the current FemtoLisp parser as something like Core.parse to be used during bootstrap and then a Meta.parse which is fully implemented in Julia.
Why?
-
It would greatly lower the barrier of entry for people to provide contributions to the parser
-
Reader macros. Having a Julia parser would make it possible to define a form of macro where code inside the macro scope is parsed according to user defined rules. These macros would essentially be string macros except you would no longer need to wrap the enclosed code in quotation marks, giving you truly arbitrary syntax inside a macro but retaining things like code highlighting, auto indentation, autocompletion, etc. Depending on the design of the Julia parser, it could also potentially offer more support when building reader macros than we currently offer for building string macros.
Reader macros wouldn't really allow for new features over string macros per se, but I think the fact that string macro code needs to be inside a string substantially hampers their adoption for many potential applications, especially DSL implementation.
Reader Macro Syntax
I'm not deadset on any particular syntax to mark a reader macro, but something like !@ could be used.
As an example Py2Jl.jl transpiler interface would go from
using Py2Jl
py2jl"""
def sum_by(f, seq):
s = 0
for e in seq:
s = s + f(e)
return s
"""to
using Py2Jl
!@py2jl(
def sum_by(f, seq):
s = 0
for e in seq:
s = s + f(e)
return s
)Obviously, there would have to be certain limitations put on reader macros, ie. if we use parens !@( ... ) to enclose code, then the parser for the macro must respect parenthesis balancing to make sure it doesn't leak out of its scope.