Rationale

Magic incorporates ideas from a variety of other programming languages. This page summarises the key features of Magic, which make it distinctive. In combination, we believe this set of ideas is unique.

Lisp (Homoiconicity)

Magic is a fully fledged Lisp, in the sense that code is data - Magic code is expressed in the data structures of the language, and can be manipulated like any other data structure.

This provides a number of advantages:

• Syntax is kept extremely simple: it is just the structure of data
• It enables powerful macro capabilities, where code generation just requires creation of appropriate data structures
• Tools can work equally well with both code and data

For more reading on the advantages of Lisps, see:

• http://www.paulgraham.com/avg.html

JVM Ecosystem

The JVM provides an excellent, optimised, cross-platform runtime. It has probably the best optimised garbage collection algorithm of any comparable runtime system, a comprehensive standard library, and excellent near-native performance support by advanced JIT compilation. This alone makes it a compelling choice for implementing a new language, an effective route taken by Scala, Clojure, Kotlin and Groovy among others.

But perhaps even more importantly, compatibility with Java provides a huge ecosystem of powerful (and largely open source) libraries and tools. By leveraging this ecosystem, Magic gains substantial functionality and reach that would otherwise be unavailable to a new language that needs to develop its own ecosystem.

Functional programming

The benefits of functional programming are widely understood, for additional reading see e.g.:

• TODO

Dynamic, Interactive Development

We believe in the power of being able to interact directly with the running language environment. This eliminates the significant overhead edit/compile/run/test feedback loops, and facilitates very experimental, bottom up styles of software development.

Magic supports this by providing comprehensive REPL-style interactive development, where the user can interact with and modify the running program environment.

It is (experimentally) possible to use Magic via the Clojure nREPL protocol using the https://github.com/mikera/magic-clojure integration library.

Static Typing

We believe there is substantial value in a type system that analyses types at compile time. Key advantages include:

• Ability to catch logical errors in code that may cause unexpected errors at runtime (even worse: undetected errors that corrupt data). Not all bugs are type errors, but all type errors are potential bugs.
• Ability to create type-aware tools that provide better support to developers (see e.g. https://clojure.org/about/spec for an attempt to provide similar tooling support at runtime)
• Opportunities to improve performance via type specialisation and elimination of runtime checks.

At the same time, we want to maintain the feel of a dynamic language. Types shouldn't get in the way by imposing excess ceremony where they are not desired. Hence declaration of types in Magic is always optional from the perspective of the user.

Immutable Contexts

Conceptually, this is the biggest innovation in Magic. In order to explain this we need to define a Context as follows:

A context is the complete set of symbolic definitions present in the current state of the programming environment, where these definitions give meaning to expressions in the syntax of the programming language.

If we think about a Lisp expression such as (foo 1 2) it should be clear that this expression only has meaning if foo is defined. A context provides a set of such definition, so that symbols can be resolved and expressions can be evaluated. If a symbol is undefined, then evaluating such an expression naturally results in an error.

Languages that are fully compiled at startup effectively just have one big context that doesn't change, containing all the statically compiled definitions. This is a good solution for compiling and distributing binary code to users, but it prevents us as developers from having the style of interactive development that we want, since we need to allow the user to create and change definitions at runtime (e.g. via the REPL)

It is possible for a dynamic language to store all symbolic definitions in a big mutable data structure, updating it as we go. This is effectively what Clojure does, via namespaces. Execution of individual forms then results in changes to the mutable environment. This presents a number of issues however:

• It's a big blob of mutable global state. This is almost always a bad idea.
• Reasoning about the environment is almost impossible, if definitions can change at any time. e.g. this makes features such as static type checking very hard (impossible?) to implement effectively
• If you want to have different sub-environments (e.g. for jailed untrusted code), you can't let them access the global environment techniques.
• It's easy to mess up your environment or pollute it with a bunch of unwanted definitions. Time to reboot....