Remove global Namespace arena in favour of structural sharing. Refactor to use absolute paths.

[mitchmindtree]

Closes #682

Part 1. Gobal State Structural Sharing

This removes the global arena used for managing namespaces during type
checking in favour of reverting to the original approach of representing
new scopes by cloning the namespace.

Prior to the introduction of the global arena, cloning the namespace to
represent new scopes caused exponential memory growth as the entirety of
each of the inner collections was cloned along with the namespace
itself.

In this new approach, we aim to avoid this by using clone-friendly,
drop-in-replacement collections from the im crate that enable
"structural sharing" of the Namespace fields. See TODO below for
whether this is done yet or not.

You can read more about structural sharing here. The gist is that
rather than deep cloning the entire structure, we clone a reference to
the original state, and new allocations only occur upon modifying the
data structure with only the memory needed to record the difference.

The result is that we can avoid the side-effecting nature of maintaining
global state, making it easier to reason about how Namespaces are
accessed and updated within functions by reading their signatures.

It should also allow us to lean on RAII for cleaning up the namespace
contents, solving the bug where we are currently unable to compile the
same project more than once within the same process as mentioned
here. This is particularly relevant to sway-lsp where we will
likely want to lean on sway-core to type check the same code many
times within one process.

This also gets us a step toward achieving referential transparency for a
lot of our type checking functions which may enable easier caching and
parallelism in the future, though this would still currently be blocked
by the type engine's global state.

Many of the newer methods previously provided by NamespaceWrapper and
NamespaceRef have been moved to Namespace itself.

Part 2. Absolute Paths

The final set of changes in this PR are a refactor of the Namespace type in
order to ensure access to the whole module hierarchy (as well as the init
namespace) during all of type-checking.

Splitting Namespace into Root, Module and Items.

Previously, we used a single Namespace structure to represent not only the
module tree but the items within each module too.

This PR refactors the original Namespace type into:

• Module: Represents a single module. Stores the set of items available within
  the module as well as the set of submodules.
• Items: The set of declarations, implementations, synonyms and aliases
  available within a particular scope.

This allows for moving module-level behaviour onto the Module type, and
item-level behaviour to the Items type. This is particularly useful in
type-checking for distinguishing between whether or not we need the entire
module tree or just the set of items for a particular module (e.g. the
monomorphize methods).

The new Root type is a wrapper around the project's root Module. Methods
that should only be called on the root module (e.g. those that involve resolving
absolute paths, or internally resolving synonyms) have been moved onto this
type.

Module derefs to its inner Items, and Root derefs to its inner Module.
This is useful for retaining the ergonomics of the old Namespace type while
gaining the separation-of-concerns benefits of the new type separation in
function signatures.

Adding a new Namespace contextual type

A new higher-level Namespace type has been added that acts as a context,
making it easier to enter/exit submodules and scopes, and provides some
short-hand methods to avoid the need for repeatedly indexing into root with
the current module path.

The Namespace derefs to the Module that is currently being checked, allowing
to minimize the changes throughout the rest of type-checking.

TODO

• Replace collections in Namespace fields with clone-friendly
  alternatives provided by im.
• Solve E2E errors. Since the arena was introduced, a lot of new
  global mutation has also been introduced disguised by the RwLock.
  E.g. the monomorphize functions now mutate the namespace at the
  given reference, meaning that ns.resolve_self_* is also
  self-mutating. Need to review how these methods are used to ensure the
  new behaviour achieves the same intended results.

[mitchmindtree]

This is just about working! Almost all E2E tests pass besides:

• should_pass/language/dependencies and
• should_pass/language/multi_item_import.

The Problem

The reason why these don't pass is because while these changes do fix the issue where we could leak submodules into the crate root, they've also broken absolute paths.

In my refactor, I've made the incorrect assumption that the crate_namespace always referred to the initial state of the root of the namespace, i.e. containing library dependencies and perhaps one day the prelude items. However, it appears the intention for it is to simply act as an index into the root of the namespace (which will require mutation as modules and items are added to the root) so that absolute paths may be used to access items from within submodules.

Potential Solution

In order to address this, I think the correct approach is to pass around a &mut Namespace that always refers to the root namespace, along with a Vec<Ident> that represents the absolute path of the current scope or module.

More specifically my current plan is to remove the existing namespace: &mut Namespace and crate_namespace: &Namespace fields from the TypeCheckArguments, and replace them with:

    /// The root namespace containing all items and modules.
    root: &mut Namespace,
    /// An absolute path from the `root` to the namespace of the current scope.
    scope: Vec<Ident>,

We can use the scope to index into the namespace at the current scope as necessary, while retaining unique access to the root in case we visit an absolute path.

[mitchmindtree]

OK this should finally be good to go! I've updated the top-level comment to add a summary of the new "absolute path" changes as a second part, hopefully it helps to navigate this a little!

[mohammadfawaz]

I'm excited to learn about this change!

Haven't looked at the code yet but I just wanted to express my appreciation to your elaborate description for this PR. It will be extremely helpful for all of us working on the compiler 🙌

[otrho]

🤘

[sezna]

🚀