Merge pull request #11442 from ethereum/path-resolution-docs-current-state-v2

Path resolution docs (current state only) v2

Author: chriseth

docs/index.rst

@@ -165,6 +165,7 @@ Contents
    security-considerations.rst
    smtchecker.rst
    resources.rst
+   path-resolution.rst
    yul.rst
    style-guide.rst
    common-patterns.rst

docs/layout-of-source-files.rst

@@ -163,7 +163,7 @@ The component does not yet support all features of the Solidity language and
 likely outputs many warnings. In case it reports unsupported features, the
 analysis may not be fully sound.
 
-.. index:: source file, ! import, module
+.. index:: source file, ! import, module, source unit
 
 .. _import:
 
@@ -184,6 +184,7 @@ At a global level, you can use import statements of the following form:
 
   import "filename";
 
+The ``filename`` part is called an *import path*.
 This statement imports all global symbols from "filename" (and symbols imported there) into the
 current global scope (different than in ES6 but backwards-compatible for Solidity).
 This form is not recommended for use, because it unpredictably pollutes the namespace.
@@ -216,101 +217,34 @@ the code below creates new global symbols ``alias`` and ``symbol2`` which refere
 
   import {symbol1 as alias, symbol2} from "filename";
 
-Paths
------
-
-In the above, ``filename`` is always treated as a path with ``/`` as directory separator,
-and ``.`` as the current and ``..`` as the parent directory.  When ``.`` or ``..`` is followed by a character except ``/``,
-it is not considered as the current or the parent directory.
-All path names are treated as absolute paths unless they start with the current ``.`` or the parent directory ``..``.
-
-To import a file ``filename`` from the same directory as the current file, use ``import "./filename" as symbolName;``.
-If you use ``import "filename" as symbolName;`` instead, a different file could be referenced
-(in a global "include directory").
-
-It depends on the compiler (see :ref:`import-compiler`) how to actually resolve the paths.
-In general, the directory hierarchy does not need to strictly map onto your local
-filesystem, and the path can also map to resources such as ipfs, http or git.
-
-.. note::
-    Always use relative imports like ``import "./filename.sol";`` and avoid
-    using ``..`` in path specifiers. In the latter case, it is probably better to use
-    global paths and set up remappings as explained below.
-
-.. _import-compiler:
-
-Use in Actual Compilers
------------------------
-
-When invoking the compiler, you can specify how to discover the first element
-of a path, and also path prefix remappings. For
-example you can setup a remapping so that everything imported from the virtual
-directory ``github.com/ethereum/dapp-bin/library`` would actually be read from
-your local directory ``/usr/local/dapp-bin/library``.
-If multiple remappings apply, the one with the longest key is tried first.
-An empty prefix is not allowed. The remappings can depend on a context,
-which allows you to configure packages to import e.g., different versions of a
-library of the same name.
-
-**solc**:
-
-For solc (the commandline compiler), you provide these path remappings as
-``context:prefix=target`` arguments, where both the ``context:`` and the
-``=target`` parts are optional (``target`` defaults to ``prefix`` in this
-case). All remapping values that are regular files are compiled (including
-their dependencies).
-
-This mechanism is backwards-compatible (as long
-as no filename contains ``=`` or ``:``) and thus not a breaking change. All
-files in or below the ``context`` directory that import a file that starts with
-``prefix`` are redirected by replacing ``prefix`` by ``target``.
-
-For example, if you clone ``github.com/ethereum/dapp-bin/`` locally to
-``/usr/local/dapp-bin``, you can use the following in your source file:
-
-::
-
-  import "github.com/ethereum/dapp-bin/library/iterable_mapping.sol" as it_mapping;
-
-Then run the compiler:
-
-.. code-block:: bash
-
-  solc github.com/ethereum/dapp-bin/=/usr/local/dapp-bin/ source.sol
-
-As a more complex example, suppose you rely on a module that uses an old
-version of dapp-bin that you checked out to ``/usr/local/dapp-bin_old``, then you can run:
-
-.. code-block:: bash
-
-  solc module1:github.com/ethereum/dapp-bin/=/usr/local/dapp-bin/ \
-       module2:github.com/ethereum/dapp-bin/=/usr/local/dapp-bin_old/ \
-       source.sol
-
-This means that all imports in ``module2`` point to the old version but imports
-in ``module1`` point to the new version.
-
-.. note::
-
-  ``solc`` only allows you to include files from certain directories. They have
-  to be in the directory (or subdirectory) of one of the explicitly specified
-  source files or in the directory (or subdirectory) of a remapping target. If
-  you want to allow direct absolute includes, add the remapping ``/=/``.
-
-If there are multiple remappings that lead to a valid file, the remapping
-with the longest common prefix is chosen.
-
-**Remix**:
-
-`Remix <https://remix.ethereum.org/>`_ provides an automatic remapping for
-GitHub and automatically retrieves the file over the network. You can import
-the iterable mapping as above,  e.g.
-
-::
-
-  import "github.com/ethereum/dapp-bin/library/iterable_mapping.sol" as it_mapping;
-
-Remix may add other source code providers in the future.
+.. index:: virtual filesystem, source unit name, import; path, filesystem path, import callback, Remix IDE
+
+Import Paths
+------------
+
+In order to be able to support reproducible builds on all platforms, the Solidity compiler has to
+abstract away the details of the filesystem where source files are stored.
+For this reason import paths do not refer directly to files in the host filesystem.
+Instead the compiler maintains an internal database (*virtual filesystem* or *VFS* for short) where
+each source unit is assigned a unique *source unit name* which is an opaque and unstructured identifier.
+The import path specified in an import statement is translated into a source unit name and used to
+find the corresponding source unit in this database.
+
+Using the :ref:`Standard JSON <compiler-api>` API it is possible to directly provide the names and
+content of all the source files as a part of the compiler input.
+In this case source unit names are truly arbitrary.
+If, however, you want the compiler to automatically find and load source code into the VFS, your
+source unit names need to be structured in a way that makes it possible for an :ref:`import callback
+<import-callback>` to locate them.
+When using the command-line compiler the default import callback supports only loading source code
+from the host filesystem, which means that your source unit names must be paths.
+Some environments provide custom callbacks that are more versatile.
+For example the `Remix IDE <https://remix.ethereum.org/>`_ provides one that
+lets you `import files from HTTP, IPFS and Swarm URLs or refer directly to packages in NPM registry
+<https://remix-ide.readthedocs.io/en/latest/import.html>`_.
+
+For a complete description of the virtual filesystem and the path resolution logic used by the
+compiler see :ref:`Path Resolution <path-resolution>`.
 
 .. index:: ! comment, natspec