Nyxt Developer’s Manual

Table of contents

• Development environment guide
• Developer’s installation guide
  • Installing SBCL
  • Lisp dependencies
  • GTK dependencies
    • GNU/Linux, FreeBSD GTK dependencies
  • Fetch the source code
  • Compile
    • Using the Makefile
    • Using Lisp
  • Interacting with a compiled version of Nyxt
    • SLIME
    • SLY
• Contributing
  • Asking for help
    • Issue tracker
    • Learning Common Lisp
    • IRC Channel
  • Commit style
  • Branch management
  • Adding a new Lisp dependency
  • Programming conventions

Development environment guide

If you are new to developing Nyxt, you may want a straightforward way to set up the environment so that you can hack on Nyxt with minimal friction. This is one way to achieve that goal. Keep in mind that this part of the manual is highly opinionated in it’s choice of tools. If you wish to use other tools, refer to the rest of the manual below.

To proceed, make sure that you have the following installed:

• SBCL
• The GUIX package manager
• Emacs
• SLY

Now that you have those working on your system, clone the Nyxt repository into ~/common-lisp. Then, in your Emacs configuration file, add the following and restart Emacs:

(load "~/common-lisp/nyxt/build-scripts/nyxt-guix.el" :noerror)

(setq sly-lisp-implementations
        '((nyxt-sbcl
           (lambda () (nyxt-make-guix-cl-for-nyxt
                  "~/common-lisp/nyxt"
                  :force t
                  :cl-implementation "sbcl"
                  :cl-system "nyxt/gi-gtk"
                  :no-grafts t
                  :ad-hoc '("emacs" "xdg-utils" "git"))))))

Next, in Emacs, run M-- M-x sly RET nyxt-sbcl RET to start the SLY REPL. Wait for it to finish and the REPL will open. At this point you are almost ready to start hacking. In the SLY REPL, write the following:

(asdf:load-system :nyxt/gi-gtk)
(nyxt:start)

With Nyxt started in this way, you can hack on Nyxt without recompiling the whole codebase or even restarting the browser. You can make your changes, recompile only the relevant file/expression with SLY and it’s done.

You can also run the tests locally with:

(asdf:test-system :nyxt)
; and
(asdf:test-system :nyxt/gi-gtk)

Keep in mind that it is recommended to restart the SLY session before and after running the tests.

Developer’s installation guide

There are several ways to install Nyxt on your system. For example, to install with Guix, follow the instructions in the ../build-scripts/nyxt.scm file of the Nyxt repository. In this section, we will provide a more tool-agnostic way.

Nyxt is written in Common Lisp. It should build with any standard Common Lisp implementation but currently, only SBCL support is tested. It is designed to be cross-platform, cross-engine compatible. Nyxt is available in both WebKit and WebEngine (experimental) flavors.

Installing SBCL

You’ll need SBCL ≥ 2.0.0 to compile Nyxt.

You can obtain SBCL from your package manager or by downloading it directly from the SBCL repository.

To install SBCL from source, download SBCL: http://www.sbcl.org/platform-table.html. Full installation instructions can be found here: http://www.sbcl.org/getting.html.

Lisp dependencies

Nyxt requires some Lisp libraries and, since some are pinned at specific versions, relying on Quicklisp is discouraged.

All Lisp dependencies are included as a submodule of this repository, so unless the submodules are disabled (i.e. NYXT_SUBMODULES=false), there is nothing more to be done.

Otherwise clone the required repositories listed in .gitmodules into ~/common-lisp (or another directory where ASDF will find it). For instance, to get the latest version of cl-webkit, you would typically run the following:

mkdir -p ~/common-lisp
git clone https://github.com/joachifm/cl-webkit ~/common-lisp/cl-webkit

GTK dependencies

GNU/Linux, FreeBSD GTK dependencies

• WebKitGTK+ also known as webkit2gtk (make sure to use the most recent version for security reasons)
• gobject-introspection (for WebKitGTK+ bindings)
• glib-networking (for WebKitGTK+)
• gsettings-desktop-schemas (for WebKitGTK+)
• libfixposix
• xclip (for clipboard support)
• enchant (for spellchecking)
• pkg-config (for web-extensions)
• Debian-based distributions:

  sudo apt install sbcl libwebkit2gtk-4.0-dev gobject-introspection glib-networking gsettings-desktop-schemas libfixposix-dev pkg-config xclip enchant-2 libssl-dev
      

• Arch Linux:

  sudo pacman -S git sbcl cl-asdf webkit2gtk gobject-introspection glib-networking gsettings-desktop-schemas enchant libfixposix
      

• Fedora:

  sudo dnf install sbcl webkit2gtk3-devel glib-networking gsettings-desktop-schemas libfixposix-devel xclip enchant pkgconf
      

• FreeBSD and derivatives

  pkg install sbcl webkit2-gtk3 glib-networking libfixposix xclip enchant rubygem-pkg-config
      

If your distribution does not install libraries in an FHS-expected location, you have to let your Lisp compiler know where to find them. To do so, add the library directories to cffi:*foreign-library-directories* list. For instance, if you are running Guix you may want to expose ~/.guix-profile/lib to the compiler by adding the following snippet to ~/.sbclrc:

(require "asdf")

(let ((guix-profile (format nil "~a/.guix-profile/lib/" (uiop:getenv "HOME"))))
  (when (and (probe-file guix-profile)
             (ignore-errors (asdf:load-system "cffi")))
    (push guix-profile
          (symbol-value (find-symbol (string '*foreign-library-directories*)
                                     (find-package 'cffi))))))

A note of caution about installing WebKit via your package manager: Your distribution supplied version of WebKit may not provide up-to-date versions of WebKit including the latest security patches. WebKitGTK+ tries to do the best job possible with maintaining security patches upstream, but it is also up to the the distribution provider to update their packages to take advantage of these fixes.

Fetch the source code

Clone the Nyxt repository into ~/common-lisp (or another directory where ASDF will find it):

mkdir -p ~/common-lisp
git clone --recurse-submodules https://github.com/atlas-engineer/nyxt ~/common-lisp/nyxt

Compile

Using the Makefile

The following command will build the Lisp core.

• GNU/Linux:

  make all
      

• FreeBSD

  gmake all
      

Inside the Makefile you’ll find many options you can specify. Run make to display some documentation or see the Makefile for more details.

Using Lisp

Start your Lisp and run the following commands:

(asdf:make :nyxt/gtk-application)

Your Lisp implementation must have produced an executable in the directory where the .asd file is located.

Interacting with a compiled version of Nyxt

After compiling Nyxt or installing it in some other way, you can use SLIME or SLY to interact with it in a REPL. This is accomplished by starting a swank or slynk server (for SLIME and SLY respectively) from Nyxt and connecting to it through Emacs.

SLIME

1. Run the command start-swank in Nyxt. Note the port number in the message buffer. The default is 4006.
2. Connect to the swank server in Emacs with M-x slime-connect RET 127.0.0.1 RET 4006.

SLY

1. Run the command start-slynk in Nyxt. Note the port number in the message buffer. The default is 4006.
2. Connect to the slynk server in Emacs with M-x sly-connect RET 127.0.0.1 RET 4006.

Contributing

Nyxt is a joint effort and we need you to make it succeed! You can find ideas on our issue tracker to suit your interests and skills. When ready to start working please fork the repository, add your changes and open a pull request on GitHub to pass the review process. Refer to the branch management section for more detailed information.

You can contribute to Nyxt without commit access. However, if you’re a frequent contributor, you may request it. Remember that with great power comes great responsibility.

Asking for help

Feel free to contact us at any point if you need guidance. There are several ways to ask for help from the community.

Issue tracker

The first and easiest one is to simply open up an issue with whatever problem or suggestion you wish to discuss.

Learning Common Lisp

See https://nyxt-browser.com/learn-lisp for a few recommendations.

IRC Channel

You can find Nyxt on Libera IRC: #nyxt.

Commit style

We follow the general Git guidelines, namely we try to commit atomic changes that are “clean”, that is, on which Nyxt builds and starts.

Make sure to make seperate commits in these cases to avoid distracting noise in commits with actual changes:

• Indentation and whitespace trimming;
• Code movements (within a file or to a different file). In this case, it’s crucial that the commit contains nothing else, otherwise “diffs” may fail to highlight the changes.

For commit messages, we follow (somewhat flexibly) the convention of prefixing the title with the basename of the file that was modified. For instance, for changes in source/mode/blocker.lisp the commit message would look like this:

mode/blocker: What and why this change.

Rest of the message here.

Your commit should clarify what it does and why (in case it’s not already obvious).

Branch management

Nyxt uses the following branches:

• master for development;
• <feature-branches> for working on particular features;
• <2,3,...>-series to backport commits corresponding to specific major versions.

It’s recommended to branch off from the target branch and to rebase onto it right before merging. This keeps the history as clear as possible and reduces the complexity of the diff.

Unless the changes are trivial and each commit is atomic (that is, leaving Nyxt fully functional), they should be followed by a merge commit. That is guaranteed by using the merge option no-ff (no fast-forward). If required, the merge commit can be reworded.

The names of the branches really matter since the merge commit references them, so please take that into account!

After the changes are merged, please do not forget to delete obsolete or dangling branches. If you merge the remote branch instead of the local one, then GitHub deletes the remote branch automatically.

Note to core contributors: since you have commit access, you can push trivial changes directly to the target branch (skipping the review process). The merge commit is required when at least one commit isn’t atomic.

Adding a new Lisp dependency

• Add and shallow clone upstream source as a Git submodule in ../_build/ directory.
• Add dependency name to ../nyxt.asd and documents/SOURCES.org.
• Add dependency to ../build-scripts/nyxt.scm, checking to make sure Guix already has it packaged.

Programming conventions

We try to follow the usual Common Lisp conventions as recommended by Norvig & Pitman’s Tutorial on Good Lisp Programming Style and Google Common Lisp Style Guide.

For symbol naming conventions, see https://www.cliki.net/Naming+conventions.

We’ve also developed some of our own:

• Prefer first and rest over car and cdr respectively.
• Use define-class instead of defclass.
• Use nyxt:define-package for Nyxt-related pacakges. Notice that it features default imports (e.g. export-always) and package nicknames (e.g. alex, sera, etc.). Prefer uiop:define-package for general purpose packages.
• Export using export-always (from Serapeum) next to the symbol definition. This helps prevent exports to go out-of-sync, or catch typos. Unlike export, export-always saves you from surprises upon recompilation.
• When sensible, declaim the function types using -> (from Serapeum). Note that there is then no need to mention the type of the arguments and the return value in the docstring.
• Use the maybe and maybe* types instead of (or null ...) and (or null (array * (0)) ...) respectively.
• Use the list-of type for typed lists.
• We make heavy use of Alexandria and Serapeum, remember to use them instead of writing the same boilerplate over and over. In particular, note these systematic uses of Serapeum:
  • sera:eval-always;
  • export-always;
  • sera:and-let*;
  • sera:lret;
  • sera:single
  • -> (declaimed types).
• Use funcall* to not error when function does not exist.
• Prefer classes over structs. Rationale:
  • Class slots have documentation.
  • Class allow for full-fledged CLOS use (metaclasses, etc.).
  • Structs have read-only slots but it’s easy enough to implement them for classes.
  • Structs have better performance, but this is usually micro-optimization, and even then class implementations can be made more efficient via MOP.
• Classes should be usable with just a make-instance.
• Slots classes should be formatted in the following way:

(slot-name
 slot-value
 ...
 :documentation "Foo.")

When slot-value is the only parameter specified then:

(slot-name slot-value)

• Prefer defmethod over defun if one of the arguments is a user-class. This allows the user to write specializations of subclasses.
• customize-instance is reserved for end users. Use initialize-instance :after or slot-unbound to initialize the slots. Set up the rest of the class in customize-instance :after. Bear in mind that anything in this last method won’t be customizable for the end user.
• Almost all files should be handled via the nfiles library.
• Specialize print-object for recurring class instances.
• (setf SLOT-WRITER) :after is reserved for “watchers”, i.e. handlers that are run whenever the slot is set. The :around method is not used by watchers, and thus the watcher may be overridden.
• A function as a slot value is often a sign that it should be a method instead. Methods give more flexibility to the end user. Example: Avoid adding a constructor slot, make it a method instead.
• Define generic functions (in particular if they are heavily used) using an explicit call to defgeneric, not with just calls to defmethod. This enables proper source location of the generic function (otherwise it cannot be found), plus it lets you write different documentation for the generic and the specialized methods.
• We use the %foo% naming convention for special local variables. But special variables are rare and ideally they should be avoided.
• We suffix predicates with -p. Unlike the usual convention, we always use a hyphen even for single word predicates.
• Prefer the term url over uri.
• URLs should be of type quri:uri. If you need to manipulate a URL string, call it url-string. In case the value contains a URL, but is not quri:url, use url-designator and its url method to normalize into quri:uri.
• Paths should be of type cl:pathname. Use uiop:native-namestring to “send” to OS-facing functions, uiop:ensure-pathname to “receive” from OS-facing functions or to “trunamize”.
• Prefer handler-bind over handler-case: when running from the REPL, this triggers the debugger with a full stacktrace; when running the Nyxt binary, all conditions are caught anyway.
• Do not handle the T condition, this may break everything. Handle error, serious-condition, or exceptionally condition (for instance if you do not control the called code, and some libraries subclass condition instead of error).
• Dummy variables are called _.
• Prefer American spelling.