Home

Welcome to the Mozilla-Learning wiki! This page will take you through the general information about how the learning.mozilla.org codebase is structured, see the nav menu on the right for more specific additional pages.

Learning.mozilla.org

The following is an onboarding document that describes how the learning site works from a code and run-time perspective.

1. content serving via app.js

• route checks:

  • redirect -> redirect client
  • route based on lib/routes.jsx
    • render page to HTML
    • serve response
  • try to resolve as WP stub
  • try to resolve as static resource
  • check if there's a locale in the path. If not, figure out which one to add and start route checks again with a redirect (see below)
  • treat as 404

• locale negotiation: When a path is requested that doesn't exactly match an existing route, app.js hands off the path to mofo-localize, along with the json in the build step generated from the .properties files and the accept-language http request header. Mofo-localize does some best guesses at which locale to send to the user based on the available ones. If it returns a redirect string, app.js handles that and starts the route matching process again.

2. content building via routes.jsx

• all <Route> entries are nested as content to the components/page.jsx component, <Page>.
  • <Page> is the master "template" component, and loads all component scaffolding.
• Specific pages are loaded from the ./pages directory, and are set as this.props.children content to the Page template.
  • technically, they are cloned because additional props need to be added in, which cannot be done directly on the this.props.children value, but can be done using React.cloneElement.
• Specific pages all specify a static pageClassName value, which is used by the <Page> component to set the outermost CSS class.
• Specific pages all specify a static pageTitle value, which is used by the <Page> component to set the appropriate page title.

3. Pages vs Components

Broadly speaking, content is modelled using React, and React code is split over two locations.

1. ./pages houses all "webpage" files, where in the ideal case each one file mirrors one page of the learning website.
2. ./components houses all the modular code used by one or more pages for specific kinds of content. Note that the <Page> template is considered a component, not a page on its own.

Practically, some pages contain code either in their own file or in sibling files that should really be components, but were kept near their page so that they were easy to find or easy to edit.

The Sidebar

One special component is the <Sidebar> component, which acts as navigation element for the learning site.

Due to legacy reasons, it does not build a sidebar based on information in the routes.jsx file, but has a hardcoded list of top level navigation items and nested sub items.

Force-navigating to "the page you're already on"

If you're on a page associated with some route, and this route allows for page content that swaps out the original page without affecting the URL users see, then our users have no way of returning to the base page content other than to first navigate "somewhere else" and then come back. In order to deal with this, there is a ./lib/resetreload library that can be required in, with a one line API:

resetreload.shouldResetOnReload(true);

Calling this in your code (anywhere) will result in the Sidebar's entry for the current page to bypass normal React-Router behaviour (which is "you are navigating to the page you are already on, I will do nothing") and will instead call the page component's reset() function, if it has one defined. If the page component has no reset() function, nothing happens.

Note that anything can instruct the sidebar to reset the current page, but the reset function needs to live on the component that is loaded in for the route you are on, based on routes.jsx. In the /clubs example, while something deep inside a component inside a component on the page could call resetreload.shouldResetOnReload(true), the reset function must be defined for in the ClubsPage.jsx component code, because that is the entry point for the /clubs route.

4. Styling uses LESS

CSS is managed through LESS, housed in the ./less directory and aggregrated through ./less/index.less. There are a few common files (most notably variables.less for the site colouring) and then for actual site content we use the same split for components and pages, to keep the style/code relation clear for developers.

The LESS compile is managed by gulp via the less task that runs as part of ./gulp/shared/minimal.build.tasks.js, defined in ./gulp/less.task.js, which writes the compiled less as styles.css into the ./dist dir, where it will sit next to the site code bundle.

5. User management

The learning site does not do its own user/session management, instead deferring to the Teach-API for this work.

The ./lib/teach-api.js file is a connector that is loaded as part of the <Page> template and is used to check whether the client should currently be treated as logged in or not: if the teach-api's getLoginInfo returns a username, the client is to be considered logged in, as the indicated user. If the return is null, there is no active session and the user is an anonymous user.

The Teach-APi itself is a Django API server that, itself, defers actual user authentication to the id.webmaker.org service. When a user wants to log into the learning website, the following series of events occur:

initial sign in/up resolution

1. User clicks sign in or sign up on a page on the learning website.
2. These links redirect the user to the Teach-API, with a reference to the learning site URL they came from
3. the Teach-API immediately redirects the user to id.webmaker.org ("id.wmo") with a query argument that ensures the original learning site location is not lost
4. the user runs through the id.wmo motions, filling in their username and password, and id.wmo performs its authentication routine.
5. when the user's credentials are correct, id.wmo performs a callback to the Teach-API, not the learning site, informing it that the user is authenticated
6. the Teach-API marks the user as having an active session using Django session management, and then immediately redirects the user's client to the learning site URL the user originally came from.
7. At this point the flow is the same as the next section

Page load session check

1. The user loads a page, which loads the teach-api.js code, which checks to see if a localStorage entry exists for a teach api session.
2. If one exists, the user is considered logged in and they will see their personal information as part of the site navigation.
3. If no entry exists, the teach-api.js code contacts the Teach-API to see if, based on cookie information, the Django session manager thinks there should be an active session. If so, the teach-api.js code builds the localStorage entry and the user is considered logged in.
4. If no Django session exists, the user is considered anonymous and they will see sign in/sign up options as part of the site navigation.

6. Clubs

A significant part of the learning site is around Mozilla clubs, which are saved and loaded via the Teach-API, which stores club information as user-submitted structured data.

Club information is consulted and manipulated through the teach-api.js interface.

7. Badges

Locked behind a ENABLE_BADGES feature flag, the learning site has a series of pages for Credly badges that are tied to user accounts.

Badge data is negotiated through the ./lib/badges-api.js code. This code, like the clubs code, relies on the Teach-API, but because of contracting work specifically around badges, ended up having its own accessor for talking to the Teach-API.

(this file may be merged into the teach-api.js file at some point, once Badges have been fully integrated into the learning website)

The build system

The build system comprises testing and building, using the following tools:

• eslint for JS rule enforcement
• mocha for unit and coverage testing
• lessc with chokidar for compiling less into CSS
• webpack for creating client and server bundles
• a custom site crawler for internal link checking
• chokidar for performing file-watching outside of webpack's ability to do the same.

1. eslint

Eslint runs on the entire codebase with rules very similar to, but currently not quite on par with, the mofo-style rules. Work is ongoing to make sure the entire codebase gets uplifted to follow our intended rules.

There are two eslint tasks:

1. npm run lint - runs linting on the entire codebase
2. npm run lint:fix - runs linting with automatic "fix everything that can be fixed" file modifications

2. Mocha

Mocha is used to run tests against the compiled client and server code. Of specific use is the app.test.js file, which runs the server and can test actual results obtained via normal HTTP requests.

There is one mocha task:

1. npm run test:mocha - runs all the mocha tests

3. lessc with chokidar

lessc isn't very good at monitoring lots of files, so we have a chokidar task that watches for changes to any file in the ./less directory, and if it sees any, runs npm run less to recompile the CSS. This allows us to edit arbitrary LESS files without needing to artificially resave the index.less file to trigger a proper less compile.

There are two lessc tasks:

1. npm run less - runs a single less compile
2. npm run start:less - starts a watch test that triggers recompiles everytime a file in ./less changes

4. Webpack

There are two webpack tasks, one for building the server library that we use in app.js, and one for building the client bundle that users get served via their browser. Both these tasks can either run "once, to completion", for deployment purposes, or "in watch mode", where they monitor changes to any files involved in the library and bundle, doing recompiles as necessary if any of these files are modified.

There are two webpack tasks:

1. npm run webpack:client - runs the client bundle compile
2. npm run webpack:server - runs the server library compile

These tasks also have watch counterparts:

1. npm run start:client - runs webpack:client in --watch mode
2. npm run start:server - runs webpack:server in --watch mode

5. Web crawler

We run a custom web crawler on the server that app.js starts, run as node spider.js, which aggregates the list of internal links across out pages, looking for 404 errors and locale quirks. The first behaviour is an error, the second a warning-only.

6. Chokidar watching inside app.js

Since the server needs to make sure it's always running with the latest code, there is a chokidar task defined inside app.js that looks for changes to content in the ./build directory, which houses the server library based on our code. If that changes, the app uncaches its already-required server library, re-requires it, and then rebinds its router.

Running the tests

Standard procedure applies: npm test

Running in dev mode

Running the codebase for development purposes is simply a matter of running npm start, which is an alias for a whole bunch of npm scripts that kick off the various compile and watch tasks. Once you see the following text fly by, you can fire up http://localhost:8008 successfully:

==================================================
=                                                =
=          Server listening on port 8008         =
=                                                =
==================================================

Localisation

Please see https://github.com/mozilla/learning.mozilla.org/blob/master/L10N.md for now on how we do localisation in this project.

Full local installation

In order to run learning.mozilla.org completely via localhost, you will need the following projects and dependencies:

1. Node.js, npm, python, pip, virtualenv, PostgreSQL
2. learning.mozilla.org
3. teach-api
4. id.webmaker.org
5. login.webmaker.org

Most of the dependencies are easily installed, PostgreSQL might be a bit of a jerk depending on the OS you use.

The git repos all come with their own instructions in their respective README.md files, and it's easy to forget but the learning.mozilla.org repo needs an .env file made with the content:

TEACH_API_URL=http://localhost:8000

and the Teach API needs the following enviroment values set (or ./teach/settings.py updated):

IDAPI_URL = os.environ.get('IDAPI_URL', 'http://localhost:1234')
IDAPI_CLIENT_ID = os.environ.get('IDAPI_CLIENT_ID', 'test')
IDAPI_CLIENT_SECRET = os.environ.get('IDAPI_CLIENT_SECRET', 'test')
LOGINAPI_URL = os.environ.get('LOGINAPI_URL', 'http://localhost:3000')
TEACH_SITE_URL = os.environ.get('TEACH_SITE_URL', 'http://localhost:8008')

Finally, there is one crucial bit missing pertaining to the id.webmaker.org system:

bootstrapping id.webmaker.org

In order to use learning.mozilla.org with webmaker user accounts, we need the teach-api installed to manage user sessions, id.webmaker.org installed to broker user authentication, and login.webmaker.org installed to act as user database endpoint. However, for the teach-api to be able to negotiate with id.webmaker.org it needs a "client id" so that id.webmaker.org can verify that the authentication calls are being made by an authorized service.

The README.md does not currently cover how to do that, so: after installing id.webmaker.org, create a new database through bash (or whatever terminal you use) using the createdb command that you get when installing PostgreSQL:

$> createdb webmaker_oauth_test

After this, tell id.webmaker.org to run its tests using npm test. This will connect to the database, populate it, clear most of it, but leave three client ids for testing purposes. We need to pick one of these to act as the teach-api's client id.

Start the postgres CLI client and connect to the webmaker_oauth_test database:

$> psql webmaker_oauth_test

note: if you are a Windows user, you will probably get an error because there is "no role " defined. Using the postgresql administrative username and password you picked when you installed postgresql, run the createuser CLI utilily:

$> createuser -s -U YourPostgresAdminUsername -W YourWindowsUserName

This creates a new superuser with your windows user name (-s setting you use as superuser, -U ... being the flag that runs the CLI utility as the postgres admin, and -W being the flag that requires you to type the admin password). You should now be able to run the psql webmaker_oauth_test command properly.

Once connected with the postgres CLI, issue the following update command:

psql> update clients set redirect_uri = 'http://localhost:8000/auth/oauth2/callback' where client_id = 'test';

This sets the callback URL for the test clientid to the URL that the teach-api exposes for authentication purposes. With this done, in the teach-api project open the ./teach/settings.py file and update the IDAPI_CLIENT_ID and IDAPI_CLIENT_SECRET to say "test" if they don't already.

You should now be able to run the whole shebang locally by:

1. starting the login.wmo service
2. starting the id.wmo service
3. starting the teach-api (in its virtualenv)
4. starting learning.moz
5. navigating to http://localhost:8008
6. sign in/up (which only works if everything's set up correctly)
7. finally getting things done ✨

Changing environment variables used in the code

Whether you are renaming, removing, or changing the default value for an existing environment variable, make sure that:

1. The environment variables section on README.md is up to date, and note that:
2. we're following 12-factor practices, so your variable should have a default value that "just works"(tm), and also note that:
3. config/webpack.config.js reflects your changes, and:
4. your .env file reflects the overrides on the default that you need.