PacktPub: Learning Express Web Application Development

Index

PacktPub: Learning Express Web Application Development

Introduction

This course is a few years old now (December 2014)

I am using NodeJS (v8.11.4) and Yarn (v1.9.4) package manager instead of npm. My development environment is Ubuntu 18.04.1 LTS (Linux). gi The following sections are my notes for the PacktPub video course. The structure below maps directly to the chapter structure of the course. Missing sections from the PacktPub structure indicate I didn't watch or didn't take any notes for the chapter.

Getting Started

Install Express.js and command-line tools

Bootstrapping Your First App

ROOT_PATH=~/git/github/pvspain/learning-express-web-application-development
mkdir --parents $ROOT_PATH/project1
cd $ROOT_PATH
# Initialise new Git repository
git init
cd project1

yarn add express-generator
# Validate installation
express --version
# Create a new express project with support for Stylus CSS engine
express --css stylus
# Install dependencies
yarn install
# Start Express webserver on localhost:3000
DEBUG=project1:* yarn start

Initialise project .gitignore from NodeJS template

Structure of an Express.js App

A lot of files are generated by Express
package.json
- common to all NodeJS projects
- holds project metadata
- defines scripts
  - executed as arguments to package manager
```
yarn start
```
- lists dependencies
- note express, jade (view engine), stylus (css engine specified when project generated)
app.js
- Express application file - initially auto-generated
bin/www
- Node webserver generated by Express
node_modules
- required modules - managed by package manager (Yarn) - exclude from version control
public
- static website assets - not managed by Express
public/stylesheets
- Edit CSS styles in style.styl (Stylus) file
- style.css is generated by CSS engine, which is invoked by Express

routes

dynamic routes for application

route mappings are wired up in app.js ...

var indexRouter = require('./routes/index');
var usersRouter = require('./routes/users');
...
app.use('/', indexRouter);
app.use('/users', usersRouter);

views
- application views rendered by Jade engine

Front-End Development

Introducing the Jade Template Engine

Jade home

Introducing Stylus

Stylus home

Bootstrap with Jade and Stylus

Bootstrap home
Bootstrap documentation
three versions
- CSS, Sass, Less (see below)

Public and Static Files

stored in public folder
static website assets - not managed by Express
add public/fonts folder for web-fonts

Other templating options

HTML template engines
- Mustache
- Handlebars (superset of handlebars)
```
yarn add express-handlebars
```
  - change view engine in app.js
more popular CSS pre-processors than Stylus now.
- Sass
  - compass node module
  - written in Ruby
- Less
  - very similar to Sass
  - written in JavaScript
```
express --css less
```

Planning Our Application

Planning the Structure of Our Application

Authentication
- PassportJS
  - social: Twitter, Facebook etc
  - SSO: GitHub, OpenID
  - local: username/password
Database
- Relational: MySQL
- Key-value: Redis
- Non-relational: Mongo (Mongoose.js)
Testing
- Runner: Karma
- Framework: Mocha.js
- Assertion lib: Chai
- Mocking: Sinon.js
- Helper util: SuperTest

Installing the Necessary Modules

yarn add --dev chai karma mocha sinon supertest
yarn add mongoose passport

Creating Our Endpoints

Express doesn't restart webserver for server-side code changes. Adding supervisor to dev dependencies restarts webserver when source changes detected.

yarn add --dev supervisor

We also adjust start script in package.json to use supervisor:

"scripts": {
    "start": "node node_modules/.bin/supervisor bin/www"
  },

Remove users routes created by default by Express.

Add routes for /contacts in routes/contacts.js

Wire up new routes in app.js

Creating Our User Interface

Creating Our Application’s User Interface

The | character in Jade is a line-continuation character for the current tag - whitespace is significant in Jade, like Python.

Download bootstrap files
Add bootstrap*.css files as imports in public\stylesheets\style.stl

Adding bootstrap via `yarn`

Processing peer dependencies for bootstrap npm module

yarn add bootstrap
yarn global add install-peerdeps
install-peerdeps bootstrap

# On yarn restart, bootstrap errors:

Starting child process with 'node bin/www'
/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:122
      $$$1.fn.emulateTransitionEnd = transitionEndEmulator;
                                   ^

TypeError: Cannot set property 'emulateTransitionEnd' of undefined
    at setTransitionEndSupport (/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:122:36)
    at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:199:5
    at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:201:4
    at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:7:66
    at Object.<anonymous> (/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:10:2)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
Program node bin/www exited with code 1

# Reverting to explicitly importing bootstrap css to project...

yarn remove bootstrap popper.js jquery

Displaying Dynamic Data with Jade

lodash.js is a JS utility library - a superset of underscore.js. Many of it's functions are now included in ES6

yarn add lodash
yarn add method-override

Hacks needed to deal with PUT and DELETE requests, since browsers only support GET and POST HTTP verbs.

PUT managed by method-override Node module, and form action argument in views/edit.jade:
- action="/contacts/#{contact.id}?_method=put"
DELETE managed by adding jquery-min.js and main.js (to public/javascripts) and including them in views/layout.jade
- main.js intercepts all clicks on DELETE button and does an Ajax call to server to delete all checked contacts

Automated Testing

Micro Testing/Unit Testing Versus Full Stack Testing

Full Stack Testing (Selenium)

Headless web browser
Comprehensive - It tests
- CSS
- Client-side JavaScript
- HTML
- Databases
- Infrastructure
Slow (10 secs to multiple minutes)
Others: Protractor, SST (Python)

Microtesing (Unit testing)

Tests a small piece of code
- i.e. one function or module)
Isolated from the rest of the app
Very fast

TDD/BDD

Test Driven Development
- Write tests first
- Run the tests to be sure they fail
- Write just enough code to make the tests pass
- Refactor to improve code quality (with tests passing)
Behaviour Driven Devlopment
- Same pilosophy as TDD
- More human-readable test format
Both are compatible with the tools we'll be using

Test Client

A tool halfway between the micro-test and the full-stack test
In our app, it would test the full server side stack, including middleware
It does not launch the app, instead merely runs the code in the same way Nodejs would, but with some hooks for testing
It won't render the app and send it to a browser, but the app would not know the difference

Setting Up Test Tools

Karma
- test runner for client-side code
Mocha
- test framework
Chai
- assertion library
  - note ok() - test for "truthiness"
Sinon
- mocking library
Supertest
- test client
Create top-lvel folder test in project for tests

Server-side JS Testing Versus Client-side JS Testing

Client-side testing assumes the browser and its DOM are present.
Karma launches a test environment, loads a browser and loads environment inside browser
Karma needs intialisation.

./node_modules/karma/bin/karma init

Which testing framework do you want to use ?
Press tab to list possible options. Enter to move to the next question.
> mocha

Do you want to use Require.js ?
This will add Require.js plugin.
Press tab to list possible options. Enter to move to the next question.
> no

Do you want to capture any browsers automatically ?
Press tab to list possible options. Enter empty string to move to the next question.
> Chrome
>

What is the location of your source and test files ?
You can use glob patterns, eg. "js/*.js" or "test/**/*Spec.js".
Enter empty string to move to the next question.
> public/javascripts/**/*.js
> browser-tests/**/*.js
05 09 2018 18:13:34.188:WARN [init]: There is no file matching this pattern.

>

Should any of the files included by the previous patterns be excluded ?
You can use glob patterns, eg. "**/*.swp".
Enter empty string to move to the next question.
>

Do you want Karma to watch all the files and run the tests on change ?
Press tab to list possible options.
> yes


Config file generated at "/home/paul/git/github/pvspain/learning-express-web-application-development/project1/karma.conf.js".

Create browser-tests folder and run Karma

mkdir browser-tests
./node_modules/karma/bin/karma start --single-run

Add karma entry to package.json to simplify running tests

  "scripts": {
    ...
    "karma": "./node_modules/karma/bin/karma start --single-run"
  },

Execute

yarn run karma

Add chai and sinon to karma.conf.js as additional frameworks

yarn add --dev karma-chai karma-sinon

Storing Data in MongoDB

Installing and Configuring MongoDB

Install MongoDB

Use apt package manager on Ubuntu 18.04

sudo apt update
sudo apt install -y mongodb
# Get status
sudo systemctl status mongodb
# Further validation
mongo --eval 'db.runCommand({ connectionStatus: 1 })'

Or use instructions in Mongo docs for other Linux distros

Wiring Up Mongoose.js

We need one Mongo connection per app
Typically, Mongo will be running on a different host
We need a dev-only local instance of Mongo
Note that Mongo runs asynchronously
- In production environment, Mongo may intialise after your app is ready.
  - When the connection is live Mongoose will fire an open event.
  - Handle this event to be notified. Refer to the Mongoose docs
```
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {
  // we're connected!
});
```

Creating Our Models

MongoDB is a document database.
- Other examples: CouchDB, Cassandra
Mongoose allows us to define schemas.
Schemas:
- have helper functions to access data
- can be defined anywhere, but typically in models folder under root of project
```
mkdir models
```
- one schema per file
- file name matches schema

Differences between MongoDB and a Relational Database

Non-relational databases
MongoDB is a document database.
- Other examples: CouchDB, Cassandra
Key/Value stores
- Examples: Redis, GDBM, Memcached
Graph databases
- Example: Neo4j
Column-oriented databases
- Example: HBase

Collecting Data

Allow plain-english formatting for timestamps - use moment module

yarn add moment

Auhenticating Users

Dropping a Mongo db

$ mongo
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
testdb  0.000GB
> use testdb
switched to db testdb
> db.dropDatabase()
{ "dropped" : "testdb", "ok" : 1 }
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
>

Adding Passport.js Authentication

yarn add passport-local passport-local-mongoose express-session

Understanding Sessions

express-session provides middleware that adds session informaton to every request
Add persistent sessions, using MongoStore
```
yarn add connect-mongo
```
- Any serialisable object can be added to MongoStore
  - persisting a db connection is a bad idea!
- A great alternative is Redis store if a Redis instance is available

Using Facebook or Twitter for Authentication

Get API keys from Twitter and Facebook

Add Node modules

yarn add passport-facebook passport-twitter

Deployment Options

Deploying Your Application

NodeJS default mode is dev
Explicitly set to production via environment variable

NODE_ENV = production

Install only production packages

yarn install --production
yarn install --prod

Determine globally installed packages

yarn global list
npm ls -g --depth=0

Minify only assets in public folder
- e.g. Grunt or Gulp have uglify packages to minify styles and JavaScript
- All other assets remain server-side
Serve public assets using a webserver, not NodeJS
Get professional assistance for MongoDB in heavy-traffic scenarios
Measure performance (Selenium, ProtractorJS, jMeter)
- mongoose-cache for common response caching
- Redis for sessions
Avoid global variables
PaaS NodeJS hosting - Heroku, Azure

Deploying Your App to the Cloud via Heroku

Platform as a Service (PaaS)
Heroku
Installing Heroku CLI

sudo snap install heroku --classic
heroku login
cd /path/to/nodejs/app/root/dir
heroku create
git push heroku master
heroku ps:scale web=1
heroku open
# On error...
heroku logs --tail
# Add mongodb support
heroku addons:create mongolab:sandbox
# Use heroku addons:docs mongolab to view documentation
heroku addons:docs mongolab
# Use the heroku config command to view your app’s config variables.
heroku config:get MONGODB_URI
# Adjust OAuth callbacks for Heroku URL
heroku config:set CALLBACK_DOMAIN=https://guarded-crag-87505.herokuapp.com

Considerations for Deploying Your App to Traditional Servers

NodeJS runs as a single process, separate from the web server.
Applications to run your nodejs app behind a commodity webserver such as Apache or Nginx.
- Forever, Foreman
Common to run NodeJS app as a dedicated user with restricted permissions.
Start application automatically as a service
- Ubuntu: upstart
- RedHat or Fedora: upstart or systemd
- Debian: system init
- Windows: node-windows
Do not expose NodeJS app directly to internet
- Use non-privileged port above 1024, such as 3000 or 8000.
- Redirect requests from commodity webserver.

Final Thoughts

Custom Modules

Use module.exports to expose functionality

Sails.js

Similar to Ruby On Rails - written in Javascript
Blueprints for auto-generation of REST APIs
Database adapter called Waterline

Socket.io

Bi-directional client/server communication
- avoids client-polling
- Uses WebSockets if browser supports
- Drops back to long-polling if not supported

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
course/Node.js Express		course/Node.js Express
project1		project1
.gitignore		.gitignore
README.md		README.md

pvspain/learning-express-web-application-development

Folders and files

Latest commit

History

Repository files navigation