- PacktPub: Learning Express Web Application Development
This course is a few years old now (December 2014)
I am using NodeJS (v8.11.4) and Yarn (v1.9.4) package manager instead of npm. My development environment is Ubuntu 18.04.1 LTS (Linux). gi The following sections are my notes for the PacktPub video course. The structure below maps directly to the chapter structure of the course. Missing sections from the PacktPub structure indicate I didn't watch or didn't take any notes for the chapter.
ROOT_PATH=~/git/github/pvspain/learning-express-web-application-development
mkdir --parents $ROOT_PATH/project1
cd $ROOT_PATH
# Initialise new Git repository
git init
cd project1
yarn add express-generator
# Validate installation
express --version
# Create a new express project with support for Stylus CSS engine
express --css stylus
# Install dependencies
yarn install
# Start Express webserver on localhost:3000
DEBUG=project1:* yarn start
Initialise project .gitignore
from NodeJS template
- A lot of files are generated by Express
package.json
- common to all NodeJS projects
- holds project metadata
- defines scripts
- executed as arguments to package manager
yarn start
- executed as arguments to package manager
- lists dependencies
- note
express
,jade
(view engine),stylus
(css engine specified when project generated)
app.js
- Express application file - initially auto-generated
bin/www
- Node webserver generated by Express
node_modules
- required modules - managed by package manager (Yarn) - exclude from version control
public
- static website assets - not managed by Express
public/stylesheets
- Edit CSS styles in
style.styl
(Stylus) file style.css
is generated by CSS engine, which is invoked by Express
- Edit CSS styles in
routes
- dynamic routes for application
- route mappings are wired up in
app.js
...var indexRouter = require('./routes/index'); var usersRouter = require('./routes/users'); ... app.use('/', indexRouter); app.use('/users', usersRouter);
views
- application views rendered by Jade engine
- Bootstrap home
- Bootstrap documentation
- three versions
- CSS, Sass, Less (see below)
- stored in
public
folder - static website assets - not managed by Express
- add
public/fonts
folder for web-fonts
- HTML template engines
- Mustache
- Handlebars (superset of handlebars)
yarn add express-handlebars
- change view engine in
app.js
- change view engine in
- more popular CSS pre-processors than Stylus now.
- Sass
compass
node module- written in Ruby
- Less
- very similar to Sass
- written in JavaScript
express --css less
- Sass
- Authentication
- PassportJS
- social: Twitter, Facebook etc
- SSO: GitHub, OpenID
- local: username/password
- PassportJS
- Database
- Relational: MySQL
- Key-value: Redis
- Non-relational: Mongo (Mongoose.js)
- Testing
- Runner: Karma
- Framework: Mocha.js
- Assertion lib: Chai
- Mocking: Sinon.js
- Helper util: SuperTest
yarn add --dev chai karma mocha sinon supertest
yarn add mongoose passport
Express doesn't restart webserver for server-side code changes.
Adding supervisor
to dev dependencies restarts webserver when source changes detected.
yarn add --dev supervisor
We also adjust start
script in package.json
to use supervisor
:
"scripts": {
"start": "node node_modules/.bin/supervisor bin/www"
},
Remove users
routes created by default by Express.
Add routes for /contacts
in routes/contacts.js
Wire up new routes in app.js
The |
character in Jade is a line-continuation character for the current tag - whitespace is significant in Jade, like Python.
Download bootstrap files
Add bootstrap*.css
files as imports in public\stylesheets\style.stl
Processing peer dependencies for bootstrap
npm module
yarn add bootstrap
yarn global add install-peerdeps
install-peerdeps bootstrap
# On yarn restart, bootstrap errors:
Starting child process with 'node bin/www'
/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:122
$$$1.fn.emulateTransitionEnd = transitionEndEmulator;
^
TypeError: Cannot set property 'emulateTransitionEnd' of undefined
at setTransitionEndSupport (/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:122:36)
at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:199:5
at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:201:4
at /home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:7:66
at Object.<anonymous> (/home/paul/git/github/pvspain/learning-express-web-application-development/project1/node_modules/bootstrap/dist/js/bootstrap.js:10:2)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
Program node bin/www exited with code 1
# Reverting to explicitly importing bootstrap css to project...
yarn remove bootstrap popper.js jquery
lodash.js
is a JS utility library - a superset of underscore.js
. Many of it's functions are now included in ES6
yarn add lodash
yarn add method-override
Hacks needed to deal with PUT and DELETE requests, since browsers only support GET and POST HTTP verbs.
- PUT managed by
method-override
Node module, and form action argument inviews/edit.jade
:action="/contacts/#{contact.id}?_method=put"
- DELETE managed by adding
jquery-min.js
andmain.js
(topublic/javascripts
) and including them inviews/layout.jade
main.js
intercepts all clicks on DELETE button and does an Ajax call to server to delete all checked contacts
- Headless web browser
- Comprehensive - It tests
- CSS
- Client-side JavaScript
- HTML
- Databases
- Infrastructure
- Slow (10 secs to multiple minutes)
- Others: Protractor, SST (Python)
- Tests a small piece of code
- i.e. one function or module)
- Isolated from the rest of the app
- Very fast
- Test Driven Development
- Write tests first
- Run the tests to be sure they fail
- Write just enough code to make the tests pass
- Refactor to improve code quality (with tests passing)
- Behaviour Driven Devlopment
- Same pilosophy as TDD
- More human-readable test format
- Both are compatible with the tools we'll be using
- A tool halfway between the micro-test and the full-stack test
- In our app, it would test the full server side stack, including middleware
- It does not launch the app, instead merely runs the code in the same way Nodejs would, but with some hooks for testing
- It won't render the app and send it to a browser, but the app would not know the difference
-
Karma
- test runner for client-side code
-
Mocha
- test framework
-
- assertion library
- note
ok()
- test for "truthiness"
- note
- assertion library
-
Sinon
- mocking library
-
Supertest
- test client
-
Create top-lvel folder
test
in project for tests
- Client-side testing assumes the browser and its DOM are present.
- Karma launches a test environment, loads a browser and loads environment inside browser
- Karma needs intialisation.
./node_modules/karma/bin/karma init
Which testing framework do you want to use ?
Press tab to list possible options. Enter to move to the next question.
> mocha
Do you want to use Require.js ?
This will add Require.js plugin.
Press tab to list possible options. Enter to move to the next question.
> no
Do you want to capture any browsers automatically ?
Press tab to list possible options. Enter empty string to move to the next question.
> Chrome
>
What is the location of your source and test files ?
You can use glob patterns, eg. "js/*.js" or "test/**/*Spec.js".
Enter empty string to move to the next question.
> public/javascripts/**/*.js
> browser-tests/**/*.js
05 09 2018 18:13:34.188:WARN [init]: There is no file matching this pattern.
>
Should any of the files included by the previous patterns be excluded ?
You can use glob patterns, eg. "**/*.swp".
Enter empty string to move to the next question.
>
Do you want Karma to watch all the files and run the tests on change ?
Press tab to list possible options.
> yes
Config file generated at "/home/paul/git/github/pvspain/learning-express-web-application-development/project1/karma.conf.js".
- Create
browser-tests
folder and run Karma
mkdir browser-tests
./node_modules/karma/bin/karma start --single-run
- Add
karma
entry topackage.json
to simplify running tests
"scripts": {
...
"karma": "./node_modules/karma/bin/karma start --single-run"
},
- Execute
yarn run karma
- Add
chai
andsinon
tokarma.conf.js
as additional frameworks
yarn add --dev karma-chai karma-sinon
-
-
Use
apt
package manager on Ubuntu 18.04sudo apt update sudo apt install -y mongodb # Get status sudo systemctl status mongodb # Further validation mongo --eval 'db.runCommand({ connectionStatus: 1 })'
-
Or use instructions in Mongo docs for other Linux distros
-
- We need one Mongo connection per app
- Typically, Mongo will be running on a different host
- We need a dev-only local instance of Mongo
- Note that Mongo runs asynchronously
-
In production environment, Mongo may intialise after your app is ready.
- When the connection is live Mongoose will fire an
open
event. - Handle this event to be notified. Refer to the Mongoose docs
var db = mongoose.connection; db.on('error', console.error.bind(console, 'connection error:')); db.once('open', function() { // we're connected! });
- When the connection is live Mongoose will fire an
-
- MongoDB is a document database.
- Other examples: CouchDB, Cassandra
- Mongoose allows us to define schemas.
- Schemas:
- have helper functions to access data
- can be defined anywhere, but typically in
models
folder under root of projectmkdir models
- one schema per file
- file name matches schema
- Non-relational databases
- MongoDB is a document database.
- Other examples: CouchDB, Cassandra
- Key/Value stores
- Examples: Redis, GDBM, Memcached
- Graph databases
- Example: Neo4j
- Column-oriented databases
- Example: HBase
- Allow plain-english formatting for timestamps - use
moment
module
yarn add moment
- Dropping a Mongo db
$ mongo
> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
testdb 0.000GB
> use testdb
switched to db testdb
> db.dropDatabase()
{ "dropped" : "testdb", "ok" : 1 }
> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
>
yarn add passport-local passport-local-mongoose express-session
-
express-session
provides middleware that adds session informaton to every request -
Add persistent sessions, using MongoStore
yarn add connect-mongo
- Any serialisable object can be added to MongoStore
- persisting a db connection is a bad idea!
- A great alternative is Redis store if a Redis instance is available
- Any serialisable object can be added to MongoStore
- NodeJS default mode is
dev
- Explicitly set to
production
via environment variable
NODE_ENV = production
- Install only production packages
yarn install --production
yarn install --prod
- Determine globally installed packages
yarn global list
npm ls -g --depth=0
- Minify only assets in
public
folder - Serve
public
assets using a webserver, not NodeJS - Get professional assistance for MongoDB in heavy-traffic scenarios
- Measure performance (Selenium, ProtractorJS, jMeter)
mongoose-cache
for common response caching- Redis for sessions
- Avoid global variables
- PaaS NodeJS hosting - Heroku, Azure
- Platform as a Service (PaaS)
- Heroku
- Installing Heroku CLI
sudo snap install heroku --classic
heroku login
cd /path/to/nodejs/app/root/dir
heroku create
git push heroku master
heroku ps:scale web=1
heroku open
# On error...
heroku logs --tail
# Add mongodb support
heroku addons:create mongolab:sandbox
# Use heroku addons:docs mongolab to view documentation
heroku addons:docs mongolab
# Use the heroku config command to view your app’s config variables.
heroku config:get MONGODB_URI
# Adjust OAuth callbacks for Heroku URL
heroku config:set CALLBACK_DOMAIN=https://guarded-crag-87505.herokuapp.com
- NodeJS runs as a single process, separate from the web server.
- Applications to run your nodejs app behind a commodity webserver such as Apache or Nginx.
- Common to run NodeJS app as a dedicated user with restricted permissions.
- Start application automatically as a service
- Do not expose NodeJS app directly to internet
- Use non-privileged port above 1024, such as 3000 or 8000.
- Redirect requests from commodity webserver.
- Use
module.exports
to expose functionality
- Similar to Ruby On Rails - written in Javascript
- Blueprints for auto-generation of REST APIs
- Database adapter called
Waterline
- Bi-directional client/server communication
- avoids client-polling
- Uses WebSockets if browser supports
- Drops back to long-polling if not supported