Skip to content

data visualization of spotify discovery weekly data

License

Notifications You must be signed in to change notification settings

akshaysu12/discovery

Repository files navigation

w3tec

Express Typescript Boilerplate

dependency travis appveyor StackShare

A delightful way to building a Node.js RESTful API Services with beautiful code written in TypeScript.
Inspired by the awesome framework laravel in PHP and of the repositories from pleerock
Made with ❤️ by w3tech, Gery Hirschfeld and contributors


divider

❯ Why

Our main goal with this project is a feature complete server application. We like you to be focused on your business and not spending hours in project configuration.

Try it!! We are happy to hear your feedback or any kind of new features.

Features

  • Beautiful Code thanks to the awesome annotations of the libraries from pleerock.
  • Easy API Testing with included e2e testing.
  • Dependency Injection done with the nice framework from TypeDI.
  • Simplified Database Query with the ORM TypeORM.
  • Clear Structure with different layers such as controllers, services, repositories, models, middlewares...
  • Easy Exception Handling thanks to routing-controllers.
  • Smart Validation thanks to class-validator with some nice annotations.
  • Custom Validators to validate your request even better and stricter. custom-validation-classes.
  • API Documentation thanks to swagger.
  • API Monitoring thanks to express-status-monitor.
  • Integrated Testing Tool thanks to Jest.
  • E2E API Testing thanks to supertest.
  • Basic Security Features thanks to Helmet.
  • Easy event dispatching thanks to event-dispatch.
  • Fast Database Building with simple migration from TypeORM.
  • Easy Data Seeding with our own factories.
  • GraphQL provides as a awesome query language for our api GraphQL.
  • TypeGraphQL thanks to TypeGraphQL we have a some cool decorators to simplify the usage of GraphQL.
  • DataLoaders helps with performance thanks to caching and batching DataLoaders.

divider

❯ Table of Contents

divider

❯ Getting Started

Step 1: Set up the Development Environment

You need to set up your development environment before you can do anything.

Install Node.js and NPM

Install yarn globally

yarn install yarn -g

Install a MySQL database.

If you work with a mac, we recommend to use homebrew for the installation.

Step 2: Create new Project

Fork or download this project. Configure your package.json for your new project.

Then copy the .env.example file and rename it to .env. In this file you have to add your database connection information.

Create a new database with the name you have in your .env-file.

Then setup your application environment.

yarn run setup

This installs all dependencies with yarn. After that it migrates the database and seeds some test data into it. So after that your development environment is ready to use.

Step 3: Serve your App

Go to the project dir and start your app with this yarn script.

yarn start serve

This starts a local server using nodemon, which will watch for any file changes and will restart the sever according to these changes. The server address will be displayed to you as http://0.0.0.0:3000.

divider

❯ Scripts and Tasks

All script are defined in the package-scripts.js file, but the most important ones are listed here.

Install

  • Install all dependencies with yarn install

Linting

  • Run code quality analysis using yarn start lint. This runs tslint.
  • There is also a vscode task for this called lint.

Tests

  • Run the unit tests using yarn start test (There is also a vscode task for this called test).
  • Run the integration tests using yarn start test.integration.
  • Run the e2e tests using yarn start test.e2e.

Running in dev mode

  • Run yarn start serve to start nodemon with ts-node, to serve the app.
  • The server address will be displayed to you as http://0.0.0.0:3000

Building the project and run it

  • Run yarn start build to generated all JavaScript files from the TypeScript sources (There is also a vscode task for this called build).
  • To start the builded app located in dist use yarn start.

Database Migration

  • Run typeorm migration:create -n <migration-file-name> to create a new migration file.
  • Try typeorm -h to see more useful cli commands like generating migration out of your models.
  • To migrate your database run yarn start db.migrate.
  • To revert your latest migration run yarn start db.revert.
  • Drops the complete database schema yarn start db.drop.

Database Seeding

  • Run yarn start db.seed to seed your seeds into the database.

divider

❯ Debugger in VSCode

To debug your code run yarn start build or hit cmd + b to build your app. Then, just set a breakpoint and hit F5 in your Visual Studio Code.

divider

❯ API Routes

The route prefix is /api by default, but you can change this in the .env file. The swagger and the monitor route can be altered in the .env file.

Route Description
/api Shows us the name, description and the version of the package.json
/graphql Route to the graphql editor or your query/mutations requests
/swagger This is the Swagger UI with our API documentation
/monitor Shows a small monitor page for the server
/api/users Example entity endpoint
/api/pets Example entity endpoint

divider

❯ Project Structure

Name Description
.vscode/ VSCode tasks, launch configuration and some other settings
dist/ Compiled source files will be placed here
src/ Source files
src/api/controllers/ REST API Controllers
src/api/controllers/requests Request classes with validation rules if the body is not equal with a model
src/api/controllers/responses Response classes or interfaces to type json response bodies
src/api/errors/ Custom HttpErrors like 404 NotFound
src/api/interceptors/ Interceptors are used to change or replace the data returned to the client.
src/api/middlewares/ Express Middlewares like helmet security features
src/api/models/ Bookshelf Models
src/api/repositories/ Repository / DB layer
src/api/services/ Service layer
src/api/subscribers/ Event subscribers
src/api/validators/ Custom validators, which can be used in the request classes
src/api/resolvers/ GraphQL resolvers (query, mutation & field-resolver)
src/api/types/ GraphQL types ,input-types and scalar types
src/api/ schema.gql Generated GraphQL schema
src/api/ swagger.json Swagger documentation
src/auth/ Authentication checkers and services
src/core/ The core features like logger and env variables
src/database/factories Factory the generate fake entities
src/database/migrations Database migration scripts
src/database/seeds Seeds to create some data in the database
src/decorators/ Custom decorators like @Logger & @EventDispatch
src/loaders/ Loader is a place where you can configure your app
src/public/ Static assets (fonts, css, js, img).
src/types/ *.d.ts Custom type definitions and files that aren't on DefinitelyTyped
test Tests
test/e2e/ *.test.ts End-2-End tests (like e2e)
test/integration/ *.test.ts Integration test with SQLite3
test/unit/ *.test.ts Unit tests
.env.example Environment configurations
.env.test Test environment configurations
mydb.sql SQLite database for integration tests. Ignored by git and only available after integration tests

divider

❯ Logging

Our logger is winston. To log http request we use the express middleware morgan. We created a simple annotation to inject the logger in your service (see example below).

import { Logger, LoggerInterface } from '../../decorators/Logger';

@Service()
export class UserService {

    constructor(
        @Logger(__filename) private log: LoggerInterface
    ) { }

    ...

divider

❯ Event Dispatching

We use this awesome repository event-dispatch for event dispatching. We created a simple annotation to inject the EventDispatcher in your service (see example below). All events are listed in the events.ts file.

import { events } from '../subscribers/events';
import { EventDispatcher, EventDispatcherInterface } from '../../decorators/EventDispatcher';

@Service()
export class UserService {

    constructor(
        @EventDispatcher() private eventDispatcher: EventDispatcherInterface
    ) { }

    public async create(user: User): Promise<User> {
        ...
        this.eventDispatcher.dispatch(events.user.created, newUser);
        ...
    }

divider

❯ Seeding

Isn't it exhausting to create some sample data for your database, well this time is over!

How does it work? Just create a factory for your entities (models) and a seed script.

1. Create a factory for your entity

For all entities we want to seed, we need to define a factory. To do so we give you the awesome faker library as a parameter into your factory. Then create your "fake" entity and return it. Those factory files should be in the src/database/factories folder and suffixed with Factory like src/database/factories/UserFactory.ts.

Settings can be used to pass some static value into the factory.

define(User, (faker: typeof Faker, settings: { roles: string[] }) => {
    const gender = faker.random.number(1);
    const firstName = faker.name.firstName(gender);
    const lastName = faker.name.lastName(gender);
    const email = faker.internet.email(firstName, lastName);

    const user = new User();
    user.firstName = firstName;
    user.lastName = lastName;
    user.email = email;
    user.roles = settings.roles;
    return user;
});

Handle relation in the entity factory like this.

define(Pet, (faker: typeof Faker, settings: undefined) => {
    const gender = faker.random.number(1);
    const name = faker.name.firstName(gender);

    const pet = new Pet();
    pet.name = name;
    pet.age = faker.random.number();
    pet.user = factory(User)({ roles: ['admin'] })
    return pet;
});

2. Create a seed file

The seeds files define how much and how the data are connected with each other. The files will be executed alphabetically. With the second function, accepting your settings defined in the factories, you are able to create different variations of entities.

export class CreateUsers implements Seed {

    public async seed(factory: Factory, connection: Connection): Promise<any> {
        await factory(User)({ roles: [] }).createMany(10);
    }

}

Here an example with nested factories. You can use the .map() function to alter the generated value before they get persisted.

...
await factory(User)()
    .map(async (user: User) => {
        const pets: Pet[] = await factory(Pet)().createMany(2);
        const petIds = pets.map((pet: Pet) => pet.Id);
        await user.pets().attach(petIds);
    })
    .createMany(5);
...

To deal with relations you can use the entity manager like this.

export class CreatePets implements SeedsInterface {

    public async seed(factory: FactoryInterface, connection: Connection): Promise<any> {
        const connection = await factory.getConnection();
        const em = connection.createEntityManager();

        await times(10, async (n) => {
            // This creates a pet in the database
            const pet = await factory(Pet)().create();
            // This only returns a entity with fake data
            const user = await factory(User)({ roles: ['admin'] }).make();
            user.pets = [pet];
            await em.save(user);
        });
    }

}

3. Run the seeder

The last step is the easiest, just hit the following command in your terminal, but be sure you are in the projects root folder.

yarn start db.seed

CLI Interface

Command Description
yarn start "db.seed" Run all seeds
yarn start "db.seed --run CreateBruce,CreatePets" Run specific seeds (file names without extension)
yarn start "db.seed -L" Log database queries to the terminal
yarn start "db.seed --factories <path>" Add a different path to your factories (Default: src/database/)
yarn start "db.seed --seeds <path>" Add a different path to your seeds (Default: src/database/seeds/)

divider

❯ GraphQL

For the GraphQL part we used the library TypeGraphQL to build awesome GraphQL API's.

The context(shown below) of the GraphQL is builded in the graphqlLoader.ts file. Inside of this loader we create a scoped container for each incoming request.

export interface Context {
  requestId: number;
  request: express.Request;
  response: express.Response;
  container: ContainerInstance;
}

DataLoader

For the usage of the DataLoaders we created a annotation, which automatically creates and registers a new DataLoader to the scoped container.

Here is an example of the PetResolver.

import DataLoader from 'dataloader';
import { DLoader } from '../../decorators/DLoader';
    ...
    constructor(
        private petService: PetService,
        @Logger(__filename) private log: LoggerInterface,
        @DLoader(UserModel) private userLoader: DataLoader<string, UserModel>
    ) { }
    ...

Or you could use the repository too.

@DLoader(UserRepository) private userLoader: DataLoader<string, UserModel>

Or even use a custom method of your given repository.

@DLoader(PetRepository, {
    method: 'findByUserIds',
    key: 'userId',
    multiple: true,
}) private petLoader: DataLoader<string, PetModel>

❯ Docker

Install Docker

Before you start, make sure you have a recent version of Docker installed

Build Docker image

docker build -t <your-image-name> .

Run Docker image in container and map port

The port which runs your application inside Docker container is either configured as PORT property in your .env configuration file or passed to Docker container via environment variable PORT. Default port is 3000.

Run image in detached mode

docker run -d -p <port-on-host>:<port-inside-docker-container> <your-image-name>

Run image in foreground mode

docker run -i -t -p <port-on-host>:<port-inside-docker-container> <your-image-name>

Stop Docker container

Detached mode

docker stop <container-id>

You can get a list of all running Docker container and its ids by following command

docker images

Foreground mode

Go to console and press + C at any time.

Docker environment variables

There are several options to configure your app inside a Docker container

project .env file

You can use .env file in project root folder which will be copied inside Docker image. If you want to change a property inside .env you have to rebuild your Docker image.

run options

You can also change app configuration by passing environment variables via docker run option -e or --env.

docker run --env DB_HOST=localhost -e DB_PORT=3306

environment file

Last but not least you can pass a config file to docker run.

docker run --env-file ./env.list

env.list example:

# this is a comment
DB_TYPE=mysql
DB_HOST=localhost
DB_PORT=3306

divider

❯ Further Documentations

Name & Link Description
Express Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.
Microframework Microframework is a simple tool that allows you to execute your modules in a proper order, helping you to organize bootstrap code in your application.
TypeDI Dependency Injection for TypeScript.
routing-controllers Create structured, declarative and beautifully organized class-based controllers with heavy decorators usage in Express / Koa using TypeScript and Routing Controllers Framework.
TypeORM TypeORM is highly influenced by other ORMs, such as Hibernate, Doctrine and Entity Framework.
class-validator Validation made easy using TypeScript decorators.
class-transformer Proper decorator-based transformation / serialization / deserialization of plain javascript objects to class constructors
 event-dispatcher Dispatching and listening for application events in Typescript 
 Helmet Helmet helps you secure your Express apps by setting various HTTP headers. It’s not a silver bullet, but it can help! 
 Auth0 API Documentation Authentification service 
 Jest Delightful JavaScript Testing Library for unit and e2e tests 
 supertest Super-agent driven library for testing node.js HTTP servers using a fluent API 
 nock HTTP mocking and expectations library 
swagger Documentation   API Tool to describe and document your api.
SQLite Documentation  Getting Started with SQLite3 – Basic Commands.
GraphQL Documentation  A query language for your API.
DataLoader Documentation  DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.

divider

❯ Related Projects

divider

❯ License

MIT

About

data visualization of spotify discovery weekly data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published