From 93ab2342efebf2ef55f699aab42cf58e4b88e1c9 Mon Sep 17 00:00:00 2001 From: Jan Koehnlein Date: Thu, 1 Mar 2018 16:04:40 +0100 Subject: [PATCH] [docs] restored contributor docs, fixed links Signed-off-by: Jan Koehnlein --- README.md | 6 +- doc/Developing.md | 319 ++++++++++++++++++++++++++++++++++++++++++++++ doc/Publishing.md | 48 +++++++ doc/Testing.md | 52 ++++++++ 4 files changed, 422 insertions(+), 3 deletions(-) create mode 100644 doc/Developing.md create mode 100644 doc/Publishing.md create mode 100644 doc/Testing.md diff --git a/README.md b/README.md index 75ee211792ee9..e0d9644952fad 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ Theia is an extensible platform to develop full-fledged multi-language Cloud & D ## Getting Started Here you can find guides and examples for common scenarios: -- [Develop a new Theia extension](http://www.theia-ide.org/doc/Authoring_Extensions.md) +- [Develop a new Theia extension](http://www.theia-ide.org/doc/Authoring_Extensions.html) - Develop a new Theia application for [Cloud](examples/browser/package.json) or [Desktop](examples/electron/package.json) with [Theia CLI](dev-packages/cli/README.md) - [Run Theia IDE for Web Developers with Docker](https://github.com/theia-ide/theia-apps#theia-docker) - [Package a desktop Theia application with Electron](https://github.com/theia-ide/yangster-electron) @@ -39,8 +39,8 @@ Here you can find guides and examples for common scenarios: ## Contributing Read below to learn how to take part in improving Theia: -- Fork the repository and [run the examples from source](http://www.theia-ide.org/doc/Developing.md#quick-start) -- Get familiar with [the development workflow](http://www.theia-ide.org/doc/Developing.md), [Coding Guidelines](https://github.com/theia-ide/theia/wiki/Coding-Guidelines), [Code of Conduct](CODE_OF_CONDUCT.md) and [learn how to sign your work](CONTRIBUTING.md#sign-your-work) +- Fork the repository and [run the examples from source](doc/Developing.md#quick-start) +- Get familiar with [the development workflow](doc/Developing.md), [Coding Guidelines](https://github.com/theia-ide/theia/wiki/Coding-Guidelines), [Code of Conduct](CODE_OF_CONDUCT.md) and [learn how to sign your work](CONTRIBUTING.md#sign-your-work) - Find an issue to work on and submit a pull request - First time contributing to open source? Pick a [good first issue](https://github.com/theia-ide/theia/labels/good%20first%20issue) to get you familiar with GitHub contributing process. - First time contributing to Theia? Pick a [beginner friendly issue](https://github.com/theia-ide/theia/labels/beginners) to get you familiar with codebase and our contributing process. diff --git a/doc/Developing.md b/doc/Developing.md new file mode 100644 index 0000000000000..0e4e6df62c3bb --- /dev/null +++ b/doc/Developing.md @@ -0,0 +1,319 @@ +This file contains tips to help you take (and understand) your first steps in +the world of Theia development. Are you in a hurry? See the +[Quick Start](#quick-start). + +# How to build Theia and the example applications +Theia is a framework to build IDEs, so you can't really "run" Theia itself. +However, you can run the example applications included in its repository. One +is a browser-based IDE and the other is the Electron-based equivalent. + +The following instructions are for Linux and macOS. + +For Windows instructions [click here](#building-on-windows). + + - [**Prerequisites**](#prerequisites) + - [**Quick Start**](#quick-start) + - [Run with SSL](#run-the-browser-example-with-ssl) + - [**Clone the repository**](#clone-the-repository) + - [**The repository structure**](#the-repository-structure) + - [**Build core, extensions and examples packages**](#build-core-extensions-and-examples-packages) + - [**Build extension packages individually**](#build-extension-packages-individually) + - [**Run the browser-based example application**](#run-the-browser-based-example-application) + - [**Run the Electron-based example application**](#run-the-electron-based-example-application) + - [**Rebuilding**](#rebuilding) + - [**Watching**](#watching) + - [Watch the core and extension packages](#watch-the-core-and-extension-packages) + - [Watch the examples](#watch-the-examples) + - [Watch a specific package](#watch-a-specific-package) + - [**Debugging**](#debugging) + - [Debug the browser example's backend](#debug-the-browser-examples-backend) + - [Debug the browser example's frontend](#debug-the-browser-examples-frontend) + - [Debug the browser example's frontend and backend at the same time](#debug-the-browser-examples-frontend-and-backend-at-the-same-time) + - [Debug the Electron example's backend](#debug-the-electron-examples-backend) + - [Debug the Electron example's frontend](#debug-the-electron-examples-frontend) + - [Debug IPC servers](#debug-ipc-servers) + - [**Testing**](#testing) + - [**Code coverage**](#code-coverage) + - [**Building on Windows**](#building-on-windows) + - [**Troubleshooting**](#troubleshooting) + - [Linux](#linux) + - [Windows](#windows) + - [macOS](#macos) + - [Root privileges errors](#root-privileges-errors) + +## Prerequisites + - Node.js v8.0 or higher + - [Yarn package manager](https://yarnpkg.com/en/docs/install) v1.2.1 + - git (If you would like to use the Git-extension too, you will need to have git version 2.11.0 or higher.) + +[nvm](https://github.com/creationix/nvm) is recommended to easily switch between +Node.js versions. + +## Quick Start + +To build and run the browser example: + + git clone https://github.com/theia-ide/theia \ + && cd theia \ + && yarn \ + && cd examples/browser \ + && yarn run start + +Start your browser on http://localhost:3000. + +To build and run the Electron example: + + git clone https://github.com/theia-ide/theia \ + && cd theia \ + && yarn \ + && yarn run rebuild:electron \ + && cd examples/electron \ + && yarn run start + +### Run the browser example with SSL + +To run the browser example using SSL use: + + git clone https://github.com/theia-ide/theia \ + && cd theia \ + && yarn \ + && cd examples/browser \ + && yarn run start --ssl --cert /path/to/cert.crt --certkey /path/to/certkey.key + +Start your browser on https://localhost:3000. + +## Clone the repository + + git clone https://github.com/theia-ide/theia + +The directory containing the Theia repository will now be referred to as +`$THEIA`, so if you want to copy-paste the examples, you can set the `THEIA` +variable in your shell: + + THEIA=$PWD/theia + +## The repository structure + +Theia repository has multiple folders: + + - `packages` folder contains runtime packages, as the core package and extensions to it + - `dev-packages` folder contains devtime packages + - [@theia/cli](../dev-packages/cli/README.md) is a command line tool to manage Theia applications + - [@theia/ext-scripts](../dev-packages/ext-scripts/README.md) is a command line tool to share scripts between Theia runtime packages + - `examples` folder contains example applications, both Electron-based and browser-based + - `doc` folder provides documentation about how Theia works + - `scripts` folder contains JavaScript scripts used by npm scripts when +installing +- the root folder lists dev dependencies and wires everything together with [Lerna](https://lernajs.io/) + +## Build core, extensions and examples packages + +You can download dependencies and build it using: + + cd $THEIA + yarn + +This command downloads dev dependencies, links and builds all packages. +To learn more and understand precisely what's going on, please look at scripts in [package.json](../package.json). + +## Build extension packages individually + +From the root: + + `npx run build @theia/package-name` + +From the package: + +`yarn --ignore-scripts && yarn build` + +## Run the browser-based example application + +We can start the application from the [examples/browser](../examples/browser) directory with: + + yarn run start + +This command starts the backend application listening on port `3000`. The frontend application should be available on http://localhost:3000. + +If you rebuild native Node.js packages for Electron then rollback these changes +before starting the browser example by running from the root directory: + + yarn run rebuild:browser + +## Run the Electron-based example application + +From the root directory run: + + yarn run rebuild:electron + +This command rebuilds native Node.js packages against the version of Node.js +used by Electron. + +It can also be started from the [examples/electron](../examples/electron) directory with: + + yarn run start + +## Rebuilding + +In the root directory run: + + yarn run build + +## Watching + +### Watch the core and extension packages + +To rebuild each time a change is detected run: + + yarn run watch + +### Watch the examples + +To rebuild each time a change is detected in frontend or backend you can run: + + yarn run watch + +### Watch a specific package + +You can use `npx` to watch a single package: + + npx run watch @theia/the-package-name + +## Debugging + +### Debug the browser example's backend + + - In VS Code: start the debug tab and run the `Launch Backend` configuration. + +### Debug the browser example's frontend + + - Start the backend by using `yarn run start`. + - In a browser: Open http://localhost:3000/ and use the dev tools for debugging. + - In VS Code: start the debug tab and run the `Launch Frontend` configuration. + +### Debug the browser example's frontend and backend at the same time + + - In VS Code: Start the debug tab and run the `Launch Backend` configuration. + - Then run the `Launch Frontend` configuration. + +### Debug the Electron example's backend + + - In VS Code: Start the debug tab and run the `Launch Electron Backend` configuration. + +### Debug the Electron example's frontend + + - Start the backend by using `yarn run start`. + - In Electron: Help -> Toggle Electron Developer Tools. + +### Debug IPC servers + + - Pass `--${server-name}-inspect` arg to the backend server. + - For example `--nfsw-watcher-inspect=0` to inspect nfsw watcher processes with dynamic port allocation. + - All variations of `--inspect` flag are supported: https://nodejs.org/en/docs/inspector/#command-line-options. + - Attach the debugger to the logged port. + +In order to look up `server-name` run the backend server with `--logLevel=debug` flag to enable logging of IPC servers instantiation. +You should be able to see message of `[${server-name}: ${server-PID}]: IPC started` format, like `[nsfw-watcher: 37557] IPC started`. + +## Testing + +See the [testing](Testing.md) documentation. + +## Code coverage + + yarn run test + +By default, this will generate the code coverage for the tests in an HTML +format, which can be easily viewed with your browser (Chrome/Firefox/Edge/Safari +etc.) by opening `packages//coverage/index.html`. + +## Building on Windows + +Run cmd.exe as an administrator and install `choco` by copy-pasting the command +to your console: + + @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin" + +Install `yarn` via `choco`. The `yarn` installation ensures that you will have +Node.js and npm too: + + choco install yarn + +Install `git` via `choco` + + choco install git + +Install the correct version of `yarn` (The version is important) + + choco install yarn --version 1.2.1 -y + +Install Windows-Build-Tools. +Run PowerShell as an administrator and copy-paste the below command: + + npm --add-python-to-path install --global --production windows-build-tools + +Clone, build and run Theia. +Using Git Bash as administrator: + + git clone https://github.com/theia-ide/theia.git && cd theia && yarn && cd examples/browser && yarn run start + +## Troubleshooting + +You might see `error Could not open cafile: EISDIR: illegal operation on a directory` errors when running `yarn`. This is a known [issue in `yarn` version `1.2.1`](https://github.com/yarnpkg/yarn/issues/4695), and will be fixed in the next release. You can safely ignore it for now. + +### Linux + +The start command will start a watcher on many files in the theia directory. +To avoid ENOSPC errors, increase your default inotify watches. + +It can be done like so: + + echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p + +### Windows + +Theia uses native modules and also requires Python 2.x to be installed on the +system when building the application. + + - One can get all the [all-in-one packages] by running + `npm install --global windows-build-tools` script. + + If you are facing with `EPERM: operation not permitted` or `permission denied` +errors while building, testing or running the application then; + + - You don't have write access to the installation directory. + - Try to run your command line (`PowerShell`, `GitBash`, `Cygwin` or whatever + you are using) as an administrator. + - The permissions in the NPM cache might get corrupted. Please try to run + `npm cache clean` to fix them. + - If you experience issues such as `Error: EBUSY: resource busy or locked, rename`, + try to disable (or uninstall) your anti-malware software. + See [here](https://github.com/npm/npm/issues/13461#issuecomment-282556281). + - Still having issues on Windows? File a [bug]. We are working on Linux or OS X + operating systems. Hence we are more than happy to receive any Windows-related + feedbacks, bug reports. + +If you have accidentally installed the wrong `yarn` version, you have to remove it, then reinstall it. + + - Run PowerShell as an administrator. + - Run: choco uninstall yarn -y + - Run: choco install yarn --version 1.2.1 -y + +[all-in-one packages]: https://github.com/felixrieseberg/windows-build-tools +[bug]: https://github.com/theia-ide/theia/issues + +### macOS + +You need to have the Xcode command line tools installed in order to build and run Theia. You can install the tools by running + + xcode-select --install + +If you already have Xcode installed, but you see the `xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance` error, you need to run the following command to fix it: `sudo xcode-select --switch /Library/Developer/CommandLineTools`. + +### Root privileges errors +When trying to install with root privileges, you might encounter errors such as +`cannot run in wd`. + +Several options are available to you: + + - Install without root privileges + - Use the `--unsafe-perm` flag: `yarn --unsafe-perm` diff --git a/doc/Publishing.md b/doc/Publishing.md new file mode 100644 index 0000000000000..9e8473ad4bd7f --- /dev/null +++ b/doc/Publishing.md @@ -0,0 +1,48 @@ +# Publishing + +In order to release one should: + +- pre-publishing steps +- login to the npm registry +- publish packages +- tag the published version + +## Pre-Publishing Steps +Before publishing it's important to make sure that theia-apps builds against 'next'. Else we will have problems with "latest" after publishing + +- Make sure that there is no pending build on Theia master, otherwise a new "next" version might be published while we validate the current "next". + +- Go in the theia-apps repo [here](https://github.com/theia-ide/theia-apps/commits/master) and identify the latest commit. There should be an icon next to it; either a red X or a green checkmark. Click on it to go the Travis page. There re-trigger the build. We need to make sure that at least the various "next" builds pass. If it doesn't, it needs to be fixed before continuing. + + +## Login to the npm registry + +Follow this [instruction](https://docs.npmjs.com/cli/adduser) to login to the npm registry with a user account. + +If you don't have an account contact [Theia organization](https://www.npmjs.com/~theia) to request one. + +## Publishing packages + + yarn run publish + +This command will rebuild all packages, test them, publish to npm and bump versions. + +## Tagging the published version + + git add * + git commit -m "publish v${published.version}" -s + git push origin HEAD + git tag v${published.version} -s + git push origin v${published.version} + +The version picked during package publishing should be used as `${published.version}`. + +First two commands commit updated versions, last two create a new tag and transfer it to a remote. + +For example, if you picked `0.1.0` as a version then you should run: + + git add * + git commit -m "publish v0.1.0" -s + git push origin HEAD + git tag v0.1.0 -s + git push origin v0.1.0 \ No newline at end of file diff --git a/doc/Testing.md b/doc/Testing.md new file mode 100644 index 0000000000000..f5706b42988b4 --- /dev/null +++ b/doc/Testing.md @@ -0,0 +1,52 @@ +# Testing + +## Running tests + +To run tests on theia run: + +`yarn test` + +This will run all CI enabled tests. + +If you want to run all tests for a particular Theia extension, execute the following command from the root: + +`npx run test @theia/extension-name` + +Add the following npm script to the `package.json` of the desired Theia extension, if you would like to enable the watch mode for the tests. + +```json + "test:watch": "theiaext test:watch" +``` + +After editing the `package.json` you can run the tests in watch mode with: + +`npx run test:watch @theia/extension-name` + +## Test directory structure + +The test directory structure is as follows: + + - `src/node/foo.ts`: Code to be tested. + - `src/node/foo.spec.ts`: Unit tests for foo.ts. + - `src/node/test/test-helper.ts`: Any mocks, fixture or utility test code + goes here. + - `src/node/foo.slow-spec.ts`: Any slow running tests such as integration + tests should be labeled as such so that they can be excluded. + - `src/browser/foo.ui-spec.ts`: UI tests. + - `test-resources`: Any resources needed for the tests like configuration + files or scripts. + - `test-resources/ui`: Resources for UI testing. + - `test-resources/slow`: Resources for slow running tests. + +## Publishing + +### Published test files + +Unit tests named as `foo.spec.ts` will be published since they're also for +documentation purposes. + +### Unpublished + + - `*ui-spec.ts` + - `*slow-spec.ts` + - `test-resources`